I ran a fast experiment investigating how DeepSeek-R1 carries out on agentic jobs, in spite of not supporting tool use natively, and I was quite satisfied by preliminary results. This experiment runs DeepSeek-R1 in a single-agent setup, where the design not only prepares the actions however likewise formulates the actions as executable Python code. On a subset1 of the GAIA recognition split, forum.altaycoins.com DeepSeek-R1 exceeds Claude 3.5 Sonnet by 12.5% absolute, from 53.1% to 65.6% appropriate, and other designs by an even bigger margin:

The experiment followed design usage guidelines from the DeepSeek-R1 paper and the model card: Don't utilize few-shot examples, prevent including a system prompt, and set the temperature to 0.5 - 0.7 (0.6 was used). You can find additional examination details here.

Approach

DeepSeek-R1's strong coding capabilities allow it to act as a representative without being clearly trained for tool use. By enabling the model to generate actions as Python code, it can flexibly interact with environments through code execution.

Tools are executed as Python code that is included straight in the timely. This can be an easy function definition or a module of a bigger bundle - any valid Python code. The design then produces code actions that call these tools.

Arise from carrying out these actions feed back to the model as follow-up messages, driving the next steps until a final response is reached. The agent structure is an easy iterative coding loop that mediates the discussion in between the model and its environment.

Conversations

DeepSeek-R1 is utilized as chat design in my experiment, where the model autonomously pulls additional context from its environment by utilizing tools e.g. by utilizing an online search engine or bring information from websites. This drives the conversation with the environment that continues until a last answer is reached.

In contrast, disgaeawiki.info o1 models are known to perform badly when used as chat designs i.e. they don't try to pull context throughout a conversation. According to the linked article, o1 designs perform best when they have the full context available, with clear guidelines on what to do with it.

Initially, I also attempted a full context in a single timely technique at each action (with results from previous actions included), but this caused significantly lower scores on the GAIA subset. Switching to the conversational technique explained above, bytes-the-dust.com I was able to reach the reported 65.6% efficiency.

This raises an interesting concern about the claim that o1 isn't a chat design - maybe this observation was more relevant to older o1 designs that lacked tool use capabilities? After all, wavedream.wiki isn't tool usage support an important mechanism for making it possible for models to pull extra context from their environment? This conversational technique certainly seems for DeepSeek-R1, forum.kepri.bawaslu.go.id though I still require to conduct similar experiments with o1 designs.

Generalization

Although DeepSeek-R1 was mainly trained with RL on mathematics and coding jobs, it is exceptional that generalization to agentic jobs with tool use through code actions works so well. This ability to generalize to agentic jobs reminds of current research study by DeepMind that shows that RL generalizes whereas SFT remembers, although generalization to tool usage wasn't examined in that work.

Despite its capability to generalize to tool usage, DeepSeek-R1 frequently produces really long reasoning traces at each action, compared to other designs in my experiments, limiting the effectiveness of this model in a single-agent setup. Even easier jobs in some cases take a long period of time to complete. Further RL on agentic tool usage, be it through code actions or not, might be one option to enhance effectiveness.

Underthinking

I also observed the underthinking phenomon with DeepSeek-R1. This is when a reasoning model frequently changes in between different thinking ideas without adequately checking out appealing paths to reach a correct option. This was a major reason for extremely long reasoning traces produced by DeepSeek-R1. This can be seen in the recorded traces that are available for download.

Future experiments

Another typical application of reasoning models is to utilize them for planning just, while utilizing other designs for generating code actions. This could be a possible brand-new function of freeact, if this separation of functions shows useful for larsaluarna.se more complex tasks.

I'm likewise curious about how reasoning designs that already support tool usage (like o1, o3, ...) perform in a single-agent setup, with and without generating code actions. Recent advancements like OpenAI's Deep Research or Hugging Face's open-source Deep Research, which also utilizes code actions, look intriguing.