Reasoning models

How Reasoning Models Work

To understand the shift, think of a standard LLM as a brilliant improvisational speaker. When you ask a question, they begin talking immediately, relying on their training to predict the most likely next word. This is incredibly fast and effective for drafting emails or summarizing text, but it can struggle when a problem requires careful planning or rigorous logic.

A reasoning model, by contrast, acts more like a mathematician working on a scratchpad. Before it gives you an answer, it engages in an internal ‘thinking’ process. It decomposes the prompt, explores different paths to a solution, and checks its own work. If it realizes a specific line of reasoning is leading to an error, it can ‘backtrack’—essentially erasing its mental scratchpad and trying a different approach.

This internal deliberation is distinct from ‘chain-of-thought prompting,’ where a user asks a standard model to ‘show its work’ in the final output. In reasoning models, this chain-of-thought is built into the model’s architecture and training, often occurring in a hidden, internal space that doesn’t clutter the final answer unless requested.

The Landscape of Reasoning Models

The field has evolved rapidly, with major AI providers integrating these capabilities into their core offerings:

• OpenAI: The o-series (such as o3 and o3-mini) represents a primary example, trained specifically to utilize chain-of-thought traces. Developers can control deliberation via the reasoning_effort parameter (low/medium/high) [1].
• Anthropic: Claude models have introduced ‘extended thinking’ capabilities with an effort parameter offering 5 levels (low, medium, high, xhigh, max) [2].
• Google: Gemini models feature ‘Deep Think’ mode with a thinking_level parameter, utilizing parallel hypothesis exploration [3].
• DeepSeek: Models like DeepSeek-R1 have popularized open-source reasoning, demonstrating that native thinking modes can be implemented efficiently at low cost [4].

Practical Routing: When to Use Reasoning Models

Because reasoning models perform more computation, they often come with higher latency compared to standard models. The most effective strategy is ‘routing’:

• Use standard LLMs for tasks that prioritize speed and fluency: drafting emails, creative writing, simple Q&A, basic summarization.
• Route to reasoning models for tasks that demand high-fidelity logic: complex math, code debugging, multi-step planning, and nuanced tradeoff analysis.

Deliberation budget controls (like reasoning_effort, effort, and thinking_level) allow you to tune the balance between latency and accuracy based on your specific use case.

Why They Matter for Agentic AI

Reasoning models are the engine behind modern agentic AI workflows. Because they can plan, backtrack, and verify their own logic, they enable autonomous systems to move beyond simple chat interactions. Whether an agent is navigating a software environment, managing a multi-step project, or coordinating between different tools, the ability to ‘think before acting’ is what allows these systems to function reliably.