Context Engineering

Why context engineering exists

Large language models do not reason from first principles — they predict the next token based on everything in their context window. The quality of that context determines the quality of the output. As models moved from single-turn chatbots to long-running agents with tools, memory, and multi-step goals, the challenge shifted from “write a good prompt” to “curate the optimal set of information for every step of an agent’s trajectory.”

Anthropic describes this shift as driven by “context rot” — the phenomenon where, as the number of tokens in the context window grows, the model’s ability to accurately recall and use information degrades. Context must therefore be treated as a finite resource with diminishing marginal returns, requiring active curation rather than just well-written instructions.

Prompt engineering vs. context engineering

Prompt engineering focuses on how to write effective instructions — the text that tells the model what to do. Context engineering encompasses prompt engineering but extends to everything else the model sees: tool descriptions, retrieved documents, conversation history, memory entries, structured outputs from previous steps, and global state.

The distinction matters because an agent’s performance depends on far more than its system prompt. A perfectly crafted prompt paired with irrelevant retrieved documents, poorly described tools, or a bloated conversation history will still produce poor results. Context engineering treats the entire information environment as a design problem.

Components of agent context

The information an agent receives at each step typically includes:

• System instructions: The agent’s role, constraints, and behavioral guidelines
• Tool definitions: Descriptions of available functions and APIs the agent can call
• Retrieved knowledge: Documents or data pulled via RAG (Retrieval-Augmented Generation)
• Conversation history: Previous turns in the current interaction
• Memory state: Persistent facts, user preferences, and past decisions stored across sessions
• Tool outputs: Results from previous tool calls in the current trajectory
• Structured outputs: Compact, schema-constrained data from earlier steps
• Global state: Shared information in multi-agent systems

Each component consumes tokens in the finite context window. The engineering challenge is deciding what to include, what to summarize, what to retrieve on demand, and what to drop.

Core strategies

Context engineering uses four main strategies to manage the information environment:

Write context: Persist information outside the context window using scratchpads, notes, and long-term memory stores (vector databases, key-value stores). Update memory incrementally as the agent runs — not just at the end.

Select context: Use semantic search and RAG to pull only relevant information for the current step. Retrieve tool descriptions on demand rather than loading all available tools. Order and rank retrieved chunks by relevance before injection.

Compress context: Summarize older turns or long tool outputs to free token budget. Trim low-signal content. Use structured outputs to reduce verbosity.

Isolate context: Split context across sub-agents so each operates with a focused window. Use runtime state objects to scope data to specific fields or steps. Break complex tasks into smaller workflows, each with its own optimized context.

Agent relevance

Context engineering is particularly critical for agentic systems because:

• Tool selection: Agents must choose which tools to call from a potentially large set. Research shows that retrieving relevant tool descriptions on demand (rather than loading all tools) improves selection accuracy significantly.
• Multi-step trajectories: Long-running agents accumulate context across many steps. Without active management, the context window fills with stale or irrelevant information, degrading performance.
• Multi-agent coordination: When multiple agents collaborate, each needs a focused context that includes relevant shared state but excludes noise from other agents’ work.
• Memory and learning: Agents that learn from past interactions must decide what to remember, what to forget, and how to retrieve relevant memories at the right time.

The agentic economy’s infrastructure — MCP for tool integration, A2A for agent-to-agent communication, and persistent memory systems — are all, fundamentally, context engineering problems.

Limitations

Context engineering does not solve fundamental model limitations. A model with a perfect context window can still hallucinate, misreason, or fail at tasks beyond its capabilities. The discipline optimizes the information environment, not the model itself.

Additional challenges:

• Evaluation difficulty: Measuring context quality is harder than measuring prompt quality — there is no standard benchmark for ‘good context engineering’
• Rapidly evolving practices: Techniques change as models improve and new architectures emerge
• Vendor-specific implementations: Different frameworks (LangChain, LlamaIndex, Anthropic’s agent SDK) implement context management differently, making cross-platform best practices harder to establish
• Token cost: More context means higher inference costs, creating a tension between completeness and efficiency