Context Engineering

What is context engineering?

Context engineering is curating everything the model sees: system instructions, tool definitions, retrieved documents, conversation history, and memory. The term displaced “prompt engineering” around 2025–26 because wording tweaks stopped being the bottleneck — what information fills the context window is what determines output quality, especially for agents running long tasks.

The core insight: attention is a budget

More context is not better context. Models attend unevenly across long inputs, and irrelevant material measurably degrades answers (“context rot”). The goal is the smallest set of high-signal tokens that lets the model do the job — an optimization problem, not a stuffing problem.

The practitioner’s toolkit

• Selection — RAG or just-in-time tool retrieval brings in only what’s relevant now.
• Compaction — summarize old conversation turns; keep decisions, drop transcripts.
• Structured memory — facts and progress written to external notes/files the agent re-reads, instead of an ever-growing history.
• Isolation — sub-agents get clean windows for subtasks and return only conclusions.
• Ordering — stable content first to preserve the KV cache; instructions where the model attends best.

Cost reality

Context engineering is simultaneously a quality and cost discipline: every token in the window is billed on every call of a conversation or agent loop. Teams that audit their context composition typically find 30–70% is redundant history or boilerplate — trimming it cuts spend and improves accuracy, the rare free lunch.

What people get wrong

• Solving quality problems by adding more context. Diagnose what’s missing or distracting instead.
• Letting agent history grow unboundedly until quality collapses mid-task; compaction must be designed in from the start.
• Treating the system prompt as a junk drawer. Every edge-case instruction added “just in case” competes for attention with the ones that matter.

Context Engineering vs. Prompt Engineering

Context engineering is often distinguished from prompt engineering by scope: prompt engineering typically refers to crafting the wording and structure of a single instruction or template, while context engineering encompasses the broader system-level decisions about what information — retrieved documents, conversation history, tool outputs, memory — gets assembled into the context window at all, and in what order and format. As LLM applications have grown more complex, involving retrieval, tool use, and multi-turn memory simultaneously, context engineering has emerged as its own discipline distinct from prompt wording alone, since even a perfectly worded prompt performs poorly if the surrounding context is poorly curated, contains irrelevant information, or omits something the model actually needed to answer correctly.