Context engineering,
step by step.

1. The window is finite, and the loop keeps filling it

A model sees only its context window. Every loop iteration adds messages. The math does not end well on its own.

Context is a budget, not a free buffer. The loop spends it automatically; context engineering is how you spend it well.

2. What counts as context

Five different things compete for the same window. Each is managed differently.

Most of this essay lives in the middle rows, compression and isolation, because that is where the agent loop's runaway history gets tamed. But the cheapest token saved is the one in the system prompt you never wrote.

3. Short-term and long-term memory

Two different problems. One is "remember this conversation." The other is "remember this user."

The code

from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain.messages import HumanMessage
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.store.memory import InMemoryStore

agent = create_agent(
    model=init_chat_model("openai:gpt-4o"),
    tools=[],
    prompt="You remember our conversation.",
    checkpointer=InMemorySaver(),   # short-term: per-thread history
    store=InMemoryStore(),          # long-term: cross-session facts
)

# thread_id links messages into one conversation
config = {"configurable": {"thread_id": "user-123"}}
agent.invoke({"messages": [HumanMessage(content="My name is Alice.")]}, config=config)
result = agent.invoke({"messages": [HumanMessage(content="What is my name?")]}, config=config)
print(result["messages"][-1].content)   # "Your name is Alice."

The checkpointer gives continuity within a session; the store gives continuity across sessions. Neither solves the window problem, which is what happens when a single session's history gets too long.

4. Three ways to keep the window lean

When the history grows too long, you trim it, or you summarize it. They are not the same trade.

Trimming, by count and by tokens

from langchain.messages import SystemMessage

def trim_by_count(messages, max_messages=20):
    """Keep the system message plus the last N messages."""
    system = [m for m in messages if isinstance(m, SystemMessage)]
    other  = [m for m in messages if not isinstance(m, SystemMessage)]
    if len(other) <= max_messages:
        return messages
    return system + other[-max_messages:]

def trim_by_tokens(messages, max_tokens=8000):
    """Walk backward, keep messages until the token budget runs out."""
    system = [m for m in messages if isinstance(m, SystemMessage)]
    other  = [m for m in messages if not isinstance(m, SystemMessage)]
    kept, total = [], sum(len(m.content) // 4 for m in system)  # rough estimate
    for msg in reversed(other):
        cost = len(msg.content) // 4
        if total + cost > max_tokens:
            break
        kept.insert(0, msg)
        total += cost
    return system + kept

Trimming is fast and simple, and it is blind. It cannot tell a throwaway "ok, thanks" from the one message that decided your whole architecture.

5. Live: a filling context window

A real conversation, a tight window, and four policies. Watch what survives, and what is lost.

Pick a policy

Start on "keep everything" to see the overflow, then trim, then summarize. The slider sets how many recent messages to keep (used by trim-by-count and summarize).

Trimming keeps the window legal but can delete the one message that mattered. Summarization keeps the window legal and carries the key decision forward. That difference is the whole point of context engineering.

6. Summarization, in depth

Compress the old, keep the recent verbatim, and use a cheap model to do it.

What summarization does to the window

A 200k window at 85% full, before and after. The full history is replaced by a compact summary; the recent messages survive intact.

Dark = system prompt. Faded = full history. Rose = the summary that replaces it. Rust = recent messages, kept verbatim.

The code

By hand, then with the built-in middleware that does it for you.

from langchain.chat_models import init_chat_model
from langchain.messages import HumanMessage, SystemMessage

summarizer = init_chat_model("openai:gpt-4o-mini")   # cheap model for summaries

def summarize_old(messages, keep_recent=10):
    system = [m for m in messages if isinstance(m, SystemMessage)]
    other  = [m for m in messages if not isinstance(m, SystemMessage)]
    if len(other) <= keep_recent:
        return messages
    old, recent = other[:-keep_recent], other[-keep_recent:]
    text = "\n".join(f"{type(m).__name__}: {m.content}" for m in old)
    summary = summarizer.invoke([
        SystemMessage(content="Summarize concisely. Preserve key facts and decisions."),
        HumanMessage(content=text),
    ])
    return system + [SystemMessage(content=f"[Summary]: {summary.content}")] + recent

from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="openai:gpt-4o",
    tools=[],
    middleware=[
        SummarizationMiddleware(
            model="openai:gpt-4o-mini",   # cheaper model for summaries
            trigger=("tokens", 4000),      # summarize when history passes 4000 tokens
            keep=("messages", 20),         # always keep the 20 most recent
        )
    ],
    checkpointer=InMemorySaver(),
)

Summarization is trimming with a memory. It costs one cheap model call to avoid deleting the fact you will need three turns from now.

7. Offloading: externalize instead of compress

Some content does not belong in the window at all. Put it on disk and fetch what you need.

If the agent might need the detail later but not now, offload it. Summarize what must stay legible; relocate what only needs to be reachable.

8. Isolation: give heavy work its own window

The most effective technique is to keep the mess out of the main window entirely.

The difference, in tokens

The code

from deepagents import create_deep_agent

agent = create_deep_agent(
    model="anthropic:claude-sonnet-4-6",
    subagents=[
        {
            "name": "researcher",
            "description": "Conducts research on a topic",
            "system_prompt": (
                "You are a research assistant. "
                "IMPORTANT: return only the essential summary (under 500 words). "
                "Do NOT include raw search results or detailed tool outputs."
            ),
            "tools": [web_search],
        }
    ],
)

The cheapest token to manage is the one that never enters the main window. Delegate heavy work to a subagent and let its context be discarded when it finishes.

9. Wiring it into the loop

Run your trimming before every model call with a before-model hook.

The code

from langchain.agents import create_agent, AgentState
from langchain.agents.middleware import before_model
from langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.runtime import Runtime

@before_model
def trim(state: AgentState, runtime: Runtime):
    """Keep the first message and the last few; drop the rest."""
    messages = state["messages"]
    if len(messages) <= 3:
        return None                       # nothing to do
    keep = [messages[0]] + messages[-3:]
    return {"messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES), *keep]}

agent = create_agent(
    "openai:gpt-4o",
    tools=[],
    middleware=[trim],
    checkpointer=InMemorySaver(),
)

Context management is not a one-time setup. It is a hook that runs on every turn of the loop, keeping the window lean before the model ever sees it.

10. Which technique, when

Five levers. Each fits a different shape of problem.

In order of preference for most agents: never load it (system prompt discipline), isolate it (subagents), offload it (disk), summarize it (cheap model), and only then trim it (blind deletion) as the last resort.

11. What we left out

Real machinery, deferred to keep this essay about the core levers.

An agent is a loop that calls tools and spends context. Engineer the tools, engineer the loop, engineer the context, and you have engineered the agent.