The silent killer of multi-agent systems
You've built something useful. Several AI agents, each with a job: one retrieves data, another analyses it, a third drafts reports, a fourth manages approvals. They share a common context. It works beautifully in testing.
Then you put it into real use. And within a few days, things get strange.
The reporting agent starts referencing data from a completely different project. The approval agent routes a request to the wrong person. The analysis agent contradicts itself between the morning and afternoon runs.
Nobody changed the code. Nobody touched the prompts. The system rotted — from the inside.
This is context rot. And if you're running multi-agent workflows at any kind of scale, it's coming for you.
What context rot actually is
Context rot happens when agents accumulate stale, irrelevant, or conflicting information in their shared context, and that noise gradually overwhelms the signal.
Think of it like a whiteboard in a meeting room that never gets wiped. On Monday it holds the notes from one project. By Friday it has fragments from five different meetings, half-erased diagrams, and someone's lunch order. Now try to make a decision based on what's on that whiteboard.
That's what your agents are doing. Every interaction adds to the context. Nothing gets pruned. Relevance degrades. And because language models are designed to use all available context, they dutifully fold the garbage in alongside the gold.
The three memory layers that fix it
The fix isn't "give agents more memory" or "give agents less memory." It's structured memory with clear boundaries.
Layer 1: Session memory
This is the short-term conversational context. What the user just asked. What the agent just answered. The immediate back-and-forth.
Rule: session memory dies when the session ends. Nothing from a Monday conversation should be polluting Wednesday's context. This alone eliminates half of all context rot.
Layer 2: Task memory
This is the working memory for a specific workflow. If an agent is running a compliance audit, task memory holds the relevant standards, the documents under review, the findings so far, and the current step in the process.
Rule: task memory is scoped to the task and shared only with agents working on that task. The reporting agent for Project A should never see the task memory from Project B. When the task completes, the memory is archived — available for reference, but no longer active.
Layer 3: Retrieval memory
This is the long-term organisational knowledge: policies, procedures, historical data, reference documents. It's not loaded into context by default. It's retrieved on demand using techniques like vector search, keyword matching, or structured queries.
Rule: retrieval memory is pulled, never pushed. Agents request what they need for the current step. They don't swim in a lake of everything the organisation has ever written.
The orchestration pattern
With three memory layers, your multi-agent system looks different:
1. User makes a request → session memory captures the intent.
2. Orchestrator decomposes the task → task memory is initialised with only what's relevant.
3. Specialised agents execute their steps → each pulls from retrieval memory as needed, writes results to task memory.
4. Results are composed and delivered → session memory captures the output.
5. Session ends → session memory is cleared. Task memory is archived. Retrieval memory remains unchanged.
No context bleed between sessions. No task contamination between projects. No drowning in irrelevant organisational knowledge.
The warning signs you already have context rot
If any of these sound familiar, you've already got the problem:
- Agents give different answers to the same question at different times of day.
- Agents reference information the user never mentioned.
- Multi-step workflows produce inconsistent results between runs.
- Agent responses get slower over time as the context window fills up.
- Users report the system "forgetting" things they just told it, as new information gets pushed out by old noise.
The bottom line
Context rot isn't a bug you can fix with a prompt tweak. It's an architectural problem, and it needs an architectural solution: separate your memory into session, task, and retrieval layers with clear scoping, lifecycle, and access rules.
Build the boundaries before you build the agents. Because the agents will only ever be as reliable as the context they operate in.