Context Budgeting & Memory Degradation

Context rot is a product constraint

A large context window is capacity, not guaranteed attention. As irrelevant or weakly related material grows, the model can become less effective long before it reaches the advertised limit.

Two research patterns explain the failure. Lost in the Middle found a U-shaped effect: relevant information at the beginning or end of long input was retrieved more reliably than information buried in the middle. Context Rot testing across 18 frontier models found performance falls as input grows even when task difficulty stays constant, especially when relevant content competes with tool output, partial retrievals, and distractors. Capacity is therefore the wrong release metric: a 1M-token window can degrade materially around 50K active tokens.

Imagine the refund instruction is correct but buried between a 200-message history, fifteen skill bodies, several large tool responses, and unrelated return policies. Nothing exceeds the context-window limit, yet the model may miss the approval gate or choose the wrong market rule. That is context rot: declining effective attention before hard overflow. Context engineering prevents it by routing, retrieving, compressing, and releasing information deliberately; a larger window alone does not.

For PMs, context is a budget with a user-facing cost. Unnecessary instructions can increase latency, price, inconsistency, and correction turns. More documentation is not automatically safer if the agent cannot locate and apply the relevant part.

Progressive disclosure keeps a small amount of routing metadata available while storing most procedures and references outside the active prompt.

The figure contrasts one large prompt with a skill library that loads only the active body. The product opportunity is not simply lower token use. It is more available capability without requiring the model to attend to every workflow on every request.

Put token economics in the capability plan. In the source example, 50 workflows in one system prompt cost 15,000 active tokens every turn. As skills, about 4,000 tokens of descriptions + one 2,000-token active body = 6,000 active tokens, while 49 bodies stay on disk. A published conversion example reduced active context from roughly 150,000 to 2,000 tokens, over 98%. These are reference examples, not promises; the required artifact is a measured budget for metadata, active bodies, tool schemas, history, retrieved material, and reserved response headroom in your runtime.

Use these decomposition signals:

• A different domain team owns the procedure.
• The workflow has a distinct trigger and output artifact.
• It uses a specific tool or permission boundary.
• Its detailed examples are irrelevant to most requests.
• The team can write a focused evaluation set for it.

Move examples, long policies, templates, and source material into referenced files. Keep the main procedure short enough that a reviewer can understand the control flow and exceptions in one sitting.