Back office ops · Production

Context Engineering for AI Agents: Lessons from Building Manus

The problem

Building a production AI agent system required solving interrelated context management challenges: KV-cache efficiency for cost and latency, exploding action spaces from tool proliferation, context window exhaustion on unstructured data, and agent goal drift over long multi-step tasks.

First attempt

An early approach of dynamically adding and removing tools mid-iteration using a RAG-like mechanism was abandoned because tool changes invalidated the KV-cache and confused the model when prior actions referenced tools no longer in the context.

Workflow diagram · grounded in source
1
User input received
trigger
“After receiving a user input, the agent proceeds through a chain of tool uses to complete the task.”
2
State machine constrains tools
routing
“Manus uses a context-aware state machine to manage tool availability. Rather than removing tools, it masks the token logits during decoding to prevent (or enforce) the selection of certain actions based on the current context.”
3
Model selects action
ai_action
“the model selects an action from a predefined action space based on the current context”
4
Action executed in sandbox
integration
“That action is then executed in the environment (e.g., Manus's virtual machine sandbox) to produce an observation.”
5
File system used as memory
integration
“we treat the file system as the ultimate context in Manus: unlimited in size, persistent by nature, and directly operable by the agent itself”
6
Errors retained in context
feedback_loop
“leave the wrong turns in the context. When the model sees a failed action—and the resulting observation or stack trace—it implicitly updates its internal beliefs.”
7
Todo list recited for focus
feedback_loop
“By constantly rewriting the todo list, Manus is reciting its objectives into the end of the context. This pushes the global plan into the model's recent attention span, avoiding 'lost-in-the-middle' issues and reducing goal misalignment.”
Reported outcome

Manus arrived at a stable set of context engineering principles enabling the agent loop to ship improvements in hours instead of weeks and operate at scale across millions of users.

Reported metrics
Average input-to-output token ratioaround 100:1
KV-cache cost savings factor10x difference
Average tool calls per taskaround 50 tool calls on average
Framework rebuildsfour times
Show all 6 reported metrics
average input-to-output token ratioaround 100:1
KV-cache cost savings factor10x difference
average tool calls per taskaround 50 tool calls on average
framework rebuildsfour times
improvement shipping speedhours instead of weeks
production scalemillions of users
Reported stack
vLLMClaude Sonnet
Source
https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Manus arrived at a stable set of context engineering principles enabling the agent loop to ship improvements in hours instead of weeks and operate at scale across millions of users.

What tools did this team use?

vLLM, Claude Sonnet.

What results were reported?

Average input-to-output token ratio: around 100:1; KV-cache cost savings factor: 10x difference; Average tool calls per task: around 50 tool calls on average; Framework rebuilds: four times (source-reported, not independently verified).

What failed first in this deployment?

An early approach of dynamically adding and removing tools mid-iteration using a RAG-like mechanism was abandoned because tool changes invalidated the KV-cache and confused the model when prior actions referenced tool…

How is this back office ops AI workflow structured?

User input received → State machine constrains tools → Model selects action → Action executed in sandbox → File system used as memory → Errors retained in context → Todo list recited for focus.

Related back office ops cases