Back office ops · Production

OpenAI Codex CLI agent loop: architecture, prompt caching, and context management

The problem

Building a production software agent loop requires managing ever-growing prompt length across many tool-call iterations, avoiding costly cache misses for inference efficiency, and preventing context window exhaustion during long conversations.

First attempt

An early MCP tools integration introduced a bug where tools were not enumerated in a consistent order, causing expensive prompt cache misses.

Workflow diagram · grounded in source
1
User input received
trigger
“To start, the agent takes input from the user to include in the set of textual instructions it prepares for the model known as a prompt.”
2
Prompt construction and API dispatch
integration
“Once Codex builds up the full JSON payload to send to the Responses API, it then makes the HTTP POST request with an Authorization header depending on how the Responses API endpoint is configured”
3
Model inference via Responses API
ai_action
“the textual prompt is first translated into a sequence of input tokens—integers that index into the model's vocabulary. These tokens are then used to sample the model, producing a new sequence of output tokens”
4
Tool call execution
ai_action
“the agent executes the tool call and appends its output to the original prompt. This output is used to generate a new input that's used to re-query the model”
5
Assistant message returned to user
output
“each turn always ends with an assistant message—such as "I added the architecture.md you asked for"—which signals a termination state in the agent loop”
6
Automatic context compaction
ai_action
“Our general strategy to avoid running out of context window is to compact the conversation once the number of tokens exceeds some threshold. Specifically, we replace the input with a new, smaller list of items that is representative of t…”
Reported outcome

Codex achieves efficient inference through prompt caching that makes sampling linear rather than quadratic, automatic context compaction via a dedicated endpoint, and stateless request design that supports Zero Data Retention customers without sacrificing reasoning continuity.

Reported metrics
Inference cost with prompt cachinglinear rather than quadratic
Reported stack
MCP serversLM Studio
Source
https://openai.com/index/unrolling-the-codex-agent-loop/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Codex achieves efficient inference through prompt caching that makes sampling linear rather than quadratic, automatic context compaction via a dedicated endpoint, and stateless request design that supports Zero Data R…

What tools did this team use?

MCP servers, LM Studio.

What results were reported?

Inference cost with prompt caching: linear rather than quadratic (source-reported, not independently verified).

What failed first in this deployment?

An early MCP tools integration introduced a bug where tools were not enumerated in a consistent order, causing expensive prompt cache misses.

How is this back office ops AI workflow structured?

User input received → Prompt construction and API dispatch → Model inference via Responses API → Tool call execution → Assistant message returned to user → Automatic context compaction.

Related back office ops cases