Quality assurance · Production

OpenAI Frontier team builds >1M LOC internal product with zero human-written code using Codex agents

The problem

The team needed to develop and ship an enterprise-grade product at speed, but humans were fundamentally the bottleneck: agent code output vastly outpaced the team's capacity for synchronous review, and early Codex models produced code too slow and insufficiently modular to assemble into working software.

First attempt

Early review agents bullied the code-authoring agent into accepting every comment, causing thrashing and non-convergence; build times grew beyond what agents could iterate on effectively; and early Codex models could not assemble complex features from their constituent pieces.

Workflow diagram · grounded in source
1
Codex spawned as entry point
trigger
“we spawn the coding agent, like that's the entry point. It's just Codex. And then we give Codex via skills and scripts the ability to boot the stack if it chooses to”
2
Agent writes code and opens PR
ai_action
“it would be Codex, CLI. Locally writes the change, pushes up a PR”
3
Review agent fires on PR sync
ai_action
“on those PR synchronizations of review agent fires. It posts a comment. We instruct Codex that it has to at least acknowledge and respond to that feedback”
4
Priority-gated review convergence
validation
“The reviewer agents were instructed to bias toward merging the thing to not surface anything greater than a P two in priority”
5
Autonomous merge
output
“Most of the human review is post merge at this point”
6
Quality score and backlog update
feedback_loop
“a tiny little scaffold, like a markdown table, which is a hook for Codex to review all the business logic that we have defined in the app, assess how it matches all these documented guardrails and propose follow up work for itself”
7
Human smoke test before release
human_review
“We require a blessed human approved smoke test of the app before we promote it to distribution”
Reported outcome

Over five months, a team of three built a codebase exceeding one million lines through Codex agents, generating around 1,500 PRs with zero lines of human-written code, and achieved autonomous merging with only a post-merge human smoke test required before release.

Reported metrics
Lines of code in codebasea million lines
Human-written lines of codezero
Pull requests generated1500
Project durationfive months
Show all 8 reported metrics
lines of code in codebasea million lines
human-written lines of codezero
pull requests generated1500
project durationfive months
initial velocity vs human baseline10 times slower
inner build loop targetunder a minute
daily token usage threshold>1B tokens a day
estimated daily token cost$2-3k/day
Reported stack
Codex CLISymphonyGrafanaSlackVictoria StackElixirturbonx
Source
https://www.latent.space/p/harness-eng
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Over five months, a team of three built a codebase exceeding one million lines through Codex agents, generating around 1,500 PRs with zero lines of human-written code, and achieved autonomous merging with only a post-…

What tools did this team use?

Codex CLI, Symphony, Grafana, Slack, Victoria Stack, Elixir, turbo, nx.

What results were reported?

Lines of code in codebase: a million lines; Human-written lines of code: zero; Pull requests generated: 1500; Project duration: five months (source-reported, not independently verified).

What failed first in this deployment?

Early review agents bullied the code-authoring agent into accepting every comment, causing thrashing and non-convergence; build times grew beyond what agents could iterate on effectively; and early Codex models could…

How is this quality assurance AI workflow structured?

Codex spawned as entry point → Agent writes code and opens PR → Review agent fires on PR sync → Priority-gated review convergence → Autonomous merge → Quality score and backlog update → Human smoke test before release.

Related quality assurance cases