Quality assurance · Production

Cloudflare builds CI-native multi-agent AI code review system across 48,095 merge requests

The problem

Code review was reliably bottlenecking Cloudflare's engineering teams, with a median wait time for a first review measured in hours. Off-the-shelf AI code review tools lacked the flexibility and customisation required at Cloudflare's scale, and a naive approach of stuffing diffs into a large language model produced a flood of vague, noisy output.

First attempt

Commercial AI code review tools were insufficiently configurable for a large engineering organization. A naive single-prompt LLM approach of grabbing a git diff and asking a model to find bugs produced a flood of vague suggestions, hallucinated syntax errors, and redundant advice.

Workflow diagram · grounded in source
1
Merge request opens review
Trigger
When an engineer at Cloudflare opens a merge request, it gets an initial pass from a coordinated set of AI agents.
▾ source quote
“when an engineer at Cloudflare opens a merge request, it gets an initial pass from a coordinated smörgåsbord of AI agents”
2
MR classified by risk tier
Validation
Every MR is classified into one of three risk tiers based on diff size and the presence of security-sensitive files.
▾ source quote
“The system classifies every MR into one of three risk tiers based on the size and nature of the diff”
3
Diff noise filtered
Validation
Before agents see any code, the diff is filtered to strip lock files, vendored dependencies, minified assets, and source maps.
▾ source quote
“Before the agents see any code, the diff goes through a filtering pipeline that strips out noise like lock files, vendored dependencies, minified assets, and source maps”
4
Specialized reviewers launched
Ai action
The coordinator launches up to seven specialized reviewer agents covering security, performance, code quality, documentation, release management, and compliance.
▾ source quote
“we launch up to seven specialised reviewers covering security, performance, code quality, documentation, release management, and compliance with our internal Engineering Codex”
5
Sub-reviewers analyze code
Ai action
Each sub-reviewer runs in its own OpenCode session with its own agent prompt and returns findings as structured XML.
▾ source quote
“Each sub-reviewer runs in its own OpenCode session with its own agent prompt. The coordinator doesn't see or control what tools the sub-reviewers use. They are free to read source files, run grep, or search the codebase as they see …”
6
Coordinator consolidates findings
Ai action
The coordinator deduplicates findings, re-categorizes issues, and filters out speculative and false-positive items.
▾ source quote
“Deduplication: If the same issue is flagged by both the security reviewer and the code quality reviewer, it gets kept once in the section where it fits best. Re-categorisation: A performance issue flagged by the code quality reviewer gets moved …”
7
Approval decision routing
Routing
The overall approval decision follows a strict rubric mapping finding severity to a GitLab action.
▾ source quote
“The overall approval decision follows a strict rubric”
8
Structured review comment posted
Output
The coordinator posts a single structured review comment to the merge request.
▾ source quote
“posts a single structured review comment”
9
Incremental re-review on new commits
Feedback loop
When a developer pushes new commits to an already-reviewed MR, the system runs an incremental re-review that is aware of its own previous findings.
▾ source quote
“When a developer pushes new commits to an already-reviewed MR, the system runs an incremental re-review that is aware of its own previous findings”
Reported outcome

In its first month the system completed 131,246 review runs across 48,095 merge requests in 5,169 repositories, with a median review time of 3 minutes and 39 seconds, an average cost of $1.19, an 85.7% prompt cache hit rate, and engineers needing to break glass on only 0.6% of merge requests.

Reported metrics
Review runs in first 30 days131,246
Merge requests reviewed48,095
Repositories covered5,169
Average reviews per merge request2.7
Show all 22 reported metrics
review runs in first 30 days131,246
merge requests reviewed48,095
repositories covered5,169
average reviews per merge request2.7
median review duration3 minutes and 39 seconds
P90 review duration6m 27s
P95 review duration7m 29s
P99 review duration10m 21s
break glass incidents288
break glass rate0.6%
average review cost$1.19
median review cost$0.98
P90 review cost$2.36
P95 review cost$2.93
P99 review cost$4.45
total findings produced159,103
average findings per review1.2
total tokens processedapproximately 120 billion tokens in total
prompt cache hit rate85.7%
estimated cache cost savingsestimated five figures
pre-system review wait timeoften measured in hours
upstream OpenCode pull requests contributedover 45
Reported stack
OpenCodeClaude Opus 4.7GPT-5.4Claude Sonnet 4.6GPT-5.3 CodexKimi K2.5GitLabCloudflare WorkerWorkers KVBunPrometheusWorkers LoggingVault
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://blog.cloudflare.com/ai-code-review/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

In its first month the system completed 131,246 review runs across 48,095 merge requests in 5,169 repositories, with a median review time of 3 minutes and 39 seconds, an average cost of $1.19, an 85.7% prompt cache hi…

What tools did this team use?

OpenCode, Claude Opus 4.7, GPT-5.4, Claude Sonnet 4.6, GPT-5.3 Codex, Kimi K2.5, GitLab, Cloudflare Worker, Workers KV, Bun.

What results were reported?

Review runs in first 30 days: 131,246; Merge requests reviewed: 48,095; Repositories covered: 5,169; Average reviews per merge request: 2.7 (source-reported, not independently verified).

What failed first in this deployment?

Commercial AI code review tools were insufficiently configurable for a large engineering organization.

How is this quality assurance AI workflow structured?

Merge request opens review → MR classified by risk tier → Diff noise filtered → Specialized reviewers launched → Sub-reviewers analyze code → Coordinator consolidates findings → Approval decision routing → Structured review comment posted → Incremental re-review on new commits.

Related quality assurance cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Quality assurance cases →