Quality assurance · Production

DoorDash builds a multi-agent AI code reviewer with 60% engineer acceptance rate

The problem

Most AI code reviewers attached to pull requests get ignored because their comments are noisy and generic; the central challenge is helping an agent focus on the parts of a change that deserve review and stay quiet when it has nothing useful to add.

First attempt

A previous third-party review agent achieved only a 46% acceptance rate. DoorDash's own v1 specialist agents missed architectural issues, and v2 general-purpose reviewers spread attention too thin, losing real findings.

Workflow diagram · grounded in source
1
PR opened, agent triggered
trigger
“Many reviews fire automatically when a PR is opened; engineers don't have to ask for one.”
2
Route domain review profiles
routing
“When a PR touches the payment-service-provider gateway, the agent loads the PSP rules, the payment core rules, and the monetary-security rules, and nothing else.”
3
Lead scout identifies suspects
ai_action
“The lead scout's job isn't to verify anything. It just reads the diff and notices things that feel off: "this deletion looks suspicious," "this enum case isn't handled in the sibling file," "this error path is silently swallowing failure…”
4
Deep reviewers verify leads
ai_action
“The two deep reviewers then take those leads and dig in, verifying the ones that hold up and dropping the ones that don't.”
5
Disprove-it validation pass
validation
“Before any comment is posted, it has to survive a disprove-it pass, an explicit step where the system tries to falsify its own finding. If the claim can't hold up to scrutiny, it gets dropped.”
6
Post anchored PR findings
output
“The comments it does post are anchored to specific lines with quoted evidence”
7
Fix agent resolves findings on request
ai_action
“The fixer runs in a remote VM with a full checkout of the repository and the original review context: the PR diff, the finding, the surrounding code, and the suggested direction. It makes the change there and pushes it back to the PR”
Reported outcome

The agent reviews more than 10,000 pull requests per week across 56 repositories, with 60.2% of high and critical findings resulting in engineer code changes before merge, at roughly $3 per review compared to publicly priced comparable products at $5 to $20 per review.

Reported metrics
Acceptance rate for high/critical findings60.2%
Previous third-party agent acceptance rate46%
Webhook-triggered findings acceptance rate59.0%
Average cost per reviewroughly $3
Show all 8 reported metrics
acceptance rate for high/critical findings60.2%
previous third-party agent acceptance rate46%
webhook-triggered findings acceptance rate59.0%
average cost per reviewroughly $3
comparable product price range per review$5 to $20 per review
PRs reviewed per weekmore than 10,000
onboarded repositories56
time to post findings after PR opensabout 7 minutes
Reported stack
GitHubSlack
Source
https://careersatdoordash.com/blog/doordash-built-an-ai-code-reviewer-engineers-actually-listen-to/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The agent reviews more than 10,000 pull requests per week across 56 repositories, with 60.2% of high and critical findings resulting in engineer code changes before merge, at roughly $3 per review compared to publicly…

What tools did this team use?

GitHub, Slack.

What results were reported?

Acceptance rate for high/critical findings: 60.2%; Previous third-party agent acceptance rate: 46%; Webhook-triggered findings acceptance rate: 59.0%; Average cost per review: roughly $3 (source-reported, not independently verified).

What failed first in this deployment?

A previous third-party review agent achieved only a 46% acceptance rate.

How is this quality assurance AI workflow structured?

PR opened, agent triggered → Route domain review profiles → Lead scout identifies suspects → Deep reviewers verify leads → Disprove-it validation pass → Post anchored PR findings → Fix agent resolves findings on request.

Related quality assurance cases