Quality assurance · Production

Multi-Agents: What's Actually Working — Cognition's Code-Review Loop and Smart Friend Patterns

The problem

Multi-agent systems with parallel writers produced fragile products because parallel agents made implicit conflicting decisions about style, edge cases, and code patterns, fragmenting decision-making across the system.

First attempt

SWE-1.5 was not capable enough to serve as the primary model in the smart-friend pattern — the gap between it and Sonnet 4.5 was too wide in knowing when to escalate and what to ask the smarter model.

Workflow diagram · grounded in source
1
Coding agent writes PR
ai_action
“even on PRs written by Devin, Devin Review catches an average of 2 bugs per PR”
2
Review agent reviews diff with clean context
ai_action
“The dedicated review agent gets to skip this extraneous context, only look at the diff, and re-discover any context it needs as it reads the code from scratch. With a shorter context, the improved intelligence naturally leads to increase…”
3
Communication bridge filters bugs
validation
“does Devin properly use its broader context of user instructions, decisions, etc. to filter the bugs that come back from Devin Review”
4
Iterative review cycles
feedback_loop
“Often the system will loop through multiple code-review cycles, finding new bugs each time”
5
Human opens resolved PR
output
“most bugs are already resolved by the time a human opens the PR”
6
Primary model escalates to smart friend
routing
“let the primary/smaller model decide when a situation was tricky enough to be worth consulting the smarter/expensive model”
7
Smart friend provides over-scoped guidance
ai_action
“it's often also fruitful to ask the smart friend to look beyond the question the primary model is asking, and suggest any important guidance based on the agent trajectory, even if the primary model didn't ask for it”
8
Manager delegates to child agents
ai_action
“A manager Devin can break a larger task into pieces, spawn child Devins to work on them, and coordinate their progress through an internal MCP”
Reported outcome

The code-review loop has Devin Review catching an average of 2 bugs per PR (roughly 58% severe), with most bugs resolved before a human opens the PR.
The smart-friend pattern produced real gains in the trickiest scenarios when both models are frontier-class.

Reported metrics
bugs caught per PR by Devin Review2
Severe bugs as share of bugs caughtroughly 58%
enterprise Devin usage growth over 6 months~8x
SWE-1.5 inference speed950 tok/sec
Reported stack
DevinDevin ReviewDeepwikiWindsurfClaudeGPTMCPSonnet 4.5
Source
https://cognition.ai/blog/multi-agents-working
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The code-review loop has Devin Review catching an average of 2 bugs per PR (roughly 58% severe), with most bugs resolved before a human opens the PR.

What tools did this team use?

Devin, Devin Review, Deepwiki, Windsurf, Claude, GPT, MCP, Sonnet 4.5.

What results were reported?

bugs caught per PR by Devin Review: 2; Severe bugs as share of bugs caught: roughly 58%; enterprise Devin usage growth over 6 months: ~8x; SWE-1.5 inference speed: 950 tok/sec (source-reported, not independently verified).

What failed first in this deployment?

SWE-1.5 was not capable enough to serve as the primary model in the smart-friend pattern — the gap between it and Sonnet 4.5 was too wide in knowing when to escalate and what to ask the smarter model.

How is this quality assurance AI workflow structured?

Coding agent writes PR → Review agent reviews diff with clean context → Communication bridge filters bugs → Iterative review cycles → Human opens resolved PR → Primary model escalates to smart friend → Smart friend provides over-scoped guidance → Manager delegates to child agents.

Related quality assurance cases