Quality assurance · Production

Factory.ai Missions: multi-agent architecture for autonomous multi-day software development

The problem

Single-agent AI sessions degrade in quality as scope broadens because irrelevant context accumulates in the trajectory, and an agent that has already implemented something is biased when evaluating its own work.

Workflow diagram · grounded in source
1
User submits goal
trigger
“A user describes what they want built. The orchestrator investigates and asks clarifying questions until the requirements are unambiguous.”
2
Orchestrator writes validation contract
ai_action
“it writes the validation contract - a finite checklist of testable behavioral assertions that define completion and correctness for the mission. This ordering matters. When creating the validation contract, the orchestrator draws from it…”
3
Feature and milestone decomposition
ai_action
“it decomposes the work into features, where each feature is a bounded piece of implementation that claims which assertions it will fulfill. Features are grouped into milestones, each of which encompasses a logical unit of functionality.”
4
Workers implement per feature
ai_action
“A programmatic runner takes the feature list and spawns a worker for each feature in order. Each worker starts with a fresh context, receives its feature spec, writes tests first, then implements.”
5
Milestone validation triggered
validation
“Once all features within a milestone are complete, the runner triggers validation using fresh agents.”
6
Scrutiny and user-testing validators evaluate
validation
“Scrutiny validators review each worker's implementation and trajectory for quality and correctness, and encode relevant knowledge updates into shared state. User-testing validators exercise the system as a black box - using it the way a …”
7
Orchestrator creates fix features loop
feedback_loop
“the orchestrator reviews what workers and validators flagged. It creates fix features targeted at actionable gaps, which get executed before the milestone re-validates. This loop repeats until milestone validation passes.”
8
Blocked mission handed to user
human_review
“If implementation or validation is blocked, the orchestrator halts the mission and hands control back to the user.”
Reported outcome

A single Missions run produced a full Slack clone across six milestones, generating 38.8k lines of code with 89.25% statement coverage; validators surfaced 81 issues resolved through 21 targeted fix features, with every milestone converging in 2-4 validation rounds.

Reported metrics
Validation share of total runtime37.2%
Lines of code generated38.8k
Test lines share of total code52.5%
Statement coverage89.25%
Show all 12 reported metrics
validation share of total runtime37.2%
lines of code generated38.8k
test lines share of total code52.5%
statement coverage89.25%
issues surfaced by validators81
fix features generated21
fix feature ratio34.4%
implementation features total61
milestones completedsix
validation rounds per milestone2-4 validation rounds
median implementation run length51 assistant turns
median validation run length30 assistant turns
Reported stack
Droid
Source
https://factory.ai/news/missions-architecture
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

A single Missions run produced a full Slack clone across six milestones, generating 38.8k lines of code with 89.25% statement coverage; validators surfaced 81 issues resolved through 21 targeted fix features, with eve…

What tools did this team use?

Droid.

What results were reported?

Validation share of total runtime: 37.2%; Lines of code generated: 38.8k; Test lines share of total code: 52.5%; Statement coverage: 89.25% (source-reported, not independently verified).

How is this quality assurance AI workflow structured?

User submits goal → Orchestrator writes validation contract → Feature and milestone decomposition → Workers implement per feature → Milestone validation triggered → Scrutiny and user-testing validators evaluate → Orchestrator creates fix features loop → Blocked mission handed to user.

Related quality assurance cases