Quality assurance · Production

Airbnb migrates 3.5K Enzyme test files to React Testing Library with LLMs in 6 weeks

The problem

Airbnb needed to migrate nearly 3.5K React component test files from Enzyme to React Testing Library, but the frameworks differed too fundamentally to automate simply, and deleting the files would create significant code coverage gaps. Manual migration was estimated at 1.5 years of engineering time.

Workflow diagram · grounded in source
1
Migration initiation
trigger
“updating nearly 3.5K React component test files from Enzyme to use React Testing Library (RTL) instead”
2
State machine validation
validation
“We modeled this flow like a state machine, moving the file to the next state only after validation on the previous state passed”
3
LLM fixes failed validation
ai_action
“each file moves through stages of validation, and when a check fails, we bring in the LLM to fix it”
4
Dynamic retry loop
ai_action
“we updated our steps to use dynamic prompts for each retry, giving the validation errors and the most recent version of the file to the LLM, and built a loop runner that ran each step up to a configurable number of attempts”
5
Rich context prompting
ai_action
“our prompts had expanded to anywhere between 40,000 to 100,000 tokens, pulling in as many as 50 related files”
6
Sample, tune, sweep loop
feedback_loop
“Using these two features, we could quickly run a feedback loop to improve our prompts and tooling”
7
Manual completion of residual files
human_review
“we opted to manually fix the remaining files, working from the baseline (failing) refactors to reduce the time to get those files over the finish line”
8
Migration output
output
“we were able to replace Enzyme while maintaining original test intent and our overall code coverage”
Reported outcome

Using LLMs and a step-based automation pipeline with retry loops, Airbnb migrated 75% of files in the first four-hour bulk run and reached 97% automation after four days of refinement, completing the full 3.5K-file migration in just 6 weeks — against a 1.5-year manual estimate — while maintaining original test intent and code coverage.

Reported metrics
Manual migration estimate1.5 years
Actual migration completion time6 weeks
Target test files3.5K
Files migrated in first bulk run75%
Show all 6 reported metrics
manual migration estimate1.5 years
actual migration completion time6 weeks
target test files3.5K
files migrated in first bulk run75%
first bulk run durationfour hours
automated migration rate after refinement97%
Reported stack
React Testing LibraryEnzymefrontier models
Source
https://medium.com/airbnb-engineering/accelerating-large-scale-test-migration-with-llms-9565c208023b
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Using LLMs and a step-based automation pipeline with retry loops, Airbnb migrated 75% of files in the first four-hour bulk run and reached 97% automation after four days of refinement, completing the full 3.5K-file mi…

What tools did this team use?

React Testing Library, Enzyme, frontier models.

What results were reported?

Manual migration estimate: 1.5 years; Actual migration completion time: 6 weeks; Target test files: 3.5K; Files migrated in first bulk run: 75% (source-reported, not independently verified).

How is this quality assurance AI workflow structured?

Migration initiation → State machine validation → LLM fixes failed validation → Dynamic retry loop → Rich context prompting → Sample, tune, sweep loop → Manual completion of residual files → Migration output.

Related quality assurance cases