Quality assurance · Production

Airbnb generates type-safe GraphQL mock data at scale using LLMs and @generateMock

The problem

Airbnb engineers spent significant time manually writing and maintaining GraphQL mock data that frequently drifted out of sync with evolving queries, and client engineers were blocked from iterating on frontend features while waiting for backend server implementations to be complete.

Workflow diagram · grounded in source
1
Engineer adds @generateMock directive
trigger
“engineers can use to automatically generate mock data for a given GraphQL operation, fragment, or field”
2
Hash-based change detection
validation
“Niobe determines whether existing mocks' hashes differ from what their current hashes should be based on the GraphQL document. If they match, it skips mock generation for those types. On the other hand, if one of the hashes changed, Niob…”
3
Design snapshot generation
integration
“If the @generateMock directive includes a designURL, Niobe validates the URL to ensure it includes a node-id, then uses an internal API to produce an image snapshot of that particular node. The API, in turn, pushes this snapshot to a sto…”
4
Context aggregation and LLM call
ai_action
“the CLI aggregates all the context described in the section above — including the URL of the design snapshot — and crafts a prompt to send to the LLM. This prompt is then sent to the Gemini 2.5 Pro model, and results are streamed back to…”
5
GraphQL schema validation
validation
“Niobe performs a validation step against this data by passing the GraphQL schema, client GraphQL document, and JSON data to the graphql [NPM package]'s graphqlSync function.”
6
Self-healing retry on errors
feedback_loop
“If the validation produces errors (for example, if the LLM hallucinated an invalid enum value or failed to populate a required field), Niobe aggregates these errors and feeds them back into the LLM along with the initial mock data. This …”
7
Write mock JSON and source files
output
“Niobe writes it to a JSON file, alongside a companion source file which provides functions for loading the mock from application code”
Reported outcome

Airbnb engineers generated and merged over 700 mocks across iOS, Android, and Web using @generateMock, with engineers reporting significantly faster local development and no need to manually write or maintain mock data.

Reported metrics
Mocks generated and mergedover 700
Local development speedsignificantly sped up
Reported stack
NiobeGemini 2.5 Pro@generateMock@respondWithMock
Source
https://medium.com/airbnb-engineering/graphql-data-mocking-at-scale-with-llms-and-generatemock-30b380f12bd6
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Airbnb engineers generated and merged over 700 mocks across iOS, Android, and Web using @generateMock, with engineers reporting significantly faster local development and no need to manually write or maintain mock data.

What tools did this team use?

Niobe, Gemini 2.5 Pro, @generateMock, @respondWithMock.

What results were reported?

Mocks generated and merged: over 700; Local development speed: significantly sped up (source-reported, not independently verified).

How is this quality assurance AI workflow structured?

Engineer adds @generateMock directive → Hash-based change detection → Design snapshot generation → Context aggregation and LLM call → GraphQL schema validation → Self-healing retry on errors → Write mock JSON and source files.

Related quality assurance cases