Back office ops · Production

Netflix builds natural language Graph Search using LLMs and RAG to replace structured DSL query builders

The problem

Netflix's internal Graph Search platform required users to interact via a structured DSL, forcing them to learn bespoke UI components across many applications and creating bottlenecks for subject matter experts who needed to translate their intent into complex queries.

First attempt

The naive approach of providing all index metadata and controlled vocabulary values as full context to the LLM worked for simple cases but did not scale, increasing latency, decreasing filter correctness, and causing the LLM to frequently hallucinate values that did not exist.

Workflow diagram · grounded in source
1
Natural language query input
trigger
“A user's natural language question (with optional `@mentions` statements) are provided as input, along with the Graph Search index context”
2
Field RAG context scoping
ai_action
“At filter generation time, the user's question is chunked with an overlapping strategy. For each chunk, we perform a vector search to identify the top K most relevant values and the fields to which they belong. Deduplication: The top K f…”
3
Controlled vocabulary RAG
ai_action
“At filter generation time, the user's question is chunked. For each chunk, we perform a vector search to identify the top K most relevant values (but only for the controlled vocabularies that are associated with fields in the index)”
4
LLM filter statement generation
ai_action
“The pre-processed context and the question are fed into the LLM with an instruction asking for a syntactically and semantically correct filter statement”
5
Syntactic and semantic validation
validation
“The generated filer statement DSL is verified and checked for hallucinations”
6
LLM self-correction loop
feedback_loop
“If a hallucination is detected it can be returned as an error to a user, indicating the need to refine the query, or can be provided back to the LLM in the form of a feedback loop for self correction. This increases the filter generation…”
7
UI chips and facets output
output
“we parse it into an AST, and then map that AST to the existing "Chips" and "Facets" in our UI”
Reported outcome

Netflix built a text-to-query system that converts natural language into Graph Search Filter DSL using RAG for field and controlled vocabulary context, with deterministic validation and transparent UI Chips and Facets so users can verify and trust generated queries.

Reported metrics
Pragmatic correctness via @mentionssignificantly increasing pragmatic correctness
search accuracy with AI integrationimproved accuracy
Reported stack
LLMRAGvector storeAbstract Syntax Tree (AST) parserGraphQL
Source
https://netflixtechblog.com/the-ai-evolution-of-graph-search-at-netflix-d416ec5b1151
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Netflix built a text-to-query system that converts natural language into Graph Search Filter DSL using RAG for field and controlled vocabulary context, with deterministic validation and transparent UI Chips and Facets…

What tools did this team use?

LLM, RAG, vector store, Abstract Syntax Tree (AST) parser, GraphQL.

What results were reported?

Pragmatic correctness via @mentions: significantly increasing pragmatic correctness; search accuracy with AI integration: improved accuracy (source-reported, not independently verified).

What failed first in this deployment?

The naive approach of providing all index metadata and controlled vocabulary values as full context to the LLM worked for simple cases but did not scale, increasing latency, decreasing filter correctness, and causing…

How is this back office ops AI workflow structured?

Natural language query input → Field RAG context scoping → Controlled vocabulary RAG → LLM filter statement generation → Syntactic and semantic validation → LLM self-correction loop → UI chips and facets output.

Related back office ops cases