ecommerce_ops · ecommerce · workflow
Building the Intent Engine: How Instacart Revamped Query Understanding with LLMs
Instacart's traditional ML-based Query Understanding system struggled with long-tail searches, noisy labeled data, and a fragmented architecture of multiple independent bespoke models that introduced inconsistencies and slowed development.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · User search query submitted
When people search for items on Instacart, the Query Understanding pipeline processes their typed phrases.
Tools used
LLMsRAGLoRAA100 GPUH100 GPUs
Outcome
The LLM-powered system increased query rewrite coverage to over 95% with 90%+ precision, reduced average scroll depth by 6%, and cut user complaints for tail query search results by 50%, with the system now serving millions of cold-start queries weekly.
What failed first
The legacy query rewrite system covered only 50% of search traffic, and the category classification model produced irrelevant suggestions due to noisy conversion data while lacking contextual understanding for nuanced queries.
Results
Time savednearly 700ms
Volumeover 95%
Grounding & classification
Source type: technical build writeup
39 fields verified against source quotes.
content generationdata extractiondocument classificationenterprise searchragknowledge baseproduct catalogmetric backednamed customerproduction runtime claimedproduction verifiedtools describedworkflow describedecommerceaccuracy improvementautomation ratecustomer satisfactioncycle time reductionerror reductiontechnical build writeupecommerce opsextract classify routerag answering