Ecommerce ops · Production

Supercharging Discovery in Search with LLMs at Instacart

The problem

Instacart's Related Items section returned irrelevant alternatives when exact matches were unavailable, and the search stack failed to surface complementary products that paired with a user's primary find, leaving follow-up purchase intent unaddressed.

First attempt

An initial basic LLM generation approach misinterpreted brand queries and produced overly generic recommendations, such as suggesting raw protein foods when users consistently converted on protein bars and powders, resulting in poor engagement.

Workflow diagram · grounded in source
1
Batch query data preparation
integration
“We run a batch job to extract search queries issued by our users from historical logs, and enrich with the necessary metadata like QU signals, consecutive search terms and any other signals that are required for implementing the techniqu…”
2
Domain-enriched prompt construction
integration
“We use a predefined prompt template as a base structure. For each historical query, we populate this template with the enriched query and its associated metadata. This process creates a unique, contextually-rich prompt tailored to each s…”
3
LLM substitute and complementary generation
ai_action
“We run a batch job to invoke the LLM and store its response in a key value store. The key is the query and the value is the LLM response containing the substitute and complementary recommendations.”
4
LLM output to product mapping
integration
“we take each item in the list generated above, treat it like a search query, and invoke our existing search engine to get the best product matches for the query”
5
Post-processing and diversity reranking
validation
“We perform post processing steps to remove duplicates or similar products, and remove any irrelevant products that could have been recalled. We use a diversity based reranking algorithm to ensure that users can see a variety of options.”
6
Discovery carousel served at runtime
output
“When a user issues a query on our app, along with recalling the usual search results, we also look up the LLM-content table and display the inspirational products in a carousel with suitable titles.”
7
LLM-as-Judge content evaluation
feedback_loop
“we adopted the paradigm of using LLM as a Judge to evaluate the quality of the content”
Reported outcome

LLM-powered discovery content delivered substantial improvements in user engagement and revenue, with the next-search-term extension producing an 18% improvement in engagement rate with inspirational content.

Reported metrics
Engagement rate with inspirational content18%
User engagement and revenuesubstantial improvements
User engagement and revenue from advanced generationnotable boosts
Reported stack
LLMsQuery Understanding modelsWhole Page Ranker model
Source
https://tech.instacart.com/supercharging-discovery-in-search-with-llms-556c585d4720
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

LLM-powered discovery content delivered substantial improvements in user engagement and revenue, with the next-search-term extension producing an 18% improvement in engagement rate with inspirational content.

What tools did this team use?

LLMs, Query Understanding models, Whole Page Ranker model.

What results were reported?

Engagement rate with inspirational content: 18%; User engagement and revenue: substantial improvements; User engagement and revenue from advanced generation: notable boosts (source-reported, not independently verified).

What failed first in this deployment?

An initial basic LLM generation approach misinterpreted brand queries and produced overly generic recommendations, such as suggesting raw protein foods when users consistently converted on protein bars and powders, re…

How is this ecommerce ops AI workflow structured?

Batch query data preparation → Domain-enriched prompt construction → LLM substitute and complementary generation → LLM output to product mapping → Post-processing and diversity reranking → Discovery carousel served at runtime → LLM-as-Judge content evaluation.

Related ecommerce ops cases