Ecommerce ops · Production
Instacart optimizes e-commerce search relevance using hybrid retrieval and query entropy
The problem
Instacart's search fetched a fixed number of documents from text and semantic retrieval independently, regardless of query or retailer context, causing over-fetching and reduced precision for queries with few relevant results.
Workflow diagram · grounded in source
1
User search query received
trigger
“retrieve the relevant documents for a given search query”
2
Calculate query entropy
ai_action
“Query entropy measures the specificity of a query and models the variation or uncertainty in the number of relevant documents for that query. query_entropy = -Σ P(doc_id | query) log₂P(doc_id | query)”
3
Adjust recall thresholds
ai_action
“Adjust Recall Thresholds: Adjust the recall thresholds for text and semantic retrieval mechanisms based on the calculated entropy. recall_threshold = min(M, max(L, M * query_product_entropy / Q))”
4
Text retrieval from Postgres
integration
“For text retrieval, we rely on Postgres and SQL queries are used to fetch relevant documents based on the query context. Indexing: Documents are indexed using GIN indexes. Scoring: A customized term-frequency algorithm (ts_rank) scores d…”
5
Semantic retrieval via FAISS
ai_action
“we use an approximate-nearest neighbor (ANN) search service built using Facebook's FAISS library. This method operates on vector embeddings... We use a bi-encoder model based on the Huggingface MiniLM-L3-v2 architecture to generate query…”
6
Merge recall sets and pass to ranking
output
“The top K relevant products after merging these two lists are then passed down to the downstream ranking stages”
Reported outcome
An adaptive recall system driven by query entropy improved mean converting position by 1.7% and reduced search latency by 1.5%, overcoming the over-fetching problem.
Reported metrics
Mean converting position1.7%
Search latency1.5%
Reported stack
PostgresFAISS
Source
https://tech.instacart.com/optimizing-search-relevance-at-instacart-using-hybrid-retrieval-88cb579b959c
Read source ↗Frequently asked questions
What did this team achieve with this AI workflow?
An adaptive recall system driven by query entropy improved mean converting position by 1.7% and reduced search latency by 1.5%, overcoming the over-fetching problem.
What tools did this team use?
Postgres, FAISS.
What results were reported?
Mean converting position: 1.7%; Search latency: 1.5% (source-reported, not independently verified).
How is this ecommerce ops AI workflow structured?
User search query received → Calculate query entropy → Adjust recall thresholds → Text retrieval from Postgres → Semantic retrieval via FAISS → Merge recall sets and pass to ranking.