Ecommerce ops · Production

Instacart builds semantic IDs to power cross-category product understanding and recommendations at scale

The problem

Instacart's hierarchical product taxonomy missed cross-category connections customers naturally expect, leaving new products invisible at cold start, tail categories underserved by recommendation models, and mislabeled products impossible to detect at catalog scale.

First attempt

Vanilla RQ-VAE compression without structural guidance caused fragmentation — similar products landing in different branches — and error propagation from sparse or inconsistent product text, while the rigid taxonomy alone offered no mechanism to flag mislabeled items.

Workflow diagram · grounded in source
1
Product catalog ingestion
trigger
“Operating a grocery catalog at Instacart's scale means managing millions of products across thousands of categories”
2
ESCI model embedding
ai_action
“using our in-house ESCI (Exact, Substitute, Complementary, Irrelevant) model, which learns representations from search relevance data”
3
Gemini Flash attribute extraction
ai_action
“It first runs the product through Gemini Flash (~10x faster, ~5x cheaper than full-size models) to extract structured attributes (product type, key ingredients, dietary tags, format), stripping away marketing copy along with the metadata…”
4
RQ-VAE semantic ID generation
ai_action
“A semantic ID is a short sequence of integers generated by compressing a product's embedding through a residual vector quantizer. Products with similar meaning share prefixes; products that differ split at progressively finer levels.”
5
LLM cluster quality evaluation
validation
“we prompt LLMs to look at each leaf group and score it on three dimensions: functional coherence (do these products serve similar purposes?), purchase likelihood (would a customer buy these together?), and customer journey relevance (do …”
6
Catalog mismatch routing
routing
“automated flagging of code-vs-label mismatches, confidence scoring for how strongly a product fits its cluster versus its label, and prioritized review queues for human verification”
7
Human catalog verification
human_review
“prioritized review queues for human verification”
8
Recommendation surface output
output
“Semantic IDs now power product retrieval, replacement recommendations, and next-item prediction across Instacart”
Reported outcome

Semantic IDs delivered a 34% increase in add-to-carts on product carousels, surfaced products from 2.7x more emerging brands with tail categories seeing the largest gains, and became core infrastructure for product retrieval, replacement recommendations, and next-item prediction across Instacart.

Reported metrics
Add-to-carts on product carousels+34%
Emerging brands surfaced2.7x more emerging brands
Similarity-depth correlation0.69–0.84
highly similar pairs sharing Level 1 code98–99%
Show all 6 reported metrics
add-to-carts on product carousels+34%
emerging brands surfaced2.7x more emerging brands
similarity-depth correlation0.69–0.84
highly similar pairs sharing Level 1 code98–99%
Gemini Flash speed vs full-size models~10x faster
Gemini Flash cost vs full-size models~5x cheaper
Reported stack
RQ-VAEESCIGemini FlashGemmaLLMs
Source
https://tech.instacart.com/semantic-ids-product-understanding-at-scale-5283e0288f5a
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Semantic IDs delivered a 34% increase in add-to-carts on product carousels, surfaced products from 2.7x more emerging brands with tail categories seeing the largest gains, and became core infrastructure for product re…

What tools did this team use?

RQ-VAE, ESCI, Gemini Flash, Gemma, LLMs.

What results were reported?

Add-to-carts on product carousels: +34%; Emerging brands surfaced: 2.7x more emerging brands; Similarity-depth correlation: 0.69–0.84; highly similar pairs sharing Level 1 code: 98–99% (source-reported, not independently verified).

What failed first in this deployment?

Vanilla RQ-VAE compression without structural guidance caused fragmentation — similar products landing in different branches — and error propagation from sparse or inconsistent product text, while the rigid taxonomy a…

How is this ecommerce ops AI workflow structured?

Product catalog ingestion → ESCI model embedding → Gemini Flash attribute extraction → RQ-VAE semantic ID generation → LLM cluster quality evaluation → Catalog mismatch routing → Human catalog verification → Recommendation surface output.

Related ecommerce ops cases