Ecommerce ops · Production

Instacart builds a BERT-based contextual sequence retrieval system for in-session product recommendations

The problem

Instacart's recommendation surfaces relied on a disparate set of ad-hoc retrieval systems using product co-occurrence, similarity, and popularity signals, without leveraging in-session contextual sequence information from users.

First attempt

Legacy per-surface retrieval systems failed to use sequential context from user sessions, relied on ad-hoc co-occurrence and popularity signals, and increased maintenance burden by requiring separate systems for ads and organic surfaces.

Workflow diagram · grounded in source
1
In-session user action trigger
trigger
“Our contextual retrieval system reacts in real time to a user's actions within a shopping session and retrieves products relevant to that session.”
2
BERT-like sequence model inference
ai_action
“We train our model on historical sequences of product ids that a user interacts with in a session. At training time, we use a Masked Language Modeling(MLM) approach. At inference time, we use the encoded session representation from the t…”
3
Top-K product retrieval
routing
“we pick the top K products based on predicted probabilities for further downstream ranking”
4
Multi-surface recommendation display
output
“We use this contextual recommendation system across many surfaces at Instacart such as search, item details page, cart, pre-checkout and post-checkout recommendations.”
Reported outcome

The unified BERT-like contextual retrieval system delivered a 30% lift in user cart additions at launch and outsized impact across transaction volume and ad marketplace metrics, while allowing legacy systems to be deprecated.

Reported metrics
User cart additions lift30%
Impact on transaction volume and ad marketplaceoutsized impact across multiple metrics across transaction volume and ad marketplace
Recall@K degradation with randomized training sequences10–40% worse depending on K
Evaluation metric degradation without sequence order at inferencedegraded by 20–45% depending on the value of K
Reported stack
BERTMasked Language ModelingXLNet
Source
https://tech.instacart.com/sequence-models-for-contextual-recommendations-at-instacart-93414a28e70c
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The unified BERT-like contextual retrieval system delivered a 30% lift in user cart additions at launch and outsized impact across transaction volume and ad marketplace metrics, while allowing legacy systems to be dep…

What tools did this team use?

BERT, Masked Language Modeling, XLNet.

What results were reported?

User cart additions lift: 30%; Impact on transaction volume and ad marketplace: outsized impact across multiple metrics across transaction volume and ad marketplace; Recall@K degradation with randomized training sequences: 10–40% worse depending on K; Evaluation metric degradation without sequence order at inference: degraded by 20–45% depending on the value of K (source-reported, not independently verified).

What failed first in this deployment?

Legacy per-surface retrieval systems failed to use sequential context from user sessions, relied on ad-hoc co-occurrence and popularity signals, and increased maintenance burden by requiring separate systems for ads a…

How is this ecommerce ops AI workflow structured?

In-session user action trigger → BERT-like sequence model inference → Top-K product retrieval → Multi-surface recommendation display.

Related ecommerce ops cases