Ecommerce ops · Production

Instacart transitions from batch-oriented to real-time machine learning across its four-sided marketplace

The problem

Instacart's batch-oriented ML systems produced stale predictions, wasted compute on inactive users, could not cover long-tail user-item pairs, and lacked access to real-time signals such as live product availability and in-session shopping intent.

First attempt

Batch ML performed poorly on new queries, siloed streaming technologies across teams produced inconsistent event quality, and the shift to real-time serving introduced latency and availability risks that the existing infrastructure could not absorb.

Workflow diagram · grounded in source

Raw events published to Kafka

trigger

“The pipeline listens to raw events stored in Kafka published by services”

Flink stream processing into Feature Store

integration

“transforms them into desired features using Flink, and sinks them into Feature Store for on-demand access”

Real-time model inference via RPC

ai_action

“Online Inference Platform is a system that hosts each model as an RPC (Remote Procedure Call) endpoint”

Personalization and fulfillment output

output

“The platform transforms the shopping journey to be more dynamic and efficient, with better personalization and optimized fulfillment”

A/B experiment validation

feedback_loop

“the platform has enabled considerable GTV (gross transaction value) growth in a series of A/B experiments”

Reported outcome

The real-time ML platform reduced item availability update latency from hours to seconds, enabled session-based personalization, and reduced millions in fraud-related costs annually, with GTV growth confirmed in A/B experiments.

Reported metrics

Item availability update latencyseconds from a couple of hours

Fraud-related costs reducedmillions of fraud-related costs annually

Feature generation latencyseconds contrast to hours before

Gross transaction value growthconsiderable GTV growth

Show all 6 reported metrics

item availability update latencyseconds from a couple of hours

fraud-related costs reducedmillions of fraud-related costs annually

feature generation latencyseconds contrast to hours before

gross transaction value growthconsiderable GTV growth

item found ratedirectly improves item found rate

customer satisfactionincreases customer satisfaction

Reported stack

KafkaFlinkFeature StoreGriffinOnline Inference Platform

Source

https://tech.instacart.com/lessons-learned-the-journey-to-real-time-machine-learning-at-instacart-942f3a656af3

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The real-time ML platform reduced item availability update latency from hours to seconds, enabled session-based personalization, and reduced millions in fraud-related costs annually, with GTV growth confirmed in A/B e…

What tools did this team use?

Kafka, Flink, Feature Store, Griffin, Online Inference Platform.

What results were reported?

Item availability update latency: seconds from a couple of hours; Fraud-related costs reduced: millions of fraud-related costs annually; Feature generation latency: seconds contrast to hours before; Gross transaction value growth: considerable GTV growth (source-reported, not independently verified).

What failed first in this deployment?

How is this ecommerce ops AI workflow structured?

Raw events published to Kafka → Flink stream processing into Feature Store → Real-time model inference via RPC → Personalization and fulfillment output → A/B experiment validation.