Ecommerce ops · Production

Shopify Global Catalogue: multimodal LLMs standardize product data across billions of listings at 40 million daily inferences

The problem

Shopify's billions of product listings are created by millions of merchants in unstructured, heterogeneous formats—free-form text, custom attribute schemas, missing fields, multilingual content, and information spread across text and images—making machine understanding, semantic search, and consistent product discovery impossible at scale.

First attempt

Commercial LLM APIs (OpenAI and Gemini) proved prohibitively expensive at Shopify's inference volume, and training a fine-tuned model to predict all fields simultaneously caused a loss of generalizability at inference time.

Workflow diagram · grounded in source
1
Streaming product data ingestion
trigger
“We process over 10 million product updates daily from merchant uploads, APIs, apps, and integrations, in a streaming fashion.”
2
Multimodal LLM product understanding
ai_action
“For each catalogue entity, we fine-tune a vision large language model to perform multiple tasks simultaneously, rather than following the traditional approach of building separate models for each task. This architectural choice was not o…”
3
LLM agent annotation pipeline
ai_action
“we deploy multiple LLM agents that act as annotators. These agents independently analyze the product and suggest appropriate categories. For train samples, we scale up using an LLM arbitrator—another model trained to select the best agen…”
4
Human gold-label annotation
human_review
“For test samples, suggestions from these agents are presented to human annotators using a specialized interface. Humans resolve ambiguities and provide gold labels; consensus is established for evaluation.”
5
Product matching and edge pruning
ai_action
“A cascade of discriminator models validates these matches. The system maintains high-precision through what we call 'edge pruning.' After identifying candidate matches, we apply these discriminators to remove potentially incorrect edges.…”
6
Reconciliation to canonical record
ai_action
“the reconciliation layer constructs a canonical product record by aggregating all inferred metadata”
7
Active learning improvement loop
feedback_loop
“LLM judges flag low-quality inferences in production, queuing them for additional human review and retraining.”
8
Downstream system integration
integration
“The catalogue is actively integrated across Shopify's ecosystem”
Reported outcome

The Global Catalogue runs 40 million LLM inferences daily (16 billion tokens per day), reduced median inference latency from 2 seconds to 500 milliseconds, and cut GPU usage by 40%, enabling improved search, recommendations, and conversational commerce across Shopify.

Reported metrics
daily LLM inferences40 million
Tokens inferred per dayabout 16 billion
Daily product updates processedover 10 million
Median inference latencyreduced from 2 seconds to 500 milliseconds
Show all 5 reported metrics
daily LLM inferences40 million
tokens inferred per dayabout 16 billion
daily product updates processedover 10 million
median inference latencyreduced from 2 seconds to 500 milliseconds
GPU token usage40%
Reported stack
LlaVA 1.5 7BLLaMA 3.2 11BQwen2VL 7BTriton inference serverKafka
Source
https://shopify.engineering/leveraging-multimodal-llms
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The Global Catalogue runs 40 million LLM inferences daily (16 billion tokens per day), reduced median inference latency from 2 seconds to 500 milliseconds, and cut GPU usage by 40%, enabling improved search, recommend…

What tools did this team use?

LlaVA 1.5 7B, LLaMA 3.2 11B, Qwen2VL 7B, Triton inference server, Kafka.

What results were reported?

daily LLM inferences: 40 million; Tokens inferred per day: about 16 billion; Daily product updates processed: over 10 million; Median inference latency: reduced from 2 seconds to 500 milliseconds (source-reported, not independently verified).

What failed first in this deployment?

Commercial LLM APIs (OpenAI and Gemini) proved prohibitively expensive at Shopify's inference volume, and training a fine-tuned model to predict all fields simultaneously caused a loss of generalizability at inference…

How is this ecommerce ops AI workflow structured?

Streaming product data ingestion → Multimodal LLM product understanding → LLM agent annotation pipeline → Human gold-label annotation → Product matching and edge pruning → Reconciliation to canonical record → Active learning improvement loop → Downstream system integration.

Related ecommerce ops cases