Ecommerce ops · Production

Instacart builds PARSE, a multi-modal LLM platform for catalog attribute extraction at scale

The problem

Instacart's catalog attribute creation relied on SQL-based rules and traditional ML models that struggled with complex or context-dependent attributes, required significant per-attribute engineering effort, and could not extract information from product images — resulting in slow development cycles and inconsistent attribute quality.

First attempt

Pre-LLM approaches — SQL rules and traditional ML models — failed to scale: SQL handled only simple keyword-based extractions, ML required separate labeled datasets and pipelines per attribute, and neither could process image-based product data.

Workflow diagram · grounded in source
1
Configure attribute extraction task
trigger
“teams first use the platform in "development mode" to experiment with different models, prompts, and input sources”
2
Fetch product data via SQL
integration
“a backend orchestration layer will fetch the product data by the input SQL, and send them along with other input parameters to the subsequent components to execute the extraction”
3
LLM extracts attribute values
ai_action
“the endpoint first constructs LLM extraction prompts by inserting product features and attribute definitions into the input prompt template. Then it uses the selected LLM extraction algorithm to extract the attribute value for the product”
4
Self-verification confidence scoring
validation
“We query the LLM with a second scoring prompt. The prompt will ask LLM to do an entailment task: asking LLM if the extracted attribute value by the extraction prompt is correct based on the product features and attribute definition”
5
Human auditor reviews low-confidence items
human_review
“the component also runs a proactive error detection. This process considers the extracted values of products with a low confidence score as potentially incorrect values, and has them reviewed and corrected by human auditors”
6
Results ingested into catalog pipeline
output
“The final extraction results are passed into the catalog data pipeline for ingestion”
7
Periodic production quality monitoring
feedback_loop
“the component creates a sample set periodically from the attribute extraction results of new products, and has it evaluated by either human auditors or LLM evaluation. This can help monitor if there is a quality drop that requires attention”
Reported outcome

PARSE accelerated attribute extraction: simpler attributes now take one day of effort compared to one week previously, complex attribute iteration was reduced to just three days, multi-modal LLMs increased recall by 10% over text-only models, and simpler attributes can be handled at a 70% cost reduction using less powerful models.

Reported metrics
recall improvement of multi-modal LLM over text-only LLM10%
Accuracy for organic attribute extraction (initial prompt)95%
Development time for simpler attributesone day of effort, compared to one week previously
Iteration time for difficult attributesreduced to just three days
Show all 6 reported metrics
recall improvement of multi-modal LLM over text-only LLM10%
accuracy for organic attribute extraction (initial prompt)95%
development time for simpler attributesone day of effort, compared to one week previously
iteration time for difficult attributesreduced to just three days
cost reduction using cheaper LLM for simpler attributes70%
accuracy drop with less powerful LLM for difficult attributes60%
Reported stack
LLMsPARSE
Source
https://tech.instacart.com/multi-modal-catalog-attribute-extraction-platform-at-instacart-b9228754a527
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

PARSE accelerated attribute extraction: simpler attributes now take one day of effort compared to one week previously, complex attribute iteration was reduced to just three days, multi-modal LLMs increased recall by 1…

What tools did this team use?

LLMs, PARSE.

What results were reported?

recall improvement of multi-modal LLM over text-only LLM: 10%; Accuracy for organic attribute extraction (initial prompt): 95%; Development time for simpler attributes: one day of effort, compared to one week previously; Iteration time for difficult attributes: reduced to just three days (source-reported, not independently verified).

What failed first in this deployment?

Pre-LLM approaches — SQL rules and traditional ML models — failed to scale: SQL handled only simple keyword-based extractions, ML required separate labeled datasets and pipelines per attribute, and neither could proce…

How is this ecommerce ops AI workflow structured?

Configure attribute extraction task → Fetch product data via SQL → LLM extracts attribute values → Self-verification confidence scoring → Human auditor reviews low-confidence items → Results ingested into catalog pipeline → Periodic production quality monitoring.

Related ecommerce ops cases