Order processing · Production

Choco AI scales LLM order extraction with few-shot learning and dynamic context injection

The problem

Choco AI had to process restaurant orders arriving in highly varied formats—shorthand texts, PDF email attachments, and voicemail recordings—and information extraction from these orders introduced errors including wrong-column extraction, misinterpreted abbreviations, and speech-to-text mistranscriptions.

First attempt

Metadata-based retrieval of few-shot examples had limitations with order layout variations, and a generic catch-all prompt improved performance initially but struggled with brand names and uncommon products.

Workflow diagram · grounded in source
1
Order arrives in varied format
trigger
“Some restaurants place orders using shorthand text messages with abbreviations, while others rely on third-party tools that send semi-structured orders as PDF attachments via email”
2
Whisper ASR transcription
ai_action
“Whisper–one of the ASR models we use”
3
Semantic retrieval of few-shot examples
ai_action
“At inference, we compute the embedding of the incoming order and perform a semantic search to retrieve labeled examples with similar structure. If the retrieved candidates meet a similarity threshold, they are used as few-shot examples i…”
4
Dynamic context injection
ai_action
“We inject supplementary information to provide the LLM with richer context for improved extractions”
5
LLM information extraction to JSON
ai_action
“Each order goes through a different processing step that extracts key details–such as product names, quantities and restaurant name– and converts them into a structured JSON format required for downstream tasks”
6
Human labeling feedback loop
feedback_loop
“we developed a simple interface that enables our customer success teams to annotate orders directly within our order inspection tool. When a new order format or failure mode is identified, we can immediately add a new label, ensuring our…”
Reported outcome

Dynamic context injection and semantic few-shot retrieval consistently led to higher accuracy in information extraction, and smaller models achieved comparable performance with reduced cost.

Reported metrics
Information extraction accuracyhigher accuracy
Whisper hallucinationsremoved those hallucinations completely
Model costreducing costs and increasing agility
Voicemail transcription correctionsin some cases correcting every mistranscription
Reported stack
Whisperembeddings model
Source
https://choco.com/us/stories/life-at-choco/scaling-ai-applications-with-llms-part-2
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Dynamic context injection and semantic few-shot retrieval consistently led to higher accuracy in information extraction, and smaller models achieved comparable performance with reduced cost.

What tools did this team use?

Whisper, embeddings model.

What results were reported?

Information extraction accuracy: higher accuracy; Whisper hallucinations: removed those hallucinations completely; Model cost: reducing costs and increasing agility; Voicemail transcription corrections: in some cases correcting every mistranscription (source-reported, not independently verified).

What failed first in this deployment?

Metadata-based retrieval of few-shot examples had limitations with order layout variations, and a generic catch-all prompt improved performance initially but struggled with brand names and uncommon products.

How is this order processing AI workflow structured?

Order arrives in varied format → Whisper ASR transcription → Semantic retrieval of few-shot examples → Dynamic context injection → LLM information extraction to JSON → Human labeling feedback loop.

Related order processing cases