Marketing ops · Production

Lyft re-architects localization pipeline with iterative LLM Drafter/Evaluator, cutting translation latency from days to minutes

The problem

Lyft's localization relied exclusively on human translation with multi-day turnarounds and costs that scaled linearly with each new language. The Quebec launch required Bill 96 French-first compliance faster than multi-day cycles allowed, and European expansion demanded six new languages simultaneously.

First attempt

Traditional Neural Machine Translation providers were fast but failed to preserve Lyft-specific terminology and context. Without deterministic guardrails, LLMs consistently hallucinated variable placeholders — the most common failure mode observed in early deployments.

Workflow diagram · grounded in source
1
Source string submission
trigger
“Requesters submit source strings along with context — where the text appears in the UI and the intended tone.”
2
Pre-translation tokenization
validation
“We use regex pattern matching to identify these elements and replace them with numbered tokens (__PH_0__, __PH_1__, etc.) that the LLM is less likely to accidentally mangle.”
3
Drafter generates candidates
ai_action
“We configure it to generate three distinct candidates for every source string.”
4
Evaluator scores and selects
ai_action
“Each candidate receives a grade: pass or revise. If any candidate passes, we ask the Evaluator to select the best one. If all fail, the Evaluator provides a detailed critique explaining why each failed.”
5
Critique-driven retry loop
feedback_loop
“When the Evaluator rejects all candidates, its critique is captured and injected into the Drafter's next attempt.”
6
Post-translation validation
validation
“After the LLM returns a translation, we run deterministic validation before accepting it. Validation checks three conditions: Presence: Every expected token appears exactly once. No hallucination: No unexpected tokens were introduced. St…”
7
Early release distribution
output
“This version is distributed in minutes, unblocking product launches while linguist review takes place in parallel.”
8
Linguist review and finalization
human_review
“Professional linguists review the drafts within the TMS. Approved translations replace the early-release versions and are established as the definitive system of record. Flagged translations are corrected before being distributed.”
Reported outcome

The iterative LLM pipeline achieves a success rate of over 95% across most languages and reduces translation latency from days to minutes, with 95% of translations requiring no significant changes after linguist review.

Reported metrics
Translation latencyfrom days to minutes
batch translation SLA target30-minute SLA for 95% of translations
LLM translation success rateover 95%
Translations needing no significant changes after linguist review95%
Reported stack
SmartlingPydanticClaude HaikuGPT's mini models
Source
https://eng.lyft.com/scaling-localization-with-ai-at-lyft-b04dca99e6ee
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The iterative LLM pipeline achieves a success rate of over 95% across most languages and reduces translation latency from days to minutes, with 95% of translations requiring no significant changes after linguist review.

What tools did this team use?

Smartling, Pydantic, Claude Haiku, GPT's mini models.

What results were reported?

Translation latency: from days to minutes; batch translation SLA target: 30-minute SLA for 95% of translations; LLM translation success rate: over 95%; Translations needing no significant changes after linguist review: 95% (source-reported, not independently verified).

What failed first in this deployment?

Traditional Neural Machine Translation providers were fast but failed to preserve Lyft-specific terminology and context.

How is this marketing ops AI workflow structured?

Source string submission → Pre-translation tokenization → Drafter generates candidates → Evaluator scores and selects → Critique-driven retry loop → Post-translation validation → Early release distribution → Linguist review and finalization.

Related marketing ops cases