DoorDash builds LLM guardrail system to automate restaurant menu transcription from photos
DoorDash previously relied on humans to manually transcribe restaurant menus from photos, a process described as costly and time-consuming. LLMs alone could not achieve the required high accuracy due to diverse menu structures, incomplete menus, and low-quality photos.
LLMs used as standalone transcription tools produced errors due to inconsistent menu structures, incomplete menus, and low photo quality. Intensive efforts to improve LLM accuracy still required too much time and investment to meet production standards.
DoorDash deployed a partial automation pipeline combining LLM transcription with an ML guardrail model that routes high-confidence transcriptions to production automatically and low-confidence ones to human review, improving efficiency without sacrificing quality and enabling rapid adoption of new AI models.
Frequently asked questions
What did this team achieve with this AI workflow?
DoorDash deployed a partial automation pipeline combining LLM transcription with an ML guardrail model that routes high-confidence transcriptions to production automatically and low-confidence ones to human review, im…
What tools did this team use?
OCR, LightGBM, ResNet, DiT, CNN.
What results were reported?
Manual transcription effort: costly and time-consuming; Automation ratio: higher ratio of automation while ensuring quality (source-reported, not independently verified).
What failed first in this deployment?
LLMs used as standalone transcription tools produced errors due to inconsistent menu structures, incomplete menus, and low photo quality.
How is this data entry ops AI workflow structured?
Menu photo submitted → OCR text extraction → LLM item extraction and structuring → Guardrail model quality check → Route by accuracy threshold → Human transcription fallback.