back_office_ops · workflow
Duolingo runs STAPLE shared task to automate generation of all acceptable translations for language course sentences
Human translators at Duolingo must manually generate every acceptable translation for each course sentence, a task that can yield enormous numbers of valid variants and significantly slows down course and content creation.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Source sentences provided
Duolingo provides source sentences from its language courses as input data for the STAPLE shared task.
Tools used
Amazon Translate
Outcome
Top-performing teams in the STAPLE shared task achieved Weighted F1 scores around 0.55 and dramatically outperformed Amazon Translate; Duolingo plans to use this research to build a dedicated translation tool and autocomplete suggestions for course creators.
What failed first
Google Translate cannot solve this problem because it is designed to produce only a single translation per sentence, trained on generic data that has only one target translation per source.
Results
Volumearound 0.55
Grounding & classification
Source type: technical build writeup
17 fields verified against source quotes.
content generationtranslationknowledge basefailure mode describedmetric backednamed customertools describedworkflow describededucationaccuracy improvementcycle time reductionemployee productivitytechnical build writeupback office opsai draft human approval