marketing_ops · workflow
Duolingo builds bandit algorithm to personalize daily practice reminder notifications
Duolingo's practice reminders were selected at random from a pool, missing the opportunity to tailor messages to individual learners and potentially leaving engagement gains on the table.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · A/B test new templates
New notifications are run as experiments on a small number of learners before being permanently added to the pool.
Tools used
AWS Kinesis FirehoseSpark
Outcome
Within weeks of deployment, more learners were completing lessons more frequently, and the system was especially successful at helping tens of thousands of new learners return to their lessons.
What failed first
Standard bandit algorithms reuse the top-performing notification repeatedly, which conflicted with a confirmed novelty effect: learners are most receptive to notifications they have not recently seen, and repetition erodes effectiveness.
Results
Time savedtens of millions of records per week
Volume~200 million
Grounding & classification
Source type: technical build writeup
19 fields verified against source quotes.
personalizationpredictive analyticsrecommendation systemfailure mode describedmetric backednamed customerproduction runtime claimedsource backedtools describedworkflow describededucationconversion increasethroughput increasetechnical build writeupmarketing ops