marketing_ops · saas · workflow
Nextdoor increases email engagement with generative AI and rejection sampling
Nextdoor's notification email subject lines defaulted to the first few words of posts, which were often uninformative greetings. Off-the-shelf ChatGPT API produced subject lines that were less engaging, inauthentic, and prone to hallucination.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Post selected for notification email
A post is selected for a New and Trending notification email, requiring a subject line to be determined.
Tools used
OpenAI APIChatGPT APITenacity
Outcome
The final system increased Sessions by 3% over the prompt-only approach, raised Weekly Active Users by 0.4%, and grew Ads revenue by 1%, while cutting serving cost to 1/600 via caching.
What failed first
Initial attempts using ChatGPT API with prompt engineering produced subject lines that underperformed user-generated ones in A/B tests; even after multiple iterations, results remained inferior to the control, and the model hallucinated irrelevant content.
Results
Time saved0.4%
Volume3%
Cost replaced1%
Grounding & classification
Source type: technical build writeup
26 fields verified against source quotes.
content generationpredictive analyticsemailsocial media postfailure mode describedmetric backednamed customerproduction runtime claimedsource backedtools describedworkflow describedsoftwareconversion increasecost reductionrevenue increasetechnical build writeupmarketing opsautonomous resolutionextract classify route