How Roblox Uses AI to Moderate Content on a Massive Scale
Roblox's user-generated platform grew in both scale and speed far beyond what human moderators could handle alone, requiring scalable AI infrastructure to moderate billions of pieces of content in real time across dozens of languages.
An earlier rules-based text filter and CPU-based serving infrastructure could not keep pace with the volume and speed demands of the platform as it scaled.
AI moderation now handles over 750,000 text filter RPS and 370,000 PII filter RPS at peak.
The PII filter reduced false positives by 30% and increased automatic PII detection by 25%. The voice safety classifier achieves 92% higher recall than the initial version at a 1% false positive rate. Real-time feedback interventions reduced filtered chat messages by 5% and abuse-report consequences by 6%.
Show all 17 reported metrics
Frequently asked questions
What did this team achieve with this AI workflow?
AI moderation now handles over 750,000 text filter RPS and 370,000 PII filter RPS at peak.
What tools did this team use?
transformer-based text filter, PII filter, voice safety classifier, large language models (LLMs).
What results were reported?
Daily chat messages: 6.1 billion; Daily voice communication: 1.1 million hours; Policy violation detection rate: 0.01%; Median time to action for illegal content: ten minutes (source-reported, not independently verified).
What failed first in this deployment?
An earlier rules-based text filter and CPU-based serving infrastructure could not keep pace with the volume and speed demands of the platform as it scaled.
How is this compliance monitoring AI workflow structured?
Content submitted by users → ML text filtering → Voice safety classification → Real-time user feedback → Human expert review → Active learning update.