Compliance monitoring · Production

AWS fine-tunes BERTweet LLM to classify toxic speech for a large gaming company

The problem

A large gaming company needed to automate detection of toxic speech in player interactions but lacked sufficient labeled training data and had cost and time constraints that made training a custom language model from scratch unviable.

First attempt

The PoC two-stage model architecture required double the model monitoring, increased costs from running two models, and slower inference speed, prompting a redesign to a single-stage model before production.

Workflow diagram · grounded in source
1
Voice/text excerpt submitted
trigger
“create an English language detector that classifies voice and text excerpts into their own custom defined toxic language categories”
2
LLM fine-tuned on labeled data
ai_action
“AWS decided to first fine-tune BERTweet with the customer's labeled data”
3
Two-stage PoC classification
ai_action
“Created a two-stage prediction flow where the first model acts as a binary classifier that classifies whether a piece of text is toxic or not toxic. The second model is a fine-grained model that classifies text based on the customer's de…”
4
Single-stage production classification
ai_action
“fine-tune the bertweet-base-offensive model on all the labels, including non-toxic samples, into one model”
5
Classification result produced
output
“productionized the LLM for the customer to be used at scale”
Reported outcome

AWS ProServe MLDT productionized a single-stage bertweet-base-offensive model that met the customer's accuracy threshold while improving ease of maintenance and lowering cost; precision decreased by only 3% compared to the two-stage approach.

Reported metrics
binary classifier precision (2-stage PoC).92
binary classifier F1 (2-stage PoC).91
binary classifier AUC (2-stage PoC).92
fine-grained classifier F1 (2-stage PoC).81
Show all 8 reported metrics
binary classifier precision (2-stage PoC).92
binary classifier F1 (2-stage PoC).91
binary classifier AUC (2-stage PoC).92
fine-grained classifier F1 (2-stage PoC).81
bertweet-base-offensive 1-stage precision.88
bertweet-base-offensive 1-stage F1.86
bertweet-base-offensive 2-stage F1.90
precision decrease from 2-stage to 1-stage3%
Reported stack
Amazon SageMakerBERTweetbertweet-base-offensivebertweet-base-hateHugging FaceRoBERTa
Source
https://aws.amazon.com/blogs/machine-learning/aws-performs-fine-tuning-on-a-large-language-model-llm-to-classify-toxic-speech-for-a-large-gaming-company?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

AWS ProServe MLDT productionized a single-stage bertweet-base-offensive model that met the customer's accuracy threshold while improving ease of maintenance and lowering cost; precision decreased by only 3% compared t…

What tools did this team use?

Amazon SageMaker, BERTweet, bertweet-base-offensive, bertweet-base-hate, Hugging Face, RoBERTa.

What results were reported?

binary classifier precision (2-stage PoC): .92; binary classifier F1 (2-stage PoC): .91; binary classifier AUC (2-stage PoC): .92; fine-grained classifier F1 (2-stage PoC): .81 (source-reported, not independently verified).

What failed first in this deployment?

The PoC two-stage model architecture required double the model monitoring, increased costs from running two models, and slower inference speed, prompting a redesign to a single-stage model before production.

How is this compliance monitoring AI workflow structured?

Voice/text excerpt submitted → LLM fine-tuned on labeled data → Two-stage PoC classification → Single-stage production classification → Classification result produced.

Related compliance monitoring cases