Compliance monitoring · Production
Stripe's ML flywheel reduces successful card testing attacks by 80%
The problem
Card testing fraud is difficult to detect because it blends with legitimate traffic and attackers constantly evolve their tactics, making high-precision identification a challenging ML problem.
Workflow diagram · grounded in source
1
Multi-level ML transaction scoring
ai_action
“At the highest level of abstraction, we use ML to estimate the overall prevalence of card testing on Stripe. This allows us to update our risk posture on a daily basis. At the next level down, we apply ML to estimate where card testing i…”
2
Breakthrough attack labeling
human_review
“They include consolidating intelligence around newly identified attack vectors, automating discovery of hidden patterns by combining weaker signals, and applying manual expert review”
3
Feature engineering on Shepherd
ai_action
“Our next-generation feature engineering platform, Shepherd, makes this easy. Shepherd—which we built through a partnership with Airbnb—allows multiple teams at Stripe with different skill sets to generate new features with minimal code c…”
4
Model retraining and blue-green deployment
ai_action
“We do this using Shepherd and Flyte—an ML orchestration platform—which facilitates experimentation through standardized, automated workflows. We retrain our models using the proposed feature, deploy them on offline data, and evaluate the…”
5
Foundation transformer model augmentation
ai_action
“we augment our smaller, card testing–specific models with a large transformer model trained on billions of global transactions. It can detect patterns that are not easily discerned by simpler models. It also compresses payments into atom…”
6
Detection feeds flywheel
feedback_loop
“detection enables us to add new data labels and features, which we then feed back into ML models (at different levels of abstraction) for retraining and redeployment”
Reported outcome
Successful card testing attacks on Stripe decreased by 80% over two years, even as payment volume expanded to over $1 trillion.
Reported metrics
Successful card testing attacks80%
Stripe payment volumeover $1 trillion
Proactive identification time after breakthrough attackwithin hours
Reported stack
ShepherdFlyte
Source
https://stripe.com/blog/the-ml-flywheel-how-we-continually-improve-our-models-to-reduce-card-testing
Read source ↗Frequently asked questions
What did this team achieve with this AI workflow?
Successful card testing attacks on Stripe decreased by 80% over two years, even as payment volume expanded to over $1 trillion.
What tools did this team use?
Shepherd, Flyte.
What results were reported?
Successful card testing attacks: 80%; Stripe payment volume: over $1 trillion; Proactive identification time after breakthrough attack: within hours (source-reported, not independently verified).
How is this compliance monitoring AI workflow structured?
Multi-level ML transaction scoring → Breakthrough attack labeling → Feature engineering on Shepherd → Model retraining and blue-green deployment → Foundation transformer model augmentation → Detection feeds flywheel.