Shopify's machine learning playbook: scaling from order fraud detection to hundreds of products
Shopify's rule-based order fraud detection had a high false positive rate, forcing merchants to manually investigate flagged orders and causing them to cancel legitimate sales.
The existing rule-based fraud system flagged orders whenever billing and shipping addresses differed, generating excessive false positives and prompting merchants to cancel valid orders.
The ML fraud detection model beat the baseline and now processes millions of orders a day; the same approach was extended to hundreds of other Shopify products including Shopify Capital, product categorization, and Help Center search.
Frequently asked questions
What did this team achieve with this AI workflow?
The ML fraud detection model beat the baseline and now processes millions of orders a day; the same approach was extended to hundreds of other Shopify products including Shopify Capital, product categorization, and He…
What tools did this team use?
Trino, Ruby on Rails.
What results were reported?
Daily orders processed: millions of orders a day; Model performance vs baseline: beat the baseline (source-reported, not independently verified).
What failed first in this deployment?
The existing rule-based fraud system flagged orders whenever billing and shipping addresses differed, generating excessive false positives and prompting merchants to cancel valid orders.
How is this ecommerce ops AI workflow structured?
Order placed at checkout → ML model scores order for fraud → Risk-based routing before fulfillment → Human oversight of model anomalies.