Marketing ops · Production

Shopify Kit evolves from heuristic rules to ML-driven marketing campaign recommendations

The problem

Shopify business owners, especially those new to marketing, were overwhelmed by the large number of configurations required to create effective campaigns in Facebook Ads Manager. The existing rule-based Kit system used hardcoded budget ranges identical for all business owners, failing to serve those with higher traffic, sales, and spending capacity.

First attempt

The initial Spark-based batch prediction flow produced stale budget recommendations because predictions were generated on a schedule rather than on demand, so the recommendation could be outdated by the time it reached the business owner.

Workflow diagram · grounded in source
1
Business owner messages Kit
trigger
“Kit interacts with business owners through messages over various interfaces including Shopify Ping and SMS”
2
Aggregate marketing and store features
integration
“Aggregate all relevant features. This includes the historic Facebook marketing campaigns created by business owners through Shopify, and the store state (e.g. traffic and sales) at the time when they create the marketing campaign”
3
Train regression and classification models
ai_action
“Regression: given a business owner's historic spending behavior, predict the budget range that they're likely to spend. Classification: given the budget a business owner has with store attributes such as existing traffic and sales that c…”
4
Monitor model metrics
validation
“We employed two types of monitoring strategies: 1) threshold: alert when the model metric is beyond a defined threshold; 2) outlier detection: alert when the model metrics deviates from its normal distribution”
5
Real-time prediction generates recommendation
ai_action
“Kit is proactively calling into the real time prediction service to generate the recommendation. once features are generated during the feature engineering stage, they are immediately loaded into a key value store using Google Cloud's Bi…”
6
Deliver one-step marketing recommendation
output
“Kit can generate actionable marketing recommendation that gives the business owners the best chance of making sales based on their budget range and the state of their stores”
Reported outcome

One third of all Kit marketing campaigns are powered by ML-driven recommendations, and the system serves tens of thousands of business owners.

Reported metrics
Kit marketing campaigns powered by MLone third
Business owners servedtens of thousands of business owners
Reported stack
KitApache KafkaTensorFlowscikit-learnFacebook Ads ManagerShopify Ping
Source
https://shopify.engineering/evolution-kit-automating-marketing-machine-learning
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

One third of all Kit marketing campaigns are powered by ML-driven recommendations, and the system serves tens of thousands of business owners.

What tools did this team use?

Kit, Apache Kafka, TensorFlow, scikit-learn, Facebook Ads Manager, Shopify Ping.

What results were reported?

Kit marketing campaigns powered by ML: one third; Business owners served: tens of thousands of business owners (source-reported, not independently verified).

What failed first in this deployment?

The initial Spark-based batch prediction flow produced stale budget recommendations because predictions were generated on a schedule rather than on demand, so the recommendation could be outdated by the time it reache…

How is this marketing ops AI workflow structured?

Business owner messages Kit → Aggregate marketing and store features → Train regression and classification models → Monitor model metrics → Real-time prediction generates recommendation → Deliver one-step marketing recommendation.

Related marketing ops cases