Ecommerce ops · Production

Zalando deploys RNN-based deep learning system in production for predicting consumer behavior

The problem

Zalando needed precise predictions of consumer interest in fashion articles to personalize experiences, but their existing approach relied on traditional machine learning techniques (logistic regression, random forests) that required time-intensive human feature engineering and were not well-suited to modeling sequential consumer histories.

First attempt

Logistic regression and random forests were part of Zalando's previous stack but were replaced because they required time-intensive human feature engineering and were not designed to operate directly on sequences of data.

Workflow diagram · grounded in source
1
Event stream data collected
trigger
“Our raw data consists of tracking data which is collected as an event-stream from the fashion store and saved to AWS S3.”
2
Daily data aggregation
integration
“a data processing pipeline based on Apache Spark. The aggregation jobs run daily on AWS EMR and are scheduled using AWS's data-pipelines: Once yesterday's raw data is available at S3, a new cluster is spawned to transform the newly avail…”
3
RNN model training
ai_action
“we decided to start off with a simple but powerful RNN architecture with a single LSTM-layer. We implemented the model in Torch, together with scripts for training and prediction.”
4
Model validation
validation
“After training, the models are validated on independent test data, using metrics like AUC and data likelihood. Recording these metrics allows us to monitor stability and enables us to prevent uploading models that do not achieve satisfac…”
5
Batch prediction computation
ai_action
“predictions are scheduled and performed at regular intervals for batches of data. Computing predictions is less involving and thus can be handled with regular CPUs. Calculations for several million consumer histories take about 20-30 min…”
6
Prediction monitoring
feedback_loop
“During these processing steps, we closely monitor input data as well as prediction results. Key statistics, like the number of data points and the distribution of variables and targets, help to detect major changes in the incoming data d…”
7
Predictions served live
output
“The system is live, serving consumers on Zalando today.”
Reported outcome

The RNN-based deep learning system is live in production, serving consumers on Zalando, with positive first experiences in both performance and robustness.

Reported metrics
Model training timeabout two hours
Batch prediction computation timeabout 20-30 minutes
GPU vs CPU training performancemultiple times the performance as with CPUs
Live system experiencepositive, both in regards to performance and robustness
Reported stack
RNNsLSTMTorchApache SparkAWS S3AWS EMRAWS EC2AWS data-pipelinesCuda
Source
https://engineering.zalando.com/posts/2017/03/deep-learning-in-production-for-predicting-consumer-behavior.html
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The RNN-based deep learning system is live in production, serving consumers on Zalando, with positive first experiences in both performance and robustness.

What tools did this team use?

RNNs, LSTM, Torch, Apache Spark, AWS S3, AWS EMR, AWS EC2, AWS data-pipelines, Cuda.

What results were reported?

Model training time: about two hours; Batch prediction computation time: about 20-30 minutes; GPU vs CPU training performance: multiple times the performance as with CPUs; Live system experience: positive, both in regards to performance and robustness (source-reported, not independently verified).

What failed first in this deployment?

Logistic regression and random forests were part of Zalando's previous stack but were replaced because they required time-intensive human feature engineering and were not designed to operate directly on sequences of d…

How is this ecommerce ops AI workflow structured?

Event stream data collected → Daily data aggregation → RNN model training → Model validation → Batch prediction computation → Prediction monitoring → Predictions served live.

Related ecommerce ops cases