Compliance monitoring · Production

Monzo's machine learning stack: principles, platform architecture, and tools

The problem

Monzo needed a scalable machine learning platform that let ML practitioners deploy models end-to-end without backend engineer handoffs, remained flexible across frameworks and model types, and reused existing infrastructure rather than building an isolated ML-specific stack.

Workflow diagram · grounded in source
1
Prototype in notebooks
trigger
“we encourage people to use Google Colab notebooks in a similar way that engineers use white boarding tools. They help us do rapid exploration and prototyping, and to get a sense of what could work”
2
Dataset creation
integration
“Dataset creation jobs orchestrate several BigQuery SQL queries to create the data we need to train a model, and export a versioned snapshot of that data to Google Cloud Storage”
3
Model training
ai_action
“Model training jobs are run as custom containers on Google's AI Platform and will often begin by downloading a snapshot of data and end by uploading a trained model”
4
Save to model registry
output
“The final steps of a model training job is to save the trained model by uploading it to our model registry. This is a single place where all machine learning models at Monzo are uploaded and enables us to: retrieve the models for inferen…”
5
Batch prediction pipeline
ai_action
“we write dbt models to transform any data and prepare the input for the batch prediction job, and then we write a Python job that pulls in the data, loads the model, and spits out the predictions. The dbt models and the batch job are orc…”
6
Publish predictions as events
integration
“Our batch inference jobs publish predictions as events onto Google Pub Sub so that they can be picked up by consumers who trigger actions”
7
Real-time inference
ai_action
“all of our fraud classifiers need to run every time a transaction is initiated”
8
System health monitoring
validation
“System monitoring covers things related to how healthy the system is: inbound requests, timing, memory and CPU usage, and so on. For this, we use Grafana, just like every backend engineer at Monzo uses for their own services”
9
Model performance monitoring
validation
“Model performance monitoring covers all things related to how impactful the model is: from precision & recall through to impact on specific company metrics (like fraud rates). For this, we use Looker, just like every data scientist at Mo…”
Reported outcome

The platform lets the ML team spend little to no time managing infrastructure.
A uniform approach means engineers can move between ML systems easily, and upgrades are applied once and benefit everyone.

Reported metrics
Infrastructure management timelittle to no time managing infrastructure
Reported stack
Google ColabGitHubcookiecutterBigQueryGoogle Cloud StorageGoogle Container RegistrydbtAirflowGoogle Pub SubNSQCassandraGrafanaLookerscikit-learnXGBoostLightGBMPyTorchtransformersskorchGensim
Source
https://monzo.com/blog/2022/04/26/monzos-machine-learning-stack
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The platform lets the ML team spend little to no time managing infrastructure.

What tools did this team use?

Google Colab, GitHub, cookiecutter, BigQuery, Google Cloud Storage, Google Container Registry, dbt, Airflow, Google Pub Sub, NSQ.

What results were reported?

Infrastructure management time: little to no time managing infrastructure (source-reported, not independently verified).

How is this compliance monitoring AI workflow structured?

Prototype in notebooks → Dataset creation → Model training → Save to model registry → Batch prediction pipeline → Publish predictions as events → Real-time inference → System health monitoring → Model performance monitoring.

Related compliance monitoring cases