Lead processing · Production
Building a machine learning pipeline with DBT and BigQuery ML to predict user applications
The problem
Building a proper data pipeline for feature engineering, model training, and predictions is complicated, and moving data out of the database adds operational overhead.
Workflow diagram · grounded in source
1
Load user event data
integration
“the next step is to upload some sample data to BigQuery and perform some machine learning on it. For this lesson, we can use some sample user event data.”
2
Feature engineering via SQL
integration
“convert our data so that each user is represented by one row, and that we have columns indicating the number of times each user performed certain actions like visiting different pages”
3
Train logistic regression model
ai_action
“model_type='logistic_reg',
'early_stop': true”
4
Generate predictions with probabilities
output
“not only are there related predictions, but that we also have a probability associated with each prediction. For example, in the first record, DBT is predicting with .93 confidence that the observation has a label of 0”
5
DBT pipeline orchestration
integration
“DBT allows us to define the dependencies of each step so that it imposes a sequence to these steps”
Reported outcome
A DBT and BigQuery ML pipeline was built that predicts which users are most likely to apply to a program, enabling the sales team to target top leads and identify which site events drive applications.
Reported metrics
Model prediction confidence.93
Reported stack
BigQuery MLDBTdbt_ml
Frequently asked questions
What did this team achieve with this AI workflow?
A DBT and BigQuery ML pipeline was built that predicts which users are most likely to apply to a program, enabling the sales team to target top leads and identify which site events drive applications.
What tools did this team use?
BigQuery ML, DBT, dbt_ml.
What results were reported?
Model prediction confidence: .93 (source-reported, not independently verified).
How is this lead processing AI workflow structured?
Load user event data → Feature engineering via SQL → Train logistic regression model → Generate predictions with probabilities → DBT pipeline orchestration.