Back office ops · Production

Teads lets AI agents orchestrate ML experiments via Datakinator MCP, yielding 5–10% model uplift and nearly $1M in margin gain

The problem

Even after Datakinator gained a UI, manually adding features or choosing hyperparameters still took several minutes and left room for human error; earlier, the platform required launching a Scala notebook on the cloud that took up to 15 minutes each time, limiting the team to a few hundred experiments a year.

First attempt

The first agentic iteration relied only on existing API routes without enriched context, causing failures such as using the wrong date and referencing features that did not exist in specific datasets.

Workflow diagram · grounded in source
1
Data scientist triggers agent
trigger
“it was able to do all the boring tasks and start experiments”
2
Context probing
ai_action
“we enriched the MCP with context tools, making it capable of probing the datasets, retrieving the exact error of a run, and gathering more context about the features”
3
Cost estimation and approval
human_review
“I just added a tool for the agent to estimate the price and ask permission before starting something expensive; if it is too expensive, it is forbidden”
4
Experiment execution
output
“An experiment is basically the combination of processing the data, training, and evaluating the result on out-of-sample data”
5
Failure correction loop
feedback_loop
“it was capable of archiving failed runs, correcting them, and starting them again”
Reported outcome

After enriching the MCP with context tools and adding cost guardrails, the agent enabled over 200 experiments in 48 hours, delivered up to a 5–10% uplift on offline metrics across multiple models, and translated to nearly a million in direct margin gain.

Reported metrics
Experiments started in 48 hoursmore than 200 experiments
Offline metric uplift5–10%
Direct margin gainnearly a million in direct margin gain
ROI headline$1M ROI
Show all 7 reported metrics
experiments started in 48 hoursmore than 200 experiments
offline metric uplift5–10%
direct margin gainnearly a million in direct margin gain
ROI headline$1M ROI
annual experiments post-UI3,000+
gross margin from platform improvementsmillions in gross margin
legacy notebook startup timeup to 15 minutes
Reported stack
DatakinatorMCPcoding agent
Source
https://medium.com/teads-engineering/we-let-ai-agents-orchestrate-our-ml-experiments-fc8606816fde
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

After enriching the MCP with context tools and adding cost guardrails, the agent enabled over 200 experiments in 48 hours, delivered up to a 5–10% uplift on offline metrics across multiple models, and translated to ne…

What tools did this team use?

Datakinator, MCP, coding agent.

What results were reported?

Experiments started in 48 hours: more than 200 experiments; Offline metric uplift: 5–10%; Direct margin gain: nearly a million in direct margin gain; ROI headline: $1M ROI (source-reported, not independently verified).

What failed first in this deployment?

The first agentic iteration relied only on existing API routes without enriched context, causing failures such as using the wrong date and referencing features that did not exist in specific datasets.

How is this back office ops AI workflow structured?

Data scientist triggers agent → Context probing → Cost estimation and approval → Experiment execution → Failure correction loop.

Related back office ops cases