Quality assurance · Production

How Criteo builds better contextual advertising products with AI

The problem

Criteo's Publisher Content Analysis team could not reliably improve label quality or efficiency for their ML models, and managed unstructured image data through Excel spreadsheets with label quality defined through many back-and-forth internal emails.

Workflow diagram · grounded in source
1
Scale ML annotation need
trigger
“The Criteo team wanted to quickly scale their ability to improve the AI data used for their ML models across multiple project teams”
2
Labelbox Annotate adoption
integration
“Labelbox Annotate was used as the primary platform to enable better internal team communication, resulting in a massive reduction in the back and forth needed to convert unstructured image data for AI use”
3
Subject matter expert review
human_review
“having a dedicated platform where subject matter experts could do the human review would significantly impact model performance”
4
Annotation results delivered
output
“immediately see a 40% gain in annotation delivery speed, as well as comparable increases in data annotation quality”
Reported outcome

Criteo immediately saw a 40% gain in annotation delivery speed with comparable increases in annotation quality, and achieved a massive reduction in daily back-and-forth between product and ML teams.

Reported metrics
Annotation delivery speed40%
Data annotation qualitycomparable increases in data annotation quality
Internal back-and-forth communicationmassive reduction in the back and forth
Reported stack
Labelbox Annotate
Source
https://labelbox.com/customers/criteo-customer-story
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Criteo immediately saw a 40% gain in annotation delivery speed with comparable increases in annotation quality, and achieved a massive reduction in daily back-and-forth between product and ML teams.

What tools did this team use?

Labelbox Annotate.

What results were reported?

Annotation delivery speed: 40%; Data annotation quality: comparable increases in data annotation quality; Internal back-and-forth communication: massive reduction in the back and forth (source-reported, not independently verified).

How is this quality assurance AI workflow structured?

Scale ML annotation need → Labelbox Annotate adoption → Subject matter expert review → Annotation results delivered.

Related quality assurance cases