Quality assurance · Production

Ancestry uses Labelbox to achieve weekly ML model iteration cycles for genealogical data extraction

The problem

Ancestry's data science team needed more efficient ways to label training data for ML models decoding census records and historical artifacts, and struggled to involve domain experts meaningfully because data scientists owned the entire labeling task end-to-end while the review process was painfully slow.

Workflow diagram · grounded in source
1
Census and artifact labeling need
trigger
“The team was looking for more efficient ways to decode census data and to train their ML models faster”
2
Model-assisted labeling
ai_action
“leveraging the latest in labeling automation and collaboration including model-assisted labeling, annotation relationships, as well as Boost labeling services”
3
Domain expert annotation
human_review
“Finding easier ways to involve their domain experts during the labeling and review process was essential because it helped unlock the insights of these subject matter experts who have a wealth of knowledge in looking at historical documents”
4
Real-time inline QA
validation
“dropping a pin on any image [asset], where we'll have a question and we can respond to our labelers in line”
5
Labeler analytics and correction
feedback_loop
“The ability to evaluate how labelers were annotating data with analytics and in-depth metrics further helped the Ancestry team to be able to communicate in real-time and correct labels as needed”
6
Train and test ML models
output
“the Ancestry team is utilizing Labelbox's data engine to train models faster and evaluate validation test sets more easily”
Reported outcome

The Ancestry team reached a weekly model iteration cycle, saved time collaborating with domain experts, and can now train and test new models in record time using Labelbox's data engine.

Reported metrics
Model iteration cycleweekly iteration cycle
Time collaborating with domain expertssave time
Model training and testing speedrecord time
Labeling and review turnaround (prior state)took forever
Reported stack
LabelboxAnnotateBoost
Source
https://labelbox.com/customers/ancestry-customer-story/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The Ancestry team reached a weekly model iteration cycle, saved time collaborating with domain experts, and can now train and test new models in record time using Labelbox's data engine.

What tools did this team use?

Labelbox, Annotate, Boost.

What results were reported?

Model iteration cycle: weekly iteration cycle; Time collaborating with domain experts: save time; Model training and testing speed: record time; Labeling and review turnaround (prior state): took forever (source-reported, not independently verified).

How is this quality assurance AI workflow structured?

Census and artifact labeling need → Model-assisted labeling → Domain expert annotation → Real-time inline QA → Labeler analytics and correction → Train and test ML models.

Related quality assurance cases