Quality assurance · Production

How a Fortune 500 creative tools company shipped generative AI across its products using Labelbox

The problem

The company's R&D division spent excessive engineering cycles building custom training data infrastructure for generative AI products, resulting in project delays, siloed AI development across groups, and a fragmented data quality evaluation process that reduced model confidence and slowed ROI from AI/ML investments.

First attempt

The team's homegrown training data infrastructure was siloed and fragmented with no central platform for AI teams to collaborate or evaluate data quality consistently, leading to lower model confidence.

Workflow diagram · grounded in source
1
Training data need identified
trigger
“looking for a single platform to consolidate and unify their data labeling efforts for generative AI product development”
2
Catalog filters unstructured data
ai_action
“Labelbox Catalog provided the ability to quickly leverage metadata and custom embeddings for filtering all of this unstructured data, saving the team hours of time every day for a process that would have taken months of custom engineerin…”
3
Annotate curates training data
human_review
“The customer was then able to leverage Labelbox Annotate to develop a consistent process for curating high-quality AI data”
4
QA tools validate model quality
validation
“Using built-in quality assurance tools, the team quickly saw increases in model performance, specifically for their NLP and computer vision projects which focused on understanding the complex structure of PDF documents”
5
Teams define labeling instructions
integration
“multiple data science teams and product teams are now able to more effectively work with internal stakeholders to define what data they are interested in, and to translate requests into specific labeling instructions”
6
Dynamic queue processes PDFs
integration
“the company can also now process tens of thousands of PDFs using a dynamic queueing system, prioritizing labeled data to enhance generative AI outputs”
7
AI Assistant products deployed
output
“Their AI Assistant products, which provide a comprehensive understanding of PDF structure and content, have been released to production in 2023”
Reported outcome

The company achieved a 50% reduction in labeling operations time and a 5X increase in AI product deployment speed within 8 months, and released AI Assistant products providing comprehensive PDF understanding to production in 2023.

Reported metrics
Labeling operations time50%
AI product deployment speed5X
Time to achieve resultswithin just 8 months
Daily time savings on data filteringhours of time every day
Show all 5 reported metrics
labeling operations time50%
AI product deployment speed5X
time to achieve resultswithin just 8 months
daily time savings on data filteringhours of time every day
engineering work avoidedmonths of custom engineering work
Reported stack
Labelbox CatalogLabelbox AnnotateLabelbox Boost
Source
https://labelbox.com/customers/creative-software-customer-story/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The company achieved a 50% reduction in labeling operations time and a 5X increase in AI product deployment speed within 8 months, and released AI Assistant products providing comprehensive PDF understanding to produc…

What tools did this team use?

Labelbox Catalog, Labelbox Annotate, Labelbox Boost.

What results were reported?

Labeling operations time: 50%; AI product deployment speed: 5X; Time to achieve results: within just 8 months; Daily time savings on data filtering: hours of time every day (source-reported, not independently verified).

What failed first in this deployment?

The team's homegrown training data infrastructure was siloed and fragmented with no central platform for AI teams to collaborate or evaluate data quality consistently, leading to lower model confidence.

How is this quality assurance AI workflow structured?

Training data need identified → Catalog filters unstructured data → Annotate curates training data → QA tools validate model quality → Teams define labeling instructions → Dynamic queue processes PDFs → AI Assistant products deployed.