Deque uses Labelbox Model Diagnostics and Catalog to improve accessibility ML model performance by 5%+ and cut labeling spend by over 50%
The Deque team spent approximately 10 minutes annotating each webpage screen for accessibility compliance and managed thousands of web and mobile datasets using disparate open-source annotation tools combined with ad-hoc Jupyter Notebooks and Google Sheets, making model diagnostics and data collection a manual, disjointed process.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Model Diagnostics identifies weaknesses
Model Diagnostics in Labelbox targets the model's weaknesses, replacing the prior manual prediction visualization process.
Tools used
LabelboxModel DiagnosticsCatalog
Outcome
By filtering out one-third of less trustworthy data points and targeting data collection via Model Diagnostics and Catalog, Deque improved overall model performance by 5%+ and reduced annual labeling spend and needs by over 50%.
What failed first
Before Labelbox, Deque's team had to manually visualize predictions and calculate all metrics themselves, and data collection was a scattershot process that would have required roughly twice as much data and effort for equivalent model improvement.