Back office ops · Production

Leading AI lab trains financial reasoning LLM with Labelbox expert labeling services

The problem

A leading AI lab needed domain-specific financial datasets to train their LLM on complex financial reasoning, but lacked the necessary expertise at scale and faced a tight deadline to source qualified financial professionals capable of multi-step analysis.

Workflow diagram · grounded in source
1
AI lab shares project vision
trigger
“The AI lab shared their detailed vision for the project, which included an extensive set of documents and multi-layered instructions.”
2
Custom ontology setup
output
“Using the Labelbox text editor, a custom ontology was built that included classifications and sub-classifications with numerous free-text inputs required from the labelers”
3
Financial expert sourcing
integration
“The Labelbox team reviewed over 50 candidates to assemble the most qualified group for the project”
4
24-hour calibration
validation
“execute a 24-hour calibration period”
5
Expert evaluation and ranking
human_review
“selected experts were tasked with ranking various aspects of the model's generated outputs to complex, hypothetical prompts on a scale of 1 to 5. These rankings included evaluations of hypotheses—such as probability, importance, and feas…”
6
Real-time quality monitoring
validation
“Throughout each evaluation phase, the AI lab was able to view and monitor performance and quality metrics to ensure a successful end result”
7
Iterative workflow adjustment
feedback_loop
“The teams customized and adjusted the project workflows as needed and implemented feedback quickly.”
8
Financial dataset delivery
output
“the AI lab acquired high-quality financial datasets within their tight timeframe”
Reported outcome

Labelbox delivered high-quality financial datasets within the tight timeframe, enabling the AI lab to boost their LLM's performance and improve the accuracy and reliability of its outputs on financial tasks.

Reported metrics
Dataset qualityhigh-quality, differentiated datasets
LLM performanceboost the performance
LLM output accuracy and reliabilityimprove the accuracy and reliability
Reported stack
LabelboxLabelbox text editor
Source
https://labelbox.com/customers/financial-expertise-customer-story/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Labelbox delivered high-quality financial datasets within the tight timeframe, enabling the AI lab to boost their LLM's performance and improve the accuracy and reliability of its outputs on financial tasks.

What tools did this team use?

Labelbox, Labelbox text editor.

What results were reported?

Dataset quality: high-quality, differentiated datasets; LLM performance: boost the performance; LLM output accuracy and reliability: improve the accuracy and reliability (source-reported, not independently verified).

How is this back office ops AI workflow structured?

AI lab shares project vision → Custom ontology setup → Financial expert sourcing → 24-hour calibration → Expert evaluation and ranking → Real-time quality monitoring → Iterative workflow adjustment → Financial dataset delivery.

Related back office ops cases