Finance ops · Production

FactSet standardizes GenAI on Databricks Mosaic AI, cutting latency by over 70%

The problem

FactSet lacked a standardized LLM development platform — engineers across teams used disparate tools, cloud-native services, and bespoke solutions with no cohesive governance or data lineage framework.

First attempt

The initial commercial LLM pipeline for code generation produced over a minute of response time, and the Text-to-Formula RAG approach hit a quality ceiling that prevented scaling to more complex formulas.

Workflow diagram · grounded in source
1
News data ingested via Delta Live Tables
integration
“used Delta Live Tables to ingest and parse news data in an XML format”
2
Text chunked and embeddings indexed
ai_action
“chunked the text by length and speaker, created embeddings and updated Vector Search indexes”
3
RAG with open-source model
ai_action
“leveraged an open-source model of choice for RAG”
4
Model Serving delivers to frontend
output
“Model Serving endpoints served responses into a front end application”
5
Fine-tuning reduces code gen latency
feedback_loop
“fine-tune meta-llama-3-70b and, recently, Databricks DBRX to reduce average user request latency by over 70%”
6
SME evaluations validate dataset
human_review
“incorporate subject matter expert (SME) evaluations, and generate supplementary examples that comply with FactSet's governance and compliance policies”
7
Text-to-Formula from natural language
ai_action
“The goal of this project is to accurately generate custom FactSet formulas using natural language queries”
Reported outcome

Standardizing on Databricks Mosaic AI reduced code generation latency by over 70% and cut Text-to-Formula end-to-end latency by about 60%, while providing centralized governance and democratizing advanced AI workflows across teams.

Reported metrics
Code generation latency reductionover 70%
Text-to-Formula end-to-end latency reductionabout 60%
Initial code generation response timeover a minute
Text-to-Formula accuracy improvementnotable improvements in accuracy
Show all 6 reported metrics
code generation latency reductionover 70%
Text-to-Formula end-to-end latency reductionabout 60%
initial code generation response timeover a minute
Text-to-Formula accuracy improvementnotable improvements in accuracy
infrastructure management complexityreduced the complexity of maintaining underlying cloud infrastructure
AI workflow accessibilitydemocratized many advanced AI workflows that were traditionally gated behind traditional AI engineers
Reported stack
DatabricksDatabricks Mosaic AIMLflowDelta Live TablesVector SearchUnity CatalogModel Servingmeta-llama-3-70bDBRXFoundation Model APIsLangchainHuggingFaceLlama 3MistralOpenAIAnthropicMLflow Deployments ServerS3
Source
https://www.databricks.com/blog/factset-genai
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Standardizing on Databricks Mosaic AI reduced code generation latency by over 70% and cut Text-to-Formula end-to-end latency by about 60%, while providing centralized governance and democratizing advanced AI workflows…

What tools did this team use?

Databricks, Databricks Mosaic AI, MLflow, Delta Live Tables, Vector Search, Unity Catalog, Model Serving, meta-llama-3-70b, DBRX, Foundation Model APIs.

What results were reported?

Code generation latency reduction: over 70%; Text-to-Formula end-to-end latency reduction: about 60%; Initial code generation response time: over a minute; Text-to-Formula accuracy improvement: notable improvements in accuracy (source-reported, not independently verified).

What failed first in this deployment?

The initial commercial LLM pipeline for code generation produced over a minute of response time, and the Text-to-Formula RAG approach hit a quality ceiling that prevented scaling to more complex formulas.

How is this finance ops AI workflow structured?

News data ingested via Delta Live Tables → Text chunked and embeddings indexed → RAG with open-source model → Model Serving delivers to frontend → Fine-tuning reduces code gen latency → SME evaluations validate dataset → Text-to-Formula from natural language.

Related finance ops cases