Back office ops · Production

JetBlue uses DSPy and Databricks to build self-optimizing multi-tool LLM pipelines

The problem

Multi-stage LLM pipelines required continuous manual prompt tuning, where single words could make or break a deployment, creating brittle and time-consuming iterative development.

First attempt

Before DSPy, JetBlue relied on manually crafted, highly explicit prompt templates requiring precise hand-written instructions for each pipeline step.

Workflow diagram · grounded in source
1
User question submitted
trigger
“a generated query from user input”
2
Query reformatting via signature
ai_action
“a common, first signature would be to reformat an initial user question into a query using some pre-defined context”
3
Tool selection routing
routing
“we take a generated query from user input, choose to use a vector store if appropriate, and then generate an answer from our retrieved context”
4
Answer generation
ai_action
“generate an answer from our retrieved context”
5
Automatic prompt optimization
feedback_loop
“Before DSPy we manually optimized our prompts to improve these metrics; now we can use DSPy to directly optimize these metrics and improve quality automatically”
6
Deploy to serving endpoint
output
“deployed to one of JetBlue's internal serving endpoints”
Reported outcome

JetBlue deployed its RAG chatbot 2x faster than with Langchain, eliminated the need to manually iterate on prompts, and launched revenue-driving customer feedback classification and RAG-powered predictive maintenance chatbots.

Reported metrics
RAG chatbot deployment speed vs Langchain2x faster
Customer feedback classification business impactrevenue-driving
Predictive maintenance chatbot operational impactbolster operational efficiency
Reported stack
DSPyDatabricksDatabricks Model ServingDatabricks Vector SearchMLflowLangchainDBRXLlama 2 70BOpenAI
Source
https://www.databricks.com/blog/optimizing-databricks-llm-pipelines-dspy
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

JetBlue deployed its RAG chatbot 2x faster than with Langchain, eliminated the need to manually iterate on prompts, and launched revenue-driving customer feedback classification and RAG-powered predictive maintenance…

What tools did this team use?

DSPy, Databricks, Databricks Model Serving, Databricks Vector Search, MLflow, Langchain, DBRX, Llama 2 70B, OpenAI.

What results were reported?

RAG chatbot deployment speed vs Langchain: 2x faster; Customer feedback classification business impact: revenue-driving; Predictive maintenance chatbot operational impact: bolster operational efficiency (source-reported, not independently verified).

What failed first in this deployment?

Before DSPy, JetBlue relied on manually crafted, highly explicit prompt templates requiring precise hand-written instructions for each pipeline step.

How is this back office ops AI workflow structured?

User question submitted → Query reformatting via signature → Tool selection routing → Answer generation → Automatic prompt optimization → Deploy to serving endpoint.

Related back office ops cases