Ecommerce ops · Production

Booking.com uses supervised fine-tuning with LoRA/QLoRA to achieve 3x faster travel destination recommendations

The problem

Travelers increasingly express vacation needs in unstructured, nuanced natural language that traditional ML models struggle to handle, while prompt-based LLM solutions raise privacy concerns and cannot leverage Booking.com's proprietary behavioral data.

First attempt

The baseline production system — a prompt-based proprietary LLM accessed via external API combined with a traditional ML model — had slower inference and could not safely incorporate sensitive user data due to external processing requirements.

Workflow diagram · grounded in source
1
User submits travel request
trigger
“users describe what they want in their own words”
2
User context incorporation
integration
“This single contextual data point improved our Hit@5 by 8% demonstrating a key advantage of our fine-tuned approach: since the model runs entirely on our internal systems, we can safely incorporate sensitive user data like location witho…”
3
Fine-tuned LLM generates recommendations
ai_action
“we used parameter-efficient fine-tuning methods [LoRA](https://arxiv.org/abs/2106.09685) and [QLoRA](https://arxiv.org/abs/2305.14314) together with supervised fine-tuning (SFT). Here, SFT means training on labeled examples from our doma…”
4
LLM judge validates alignment
validation
“we apply an additional validation step using an LLM as a judge to ensure that the booked destination remains aligned with the user's original conversational intent”
5
Behavioral signals feed training labels
feedback_loop
“Booked destinations are prioritized first, as they represent the strongest positive signal of user intent”
6
K recommendations delivered
output
“the model consistently generates exactly K recommendations, improving both inference speed and output consistency”
Reported outcome

The fine-tuned open-weight model reduced p99 inference latency by 67% (about 3x faster) over the baseline, improved Hit@5 by 8% from incorporating user location context, and delivered strong improvements in recommendation quality validated via A/B test, while keeping all user data internal.

Reported metrics
P99 inference latency reduction67%
Inference speed vs baselineabout 3x faster
Hit@5 improvement from user location context8%
Recommendation quality vs production baselinestrong improvements
Reported stack
LoRAQLoRALLM as a judgeAWS
Source
https://booking.ai/beyond-prompt-engineering-how-we-used-supervised-fine-tuning-for-travel-recommendations-91e8f4711e4b
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The fine-tuned open-weight model reduced p99 inference latency by 67% (about 3x faster) over the baseline, improved Hit@5 by 8% from incorporating user location context, and delivered strong improvements in recommenda…

What tools did this team use?

LoRA, QLoRA, LLM as a judge, AWS.

What results were reported?

P99 inference latency reduction: 67%; Inference speed vs baseline: about 3x faster; Hit@5 improvement from user location context: 8%; Recommendation quality vs production baseline: strong improvements (source-reported, not independently verified).

What failed first in this deployment?

The baseline production system — a prompt-based proprietary LLM accessed via external API combined with a traditional ML model — had slower inference and could not safely incorporate sensitive user data due to externa…

How is this ecommerce ops AI workflow structured?

User submits travel request → User context incorporation → Fine-tuned LLM generates recommendations → LLM judge validates alignment → Behavioral signals feed training labels → K recommendations delivered.

Related ecommerce ops cases