Customer support · Production

Automation Platform v2: Improving conversational AI at Airbnb

The problem

Airbnb's v1 conversational AI platform relied on rigid, predefined step-by-step workflows that were not flexible enough for diverse customer scenarios and required product creators to manually build new workflows for every use case, making scaling time-consuming and error prone.

First attempt

Automation Platform v1's fixed workflow model was too rigid for emerging LLM use cases, and LLM-powered applications themselves are not yet fully production-ready for all of Airbnb's scale due to latency and hallucination concerns.

Workflow diagram · grounded in source
1
User inquiry triggers platform
trigger
“user inquiry arrives at our platform. Based on the inquiry, our platform collects relevant contextual information, such as previous chat history, user id, user role, etc.”
2
Prompt assembly and LLM call
ai_action
“our platform loads and assembles the prompt using inquiry and context, then sends it to LLM”
3
Chain of Thought reasoning
ai_action
“The core idea of Chain of Thought is to use an LLM as the reasoning engine to determine which tools to use and in which order. Tools are the way an LLM interacts with the world to solve real problems, for example checking a reservation's…”
4
Tool/service call execution
integration
“the first LLM response will be requesting a tool execution that makes a service call to fetch the most recent reservation of the current user. Our platform follows this order and does the actual service call then saves call responses int…”
5
Guardrails content check
validation
“the content moderation guardrail calls various LLMs to detect violations in communication content, and tool guardrails use rules to prevent bad execution, for example updating listings with invalid setup”
6
Response returned and recorded
output
“our platform returns LLM response and records this round of conversion for future reference”
Reported outcome

Automation Platform v2 enables developers to build LLM applications that help customer support agents work more efficiently, provide better resolutions, and deliver quicker responses, by combining LLMs with traditional workflows.

Reported metrics
Customer support agent efficiencywork more efficiently
Resolution qualitybetter resolutions
Response speedquicker responses
AI practitioner productivityefficiency and productivity gains
Reported stack
Automation PlatformChain of ThoughtLLM
Source
https://medium.com/airbnb-engineering/automation-platform-v2-improving-conversational-ai-at-airbnb-d86c9386e0cb
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Automation Platform v2 enables developers to build LLM applications that help customer support agents work more efficiently, provide better resolutions, and deliver quicker responses, by combining LLMs with traditiona…

What tools did this team use?

Automation Platform, Chain of Thought, LLM.

What results were reported?

Customer support agent efficiency: work more efficiently; Resolution quality: better resolutions; Response speed: quicker responses; AI practitioner productivity: efficiency and productivity gains (source-reported, not independently verified).

What failed first in this deployment?

Automation Platform v1's fixed workflow model was too rigid for emerging LLM use cases, and LLM-powered applications themselves are not yet fully production-ready for all of Airbnb's scale due to latency and hallucina…

How is this customer support AI workflow structured?

User inquiry triggers platform → Prompt assembly and LLM call → Chain of Thought reasoning → Tool/service call execution → Guardrails content check → Response returned and recorded.

Related customer support cases