Customer support · Production

Otter Assistant: LLM-powered in-house support agent autonomously handles ~50% of customer requests

The problem

Otter's broad restaurant software feature set created high customer support demand requiring deep system integration, but existing vendors offered only hard-coded decision trees without the integration flexibility Otter needed.

First attempt

Established vendors relied on hard-coded decision trees without a clear LLM strategy, and LLM-native startups lacked the ability to handle the complexity of Otter's resolution workflows.

Workflow diagram · grounded in source
1
Customer request triggers workflow
trigger
“based on customer request, find out the corresponding predefined procedure”
2
GetRunbook: LLM intent match and retrieval
ai_action
“GetRunbook function takes the user issue description as an input and outputs a corresponding runbook if it can find one. Otherwise it returns "Not Found". Under the hood, we use a LLM for intent matching and runbook retrieval. Based on t…”
3
API functions: data retrieval and modification
integration
“it has the ability to choose from a list of API wrapper function calls to gather information (e.g. fetch store status status) and/or modify a user's account. Fortunately, we were able to largely reuse pre-existing API calls within the Ot…”
4
Research: RAG over knowledge base
ai_action
“The research function is designed to retrieve and summarize helpful answers to user questions that don't match a runbook”
5
Widget: user confirms write operation
human_review
“For any critical write operation (e.g. Pause store, update price) we require user review and explicit click confirmation before kicking off the operation. We strictly follow this approach to mitigate risk from LLM hallucination (e.g. inc…”
6
EscalateToHuman: hand off to live agent
routing
“When the LLM detects a user's intent to escalate, we can inform the chat message interface to pass conversation control to the assigned human agent, which in turn calls Zendesk to connect to a live agent”
7
Resolution metric feedback loop
feedback_loop
“we defined and instrumented a "resolution" metric. This metric informs us of the Bot's overall performance and in turn the business impact it generates, and allows us to identify issues and improvement opportunities”
Reported outcome

Otter Assistant autonomously handles approximately 50% of inbound customer requests without human intervention and without compromising customer satisfaction, while also exposing previously undetected product and platform issues.

Reported metrics
Inbound customer requests handled autonomously~50%
Support requests resolved autonomously~half of support requests autonomously
Customer satisfaction maintainedwithout compromising customer satisfaction
Reported stack
LLMvector dbStreamlitRAGZendesk
Source
https://techblog.cloudkitchens.com/p/llm-support-agent
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Otter Assistant autonomously handles approximately 50% of inbound customer requests without human intervention and without compromising customer satisfaction, while also exposing previously undetected product and plat…

What tools did this team use?

LLM, vector db, Streamlit, RAG, Zendesk.

What results were reported?

Inbound customer requests handled autonomously: ~50%; Support requests resolved autonomously: ~half of support requests autonomously; Customer satisfaction maintained: without compromising customer satisfaction (source-reported, not independently verified).

What failed first in this deployment?

Established vendors relied on hard-coded decision trees without a clear LLM strategy, and LLM-native startups lacked the ability to handle the complexity of Otter's resolution workflows.

How is this customer support AI workflow structured?

Customer request triggers workflow → GetRunbook: LLM intent match and retrieval → API functions: data retrieval and modification → Research: RAG over knowledge base → Widget: user confirms write operation → EscalateToHuman: hand off to live agent → Resolution metric feedback loop.

Related customer support cases