Customer support · Production

Vimeo prototypes a generative AI help desk chat using RAG over Zendesk articles

The problem

Vimeo's existing customer support options—ticket submission, Help Center search, and a third-party chatbot—failed to surface relevant, immediate answers to user queries, leaving customers frustrated and often redirected to submit a support ticket.

First attempt

The existing third-party chatbot failed to surface relevant answers and redirected users to tickets. During prototype development, ChatGPT models were found to hallucinate outdated or nonexistent Vimeo Help Center links based on stale training data.

Workflow diagram · grounded in source
1
Zendesk articles indexed in vector store
integration
“The first step was to scrape all of the published articles from Vimeo's Help Center instance through Zendesk's Help Center API”
2
Customer submits support question
trigger
“customers input their questions and receive immediate, accurate, and helpful responses”
3
LLM rephrases question as standalone
ai_action
“we took any available chat history from the current session and combined it with the latest question from the user. The chat transcript was sent to the LLM to rephrase the input as a standalone question, which enabled us to use the conte…”
4
Vector store retrieval
ai_action
“the standalone question was transformed into its embedding representation using the same APIs as before when ingesting the articles into the vector store. This enabled us to query the vector store for articles with similar content to the…”
5
Off-topic query screening
validation
“detailed instructions in the prompt, directing the AI to refuse to answer any questions not related to Vimeo and its features”
6
LLM generates final answer
ai_action
“the relevant documents were passed along together with the standalone question to the LLM, to generate the final answer”
7
Response returned with source links
output
“One of the reasons we tag the metadata with the source URL is to provide links to the documents below the chat response”
Reported outcome

The prototype demonstrated a resilient system capable of handling a broad range of customer support queries.
Vimeo selected Google Vertex AI Chat Bison for its concise responses, cost effectiveness, and seamless Google Cloud authentication.

Reported metrics
Response quality vs existing chatbotimmediate, accurate, and helpful responses
Vertex AI model cost impactsome cost savings
Reported stack
ZendeskLangchainHNSWLibGoogle Vertex AI Chat BisonOpenAI ChatGPT 3.5 TurboOpenAI ChatGPT 4Azure OpenAI ChatGPT 3.5 Turbo
Source
https://medium.com/vimeo-engineering-blog/from-idea-to-reality-elevating-our-customer-support-through-generative-ai-101a2c5ea680
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The prototype demonstrated a resilient system capable of handling a broad range of customer support queries.

What tools did this team use?

Zendesk, Langchain, HNSWLib, Google Vertex AI Chat Bison, OpenAI ChatGPT 3.5 Turbo, OpenAI ChatGPT 4, Azure OpenAI ChatGPT 3.5 Turbo.

What results were reported?

Response quality vs existing chatbot: immediate, accurate, and helpful responses; Vertex AI model cost impact: some cost savings (source-reported, not independently verified).

What failed first in this deployment?

The existing third-party chatbot failed to surface relevant answers and redirected users to tickets.

How is this customer support AI workflow structured?

Zendesk articles indexed in vector store → Customer submits support question → LLM rephrases question as standalone → Vector store retrieval → Off-topic query screening → LLM generates final answer → Response returned with source links.

Related customer support cases