customer_support · saas · workflow
Elastic Field Engineering tunes RAG search for ~75% relevance improvement in Technical Support AI Assistant
The initial RAG-powered Technical Support AI Assistant returned irrelevant results for CVE and product-version queries, causing the LLM to generate uninformed answers and eroding user trust.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Knowledge base ingestion
An extensive database of over 300,000 documents including Technical Support Knowledge Articles and crawled website pages is stored in Elasticsearch as the search foundation.
Tools used
ElasticsearchELSERGPT4TypeScript/Node.jsRanking Evaluation API
Outcome
Search relevance improved by ~75% in top-3 results (precisely +78.41% average P@K improvement) and over 300,000 AI-generated summaries were produced for use across future applications.
What failed first
Three root causes degraded search quality: document summaries were just the first few characters of the body, misleading semantic embeddings; multiple versions of the same article cluttered top results; and version-specific boosting was absent so wrong article versions were returned.
Results
Volume~75%
Grounding & classification
Source type: technical build writeup
26 fields verified against source quotes, 1 dropped as unverifiable.
chatbotenterprise searchknowledge searchragsummarizationknowledge basesupport ticketfailure mode describedmetric backednamed customerproduction runtime claimedtools describedworkflow describedsoftwareaccuracy improvementtime savedtechnical build writeupcustomer supportit supportrag answering