It support · Production

ElasticGPT: Elastic's internal RAG-based generative AI assistant for secure employee knowledge discovery

The problem

Elastic's workforce lacked a secure, context-aware internal knowledge assistant, while employees using unsanctioned external AI tools ('shadow AI') with sensitive company data posed compliance risks.

First attempt

An early attempt using Hugging Face's Chat UI was abandoned when its limitations became clear as users demanded custom features, prompting a switch to Elastic's own EUI.

Workflow diagram · grounded in source
1
Employee submits a query
trigger
“When a user asks "What's our Q1 sales target?"”
2
Route to SmartSource or direct LLM
routing
“For GPT-4o and GPT-4o-mini, the API bypasses the RAG pipeline, routing queries directly to the Azure-hosted models for non-contextual tasks like brainstorming or general Q&A”
3
Vector search retrieves context
ai_action
“SmartSource performs a lightning-fast vector search in Elasticsearch to retrieve the most relevant context — perhaps a snippet from a sales report or meeting notes — and feeds it to GPT-4o for a polished response”
4
LangChain orchestrates RAG pipeline
ai_action
“LangChain manages the RAG pipeline end-to-end: chunking ingested data, generating embeddings, retrieving context from Elasticsearch, and crafting prompts for GPT-4o”
5
Response streamed to employee
output
“The frontend streams responses in real time, so users see answers unfold naturally — think of it like a conversation, not a loading screen. Source attribution and linking builds trust”
6
Interactions logged for improvement
feedback_loop
“Every interaction — whether with SmartSource, GPT-4o, or GPT-4o-mini — is meticulously logged in Elasticsearch. This includes user messages, timestamps, feedback, and metadata. Storing this data in Elasticsearch isn't just about record-k…”
Reported outcome

ElasticGPT is already slashing redundant IT queries and creating employee efficiencies, while reducing the potential impact of shadow AI by providing secure access to multiple LLMs.

Reported metrics
redundant IT query volumeslashing redundant IT queries
Employee efficienciescreating employee efficiencies
Reported stack
ElasticsearchElastic CloudEUIElastic ObservabilitySmartSourceGPT-4oGPT-4o-miniLangChainKibanaReactKubernetesOktaAzureEnterprise ConnectorsElastic APMServiceNow
Source
https://www.elastic.co/blog/generative-ai-elasticgpt
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

ElasticGPT is already slashing redundant IT queries and creating employee efficiencies, while reducing the potential impact of shadow AI by providing secure access to multiple LLMs.

What tools did this team use?

Elasticsearch, Elastic Cloud, EUI, Elastic Observability, SmartSource, GPT-4o, GPT-4o-mini, LangChain, Kibana, React.

What results were reported?

redundant IT query volume: slashing redundant IT queries; Employee efficiencies: creating employee efficiencies (source-reported, not independently verified).

What failed first in this deployment?

An early attempt using Hugging Face's Chat UI was abandoned when its limitations became clear as users demanded custom features, prompting a switch to Elastic's own EUI.

How is this it support AI workflow structured?

Employee submits a query → Route to SmartSource or direct LLM → Vector search retrieves context → LangChain orchestrates RAG pipeline → Response streamed to employee → Interactions logged for improvement.

Related it support cases