Customer support · Production

How Microsoft built Ask Learn: an Advanced RAG knowledge service powering Q&A Assist and Copilot for Azure

The problem

Microsoft Q&A users typically waited hours for community responses and had no way to receive instant, documentation-grounded answers. Building a generative AI system that could reliably answer technical questions required solving inherent LLM non-determinism at massive scale.

First attempt

A simple 'naïve RAG' approach produced only decent results, which was insufficient for Microsoft's standards requiring reliable, relevant, accurate, and verifiable answers.

Workflow diagram · grounded in source
1
User submits question
trigger
“when a user asks a question, the system generates an embedding vector for the question itself”
2
Query pre-processing
ai_action
“Pre-retrieval processing could include tasks like query rewriting, query expansion, and query clarification.”
3
Vector similarity search
ai_action
“a search is performed to find similar vectors already stored in the database. Vector databases employ a search algorithm known as 'cosine similarly' (or 'nearest neighbor') to find candidate article chunks that might help answer the user…”
4
Post-retrieval processing
ai_action
“Post-retrieval processing could include tasks like re-ranking the document chunks retrieved from the vector database, expanding the chunks to provide more context (so, adding related document chunks to the result set), filtering irreleva…”
5
LLM answer generation
ai_action
“The final step is to send the original question and the closest matching article chunks it found to the LLM to formulate a response.”
6
Safety and quality evaluation
validation
“there may be even more processing to ensure that the answer meets the quality bar, doesn't violate any safety, ethical, or operational guidelines and restrictions”
7
Answer delivered in Azure portal
output
“provides end users with a way to access the information they need while working in the Azure portal instead of opening a second tab to search the documentation for guidance, instructions, or troubleshooting information. Internally, the t…”
8
User feedback capture
feedback_loop
“Customers were asked to rate the quality of the response (thumbs up / thumbs down ratings)”
9
Root cause analysis per response
feedback_loop
“requiring the team to spend up to 30 minutes for each response investigating how the query produced the results, whether the correct chunks from Learn's documentation were used, as well as the chunking strategy used to divide up a given …”
Reported outcome

Microsoft launched Q&A Assist in May 2023 and extended it as Ask Learn, an API that grounds Copilot for Azure answers in Microsoft Learn documentation and serves as a fallback when Copilot cannot fulfill a user request directly.

Reported metrics
Root cause analysis time per feedback responseup to 30 minutes
Team readiness to improve accuracymuch better equipped to make improvements to address relevance and accuracy going forward
User cognitive burden of context switchinglessen this cognitive burden
Reported stack
Azure OpenAI.NETAzure SDKPython notebooksPrompt flowAzure AI Searchvector database
Source
https://devblogs.microsoft.com/engineering-at-microsoft/how-we-built-ask-learn-the-rag-based-knowledge-service/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Microsoft launched Q&A Assist in May 2023 and extended it as Ask Learn, an API that grounds Copilot for Azure answers in Microsoft Learn documentation and serves as a fallback when Copilot cannot fulfill a user reques…

What tools did this team use?

Azure OpenAI, .NET, Azure SDK, Python notebooks, Prompt flow, Azure AI Search, vector database.

What results were reported?

Root cause analysis time per feedback response: up to 30 minutes; Team readiness to improve accuracy: much better equipped to make improvements to address relevance and accuracy going forward; User cognitive burden of context switching: lessen this cognitive burden (source-reported, not independently verified).

What failed first in this deployment?

A simple 'naïve RAG' approach produced only decent results, which was insufficient for Microsoft's standards requiring reliable, relevant, accurate, and verifiable answers.

How is this customer support AI workflow structured?

User submits question → Query pre-processing → Vector similarity search → Post-retrieval processing → LLM answer generation → Safety and quality evaluation → Answer delivered in Azure portal → User feedback capture → Root cause analysis per response.

Related customer support cases