customer_support · saas · workflow
How Microsoft built Ask Learn: an Advanced RAG knowledge service powering Q&A Assist and Copilot for Azure
Microsoft Q&A users typically waited hours for community responses and had no way to receive instant, documentation-grounded answers. Building a generative AI system that could reliably answer technical questions required solving inherent LLM non-determinism at massive scale.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · User submits question
When a user asks a question, the system generates an embedding vector for the question itself.
Tools used
Azure OpenAI.NETAzure SDKPython notebooksPrompt flowAzure AI Searchvector database
Outcome
Microsoft launched Q&A Assist in May 2023 and extended it as Ask Learn, an API that grounds Copilot for Azure answers in Microsoft Learn documentation and serves as a fallback when Copilot cannot fulfill a user request directly.
What failed first
A simple 'naïve RAG' approach produced only decent results, which was insufficient for Microsoft's standards requiring reliable, relevant, accurate, and verifiable answers.
Results
Time savedup to 30 minutes
Volumemuch better equipped to make improvements to address relevance and accuracy going forward
Running sinceMay 2023
Grounding & classification
Source type: technical build writeup
33 fields verified against source quotes.
agentic workflowconversational aiknowledge searchragsummarizationknowledge basebuilder submittedfailure mode describedhuman review describedmetric backednamed customerproduction runtime claimedsource backedtools describedworkflow describedsoftwareaccuracy improvementtime savedtechnical build writeupcustomer supportit supportextract classify routerag answering