Incident management · Production

GraphRAG for on-call incident resolution: lessons from production deployment at Microsoft

The problem

On-call engineers responding to high-severity incidents needed to answer time-critical questions spanning multiple documents — incident reports, runbooks, architecture docs, and postmortems — but Vector RAG could retrieve individually relevant chunks without modeling or traversing the relationships among them.

First attempt

Moving GraphRAG from prototype to production exposed hard operational challenges: indexing cost spikes from LLM-heavy extraction stages, complex update management that can cause graph drift, multi-dimensional evaluation requirements, and infrastructure gaps that most GraphRAG libraries do not address.

Workflow diagram · grounded in source
1
Engineer asks during incident
trigger
“During high-severity incidents, these engineers ask time-critical questions that span multiple documents and require understanding how components relate to each other”
2
Route to GraphRAG vs Vector RAG
routing
“we recommend keeping Vector RAG as the default and treating GraphRAG as an opt-in backend for queries that clearly need cross-document or cross-entity reasoning”
3
Knowledge graph reasoning
ai_action
“GraphRAG constructs a knowledge graph from your corpus and reasons over the relationships among entities”
4
Answer delivered to engineer
output
“we needed answers that connected dots across incident reports, runbooks, architecture docs, and postmortems”
5
Evaluation suite gates changes
feedback_loop
“every meaningful change (prompt, graph config, retrieval logic, model version) runs through a repeatable eval suite, and we deploy only if it clears predefined criteria — e.g., no regression versus a baseline (or a clear win in pairwise …”
Reported outcome

The team successfully deployed GraphRAG as an additive capability alongside Vector RAG for incident resolution, and established practical operational guidelines: use GraphRAG selectively for relationship-heavy queries, be deliberate about graph scope, and treat builds as a managed service with cost models, monitoring, and repeatable evaluation.

Reported metrics
Share of build cost from entity extraction and community summarizationapproximately 90 percent
Reported stack
GraphRAGBenchmarkQEDAzure ML JobsPostgreSQLpgvector
Source
https://medium.com/data-science-at-microsoft/graphrag-beyond-the-demo-lessons-from-the-trenches-add83180f849
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The team successfully deployed GraphRAG as an additive capability alongside Vector RAG for incident resolution, and established practical operational guidelines: use GraphRAG selectively for relationship-heavy queries…

What tools did this team use?

GraphRAG, BenchmarkQED, Azure ML Jobs, PostgreSQL, pgvector.

What results were reported?

Share of build cost from entity extraction and community summarization: approximately 90 percent (source-reported, not independently verified).

What failed first in this deployment?

Moving GraphRAG from prototype to production exposed hard operational challenges: indexing cost spikes from LLM-heavy extraction stages, complex update management that can cause graph drift, multi-dimensional evaluati…

How is this incident management AI workflow structured?

Engineer asks during incident → Route to GraphRAG vs Vector RAG → Knowledge graph reasoning → Answer delivered to engineer → Evaluation suite gates changes.

Related incident management cases