How Microsoft Security used hybrid ML and LLM to synthesize Post Incident Reviews at scale
Microsoft's Security team had large volumes of unstructured Post Incident Reviews with no scalable way to extract actionable insights, causing recurring themes to go undetected and thorough reviews to be deprioritized as analysts shifted focus to the next urgent incident.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · PIR data collection and prep
A large set of past PIR documents is gathered from recent years, with sensitive details scrubbed before processing.
Tools used
Azure OpenAI Servicek-meansscikit-learnLLMsSLMs
Outcome
The hybrid pipeline significantly reduced the manual effort required to synthesize PIRs, enabling analysts to shift time toward higher-value validation and follow-up. Teams gained a concise digest of recurring themes with remediation directions, improved cross-team knowledge sharing, and more decision-focused security review meetings.
What failed first
Pure ML approaches like topic modeling lacked the contextual depth to distinguish meaningful findings from passing mentions. Pure LLM end-to-end approaches were limited by context windows, cost, and risk of overly general or inaccurate output. Early prompt attempts produced unhelpfully vague summaries.
Results
Time savedshifted analyst time away from repetitive summarization toward higher-value validation