Incident management · Production

Google uses LLMs to cut security incident summary drafting time by 51%

The problem

Writing security and privacy incident summaries for executives, leads, and partner teams was tedious and time-consuming, estimated at nearly an hour per summary and multiple hours for complex communications.

First attempt

Early LLM prompt versions produced summaries that were too long, missed key facts like impact and mitigation, had inconsistent writing style, included irrelevant email thread content, and showed hallucinations on hypotheses.

Workflow diagram · grounded in source
1
Incident reported
trigger
“when an incident is reported, our Detection & Response teams work to restore normal service as quickly as possible”
2
Incident data structured
integration
“we first replaced long and noisy sections of codes/logs by self-closing tags (<Code Section/> and <Logs/>) both to keep the structure while saving tokens for more important facts and to reduce risk of hallucinations. During prompt engine…”
3
Token size gate
validation
“If the input size is smaller than 200 tokens, we won't call the LLM for a summary and let the humans write it”
4
LLM generates summary draft
ai_action
“For the final prompt, we inserted 2 human-crafted summary examples and introduced a <Good Summary> tag to highlight high quality summaries but also to tell the model to immediately start with the summary without first repeating the task …”
5
Human review and acceptance
human_review
“A human user can then either accept the summary and have it added to the incident, do manual changes to the summary and accept it, or discard the draft and start again”
6
Summary added to incident
output
“accept the summary and have it added to the incident”
Reported outcome

Using generative AI, Google's team writes summaries 51% faster with improved quality rated 10% higher than human-written equivalents, and reduced executive communication drafting time by 53%.

Reported metrics
Summary writing time saved51%
LLM summary quality vs human10% higher
Executive communication drafting time reduction53%
Reported stack
LLMsgenerative AI
Source
https://security.googleblog.com/2024/04/accelerating-incident-response-using.html
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Using generative AI, Google's team writes summaries 51% faster with improved quality rated 10% higher than human-written equivalents, and reduced executive communication drafting time by 53%.

What tools did this team use?

LLMs, generative AI.

What results were reported?

Summary writing time saved: 51%; LLM summary quality vs human: 10% higher; Executive communication drafting time reduction: 53% (source-reported, not independently verified).

What failed first in this deployment?

Early LLM prompt versions produced summaries that were too long, missed key facts like impact and mitigation, had inconsistent writing style, included irrelevant email thread content, and showed hallucinations on hypo…

How is this incident management AI workflow structured?

Incident reported → Incident data structured → Token size gate → LLM generates summary draft → Human review and acceptance → Summary added to incident.

Related incident management cases