Incident management · Production

Agoda improves security incident response with LLM-powered alert triage, phishing classification, and incident report generation

The problem

Agoda's Security IR team faced growing volume across three core workflows — alert triage, phishing review, and incident reporting — that manual processes could not sustain at scale: each alert took 20–40 minutes to handle, phishing reports required individual analyst review despite most being harmless, and incident reports took 5–7 hours to compile from scattered sources.

Workflow diagram · grounded in source
1
Security alert enters pipeline
Trigger
Alerts enter the system through Agoda's existing security pipeline.
▾ source quote
“Alerts enter the system through Agoda's existing security pipeline.”
2
RAG queries historical incidents
Ai action
A Retrieval-Augmented Generation (RAG) system queries a vector database of past incidents and root cause analyses.
▾ source quote
“A Retrieval-Augmented Generation (RAG) system queries a vector database of past incidents and root cause analyses.”
3
LLM triage generates verdict
Output
The model generates a summary, impact assessment, and a verdict.
▾ source quote
“The model generates a summary, impact assessment, and a verdict.”
4
Phishing email parsed and enriched
Integration
When a phishing report comes in, the email is parsed and enriched with threat intelligence based on its headers and metadata.
▾ source quote
“When a phishing report comes in, the email is parsed and enriched with threat intelligence based on its headers and metadata.”
5
LLM classifies phishing email
Ai action
The LLM receives the full content and context and classifies the email as phishing, spam, or safe.
▾ source quote
“The LLM receives the full content and context and classifies the email as phishing, spam, or safe.”
6
Automated response sent to reporter
Output
Based on that verdict, an automated response is sent to the reporter in real time.
▾ source quote
“Based on that verdict, an automated response is sent to the reporter in real time.”
7
Multi-source investigation data collected
Integration
Gathering the necessary context often meant reviewing Slack conversations, Jira tickets, Confluence pages, Teams meeting transcripts, and email chains.
▾ source quote
“Gathering the necessary context often meant reviewing Slack conversations, Jira tickets, Confluence pages, Teams meeting transcripts, and email chains, sometimes across multiple teams.”
8
LLM drafts structured incident report
Ai action
An LLM summarizes the incident timeline, detection signals, impact, and resolution steps in a structured report format.
▾ source quote
“An LLM summarizes the incident timeline, detection signals, impact, and resolution steps in a structured report format.”
9
Human review and publication
Human review
A reviewer validates the content and publishes the final version to the internal documentation system.
▾ source quote
“A reviewer validates the content and publishes the final version to our internal documentation system.”
Reported outcome

LLM-powered workflows reduced alert analysis time from 20–40 minutes to under 5 minutes with 97%+ human-analyst alignment; cut phishing response time to under 25 seconds with 99%+ classification precision and no known false negatives; and reduced incident report drafting from 5–7 hours to under 10 minutes with 95%+ factual accuracy.

Reported metrics
Alert analysis time before automation20 to 40 minutes
Alert analysis time after automationunder 5 minutes
Alerts handled per 15 daysover 400 alerts every 15 days
LLM-human analyst alignment97%+
Show all 13 reported metrics
alert analysis time before automation20 to 40 minutes
alert analysis time after automationunder 5 minutes
alerts handled per 15 daysover 400 alerts every 15 days
LLM-human analyst alignment97%+
actual phishing rate among user-reported emailsfewer than 2%
phishing classification response timeunder 25 seconds
phishing classification precisionover 99%
phishing false negativesno known false negatives
incident report drafting time before automation5 to 7 hours
incident report drafting time after automationunder 10 minutes
incident report human review time30 minutes
incident report factual accuracyover 95%
phishing response queue backlogvirtually eliminated
Reported stack
LLMsRAGvector databaseSlackJiraConfluenceTeams
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://medium.com/agoda-engineering/improving-security-incident-response-at-agoda-with-large-language-models-78b1f33151e0
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

LLM-powered workflows reduced alert analysis time from 20–40 minutes to under 5 minutes with 97%+ human-analyst alignment; cut phishing response time to under 25 seconds with 99%+ classification precision and no known…

What tools did this team use?

LLMs, RAG, vector database, Slack, Jira, Confluence, Teams.

What results were reported?

Alert analysis time before automation: 20 to 40 minutes; Alert analysis time after automation: under 5 minutes; Alerts handled per 15 days: over 400 alerts every 15 days; LLM-human analyst alignment: 97%+ (source-reported, not independently verified).

How is this incident management AI workflow structured?

Security alert enters pipeline → RAG queries historical incidents → LLM triage generates verdict → Phishing email parsed and enriched → LLM classifies phishing email → Automated response sent to reporter → Multi-source investigation data collected → LLM drafts structured incident report → Human review and publication.

Related incident management cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Incident management cases →