Incident management · Production

Agoda improves security incident response with LLM-powered alert triage, phishing classification, and incident report generation

The problem

Agoda's Security IR team faced growing volume across three core workflows — alert triage, phishing review, and incident reporting — that manual processes could not sustain at scale: each alert took 20–40 minutes to handle, phishing reports required individual analyst review despite most being harmless, and incident reports took 5–7 hours to compile from scattered sources.

Workflow diagram · grounded in source
1
Security alert enters pipeline
trigger
“Alerts enter the system through Agoda's existing security pipeline.”
2
RAG queries historical incidents
ai_action
“A Retrieval-Augmented Generation (RAG) system queries a vector database of past incidents and root cause analyses.”
3
LLM triage generates verdict
output
“The model generates a summary, impact assessment, and a verdict.”
4
Phishing email parsed and enriched
integration
“When a phishing report comes in, the email is parsed and enriched with threat intelligence based on its headers and metadata.”
5
LLM classifies phishing email
ai_action
“The LLM receives the full content and context and classifies the email as phishing, spam, or safe.”
6
Automated response sent to reporter
output
“Based on that verdict, an automated response is sent to the reporter in real time.”
7
Multi-source investigation data collected
integration
“Gathering the necessary context often meant reviewing Slack conversations, Jira tickets, Confluence pages, Teams meeting transcripts, and email chains, sometimes across multiple teams.”
8
LLM drafts structured incident report
ai_action
“An LLM summarizes the incident timeline, detection signals, impact, and resolution steps in a structured report format.”
9
Human review and publication
human_review
“A reviewer validates the content and publishes the final version to our internal documentation system.”
Reported outcome

LLM-powered workflows reduced alert analysis time from 20–40 minutes to under 5 minutes with 97%+ human-analyst alignment; cut phishing response time to under 25 seconds with 99%+ classification precision and no known false negatives; and reduced incident report drafting from 5–7 hours to under 10 minutes with 95%+ factual accuracy.

Reported metrics
Alert analysis time before automation20 to 40 minutes
Alert analysis time after automationunder 5 minutes
Alerts handled per 15 daysover 400 alerts every 15 days
LLM-human analyst alignment97%+
Show all 13 reported metrics
alert analysis time before automation20 to 40 minutes
alert analysis time after automationunder 5 minutes
alerts handled per 15 daysover 400 alerts every 15 days
LLM-human analyst alignment97%+
actual phishing rate among user-reported emailsfewer than 2%
phishing classification response timeunder 25 seconds
phishing classification precisionover 99%
phishing false negativesno known false negatives
incident report drafting time before automation5 to 7 hours
incident report drafting time after automationunder 10 minutes
incident report human review time30 minutes
incident report factual accuracyover 95%
phishing response queue backlogvirtually eliminated
Reported stack
LLMsRAGvector databaseSlackJiraConfluenceTeams
Source
https://medium.com/agoda-engineering/improving-security-incident-response-at-agoda-with-large-language-models-78b1f33151e0
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

LLM-powered workflows reduced alert analysis time from 20–40 minutes to under 5 minutes with 97%+ human-analyst alignment; cut phishing response time to under 25 seconds with 99%+ classification precision and no known…

What tools did this team use?

LLMs, RAG, vector database, Slack, Jira, Confluence, Teams.

What results were reported?

Alert analysis time before automation: 20 to 40 minutes; Alert analysis time after automation: under 5 minutes; Alerts handled per 15 days: over 400 alerts every 15 days; LLM-human analyst alignment: 97%+ (source-reported, not independently verified).

How is this incident management AI workflow structured?

Security alert enters pipeline → RAG queries historical incidents → LLM triage generates verdict → Phishing email parsed and enriched → LLM classifies phishing email → Automated response sent to reporter → Multi-source investigation data collected → LLM drafts structured incident report → Human review and publication.

Related incident management cases