incident_management · saas · workflow
Slack Security Engineering builds multi-agent AI pipeline to streamline on-call security investigations
Slack's Security Engineering team handles a vast volume of security events from diverse data sources and spends on-call shifts doing the laborious work of manually gathering and reviewing alert evidence.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Detection alert triggers investigation
Investigations are triggered by existing detection tools and queued for worker pickup.
Tools used
coding agent CLI
Outcome
Slack's on-call team has shifted from manually gathering evidence to supervising AI investigation teams, with agents making spontaneous and unprompted discoveries and yielding interactive, verifiable investigation reports.
What failed first
A prompt-based prototype produced highly variable results — sometimes jumping to spurious conclusions without adequately questioning its methods — and prompt refinement alone could not achieve the fine-grained control needed for consistent investigation performance.
Results
Time savedbillions
Volumeswitching to supervising investigation teams, rather than doing the laborious work of gathering evidence
Running sinceend of May 2025
Grounding & classification
Source type: technical build writeup
21 fields verified against source quotes, 2 dropped as unverifiable.
agentic workflowmulti agent workflowsummarizationknowledge basefailure mode describednamed customerproduction runtime claimedworkflow describedsoftwareaccuracy improvementemployee productivitytechnical build writeupcompliance monitoringincident managementagentic task executionescalation workflow