Incident management · Production

You can't vibe code a prompt: incident.io's AI agent for Slack-based incident investigation

The problem

incident.io's AI agent for scanning Slack during incidents was misclassifying messages — confidently surfacing irrelevant discussions to responders. Attempts to fix the prompt by letting an LLM optimize it autonomously produced an overfitted prompt that memorized eval examples instead of learning to generalize.

First attempt

Autonomous prompt optimization by Claude Code overfitted to the eval suite: all tests passed by hardcoding specific examples in the prompt, but the prompt had ballooned to 7× its original size and deleting those hardcoded examples restored the original failures, demonstrating no genuine generalization.

Workflow diagram · grounded in source

Incident alert triggers agent

trigger

“if an alert comes in about database CPU being high, and someone recently posted a message in #engineering about how to handle situations like this, our agent would highlight this”

Scan Slack for relevant discussions

ai_action

“scan Slack for relevant discussions that might clue you in on what's gone wrong”

LLM classifies message relevance

ai_action

“Assign one of these confidence levels: * certain: There is clear evidence that the message would help responders resolve the incident * potentially_related: The message might help responders resolve the incident, but there is some ambigu…”

Surface certain discussions to responders

output

“If an LLM is certain that there's a useful discussion, we surface it to responders”

Eval suite gates prompt changes

validation

“build up a set of evals which act as unit tests for the prompt—test cases along with their expected outcome”

Human-authored prompt refinement

human_review

“you still have to write your prompts. The best way to improve a prompt like this one? Go through historical cases and describe—like you're explaining to a five-year-old—why certain messages are useful”

LLM prompt health check

validation

“We run a health check which reports on these sort of errors. This means LLMs nit-pick our prompts so we don't have to.”

Interaction scoring for performance tracking

feedback_loop

“we let a cheap LLM classify interactions, score them across key metrics, and assign an overall rating. Calculating scores and interaction types also means that you can track how your performance changes over time.”

Reported outcome

incident.io recommends human-controlled prompt engineering: build eval suites from historical cases, make intentional refinements based on human understanding, and use LLMs only for specific subtasks like eval generation, prompt health checks, and interaction scoring.

Reported metrics

Prompt size increase from autonomous optimization (vibe coding experiment)7× its original size

Eval pass rate after autonomous optimization (vibe coding experiment)100%

Claude Code iterations to pass all evals (vibe coding experiment)four iterations

Relative cost of o4 vs mini~16x more expensive than mini

Reported stack

Claude CodeClaude4o-miniSlack

Source

https://incident.io/building-with-ai/you-cant-vibe-code-a-prompt

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

incident.io recommends human-controlled prompt engineering: build eval suites from historical cases, make intentional refinements based on human understanding, and use LLMs only for specific subtasks like eval generat…

What tools did this team use?

Claude Code, Claude, 4o-mini, Slack.

What results were reported?

Prompt size increase from autonomous optimization (vibe coding experiment): 7× its original size; Eval pass rate after autonomous optimization (vibe coding experiment): 100%; Claude Code iterations to pass all evals (vibe coding experiment): four iterations; Relative cost of o4 vs mini: ~16x more expensive than mini (source-reported, not independently verified).

What failed first in this deployment?

How is this incident management AI workflow structured?

Incident alert triggers agent → Scan Slack for relevant discussions → LLM classifies message relevance → Surface certain discussions to responders → Eval suite gates prompt changes → Human-authored prompt refinement → LLM prompt health check → Interaction scoring for performance tracking.