Incident management · Production

InfoQ Panel: DevOps Modernization with AI Agents — Intelligent Observability, Log Triage, and Automated Remediation

The problem

DevOps and SRE teams waste significant human attention on manual log triage, alert noise, and incident communication lacking context — engineers spend time determining whether a signal is real, new, or customer-impacting rather than making decisions.

First attempt

An AI-driven canary rollout analysis system consistently missed failures visible only in shadow canaries because it lacked complete context of the deployment traffic shape — it was reasoning correctly over an incomplete picture of production, not a model error.

Workflow diagram · grounded in source

Alert triggers investigation

trigger

“I receive an alert. I go into the log. I check the log, try to understand it”

AI analyzes logs

ai_action

“I give it the logs in a file, then it figures out what might be wrong, then tells me which part in the application needs to be fixed and what I need to do to fix it”

AI drafts incident timeline

output

“if they can write a first draft of incident timelines. Like what happened, correlate the logs to the deploys, or maybe even just summarize what changed in the last 24 hours”

AI generates fix hypotheses

ai_action

“give me three options of how you would solve this. Give me three options of why it will fail”

AI creates pull request

output

“ask it to get the repository and ask it to fix it and create a pull request”

Engineer reviews and approves PR

human_review

“when somebody comes to review that PR, they can say, ok, that makes sense, the code looks correct, great, approve”

Continuous verification monitors deploy

validation

“it learns what good looks like for your application when it's deployed. It learns that, you get these errors in the log, but you get those every time you deploy. That's not something you need to worry about.”

Reported outcome

AI assistance reduced a real incident resolution from hours to under 15 minutes and shortened outage durations by guiding teams through triage faster; trust in AI automation was built by requiring explainability before autonomous action.

Reported metrics

Incident resolution timeless than 15 minutes (vs at least 5 hours if not more)

Outage duration (illustrative)30 minutes or 5-minute outage (vs one-day outage)

Triage time (hypothetical)5 minutes (vs 15 hours hypothetically)

Developer time saved per query (illustrative)massive time-saving

Reported stack

SlackConfluenceLLMRAGHarness

Source

https://www.infoq.com/presentations/devops-modernization-ai-agents

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

AI assistance reduced a real incident resolution from hours to under 15 minutes and shortened outage durations by guiding teams through triage faster; trust in AI automation was built by requiring explainability befor…

What tools did this team use?

Slack, Confluence, LLM, RAG, Harness.

What results were reported?

Incident resolution time: less than 15 minutes (vs at least 5 hours if not more); Outage duration (illustrative): 30 minutes or 5-minute outage (vs one-day outage); Triage time (hypothetical): 5 minutes (vs 15 hours hypothetically); Developer time saved per query (illustrative): massive time-saving (source-reported, not independently verified).

What failed first in this deployment?

How is this incident management AI workflow structured?

Alert triggers investigation → AI analyzes logs → AI drafts incident timeline → AI generates fix hypotheses → AI creates pull request → Engineer reviews and approves PR → Continuous verification monitors deploy.