Quality assurance · Production

PerfInsights: Uber's GenAI system detects Go performance antipatterns with 93.10% engineering time savings

The problem

Uber's top 10 Go services cost more than multi-million dollars in compute per month, while traditional performance tuning demanded deep expertise and days or weeks of profiling work, making systematic optimization non-trivial for most engineering teams.

First attempt

Initial single-shot LLM-based antipattern detection produced inconsistent and unreliable results with hallucinations and often non-runnable code, and false positive rates exceeded 80%.

Workflow diagram · grounded in source
1
Production profile collection
Trigger
PerfInsights leverages CPU and memory profiles from production services using Uber's daily fleet-wide profiler during peak traffic periods.
▾ source quote
“PerfInsights leverages CPU and memory profiles from production services using Uber's daily fleet-wide profiler during peak traffic periods”
2
Hotpath function identification
Validation
PerfInsights identifies the top 30 most expensive functions based on flat CPU usage.
▾ source quote
“it identifies the top 30 most expensive functions based on flat CPU usage”
3
Static noise filtering
Validation
A static filter excludes open-source dependencies and internal runtime functions from analysis.
▾ source quote
“PerfInsights applies a static filter that excludes open-source dependencies and internal runtime functions”
4
LLM antipattern detection
Ai action
PerfInsights passes full source code and a list of antipatterns to a large language model for analysis.
▾ source quote
“PerfInsights passes their full source codes and a list of antipatterns to a large language model (LLM) for analysis. By combining profiling context with pattern awareness, the model can pinpoint inefficient constructs with high precision. For example, if a function …”
5
LLM jury validation
Validation
A jury of large language models independently assesses whether each detected antipattern is present and whether the suggested optimization is valid.
▾ source quote
“PerfInsights leverages a jury of large language models to validate each detected antipattern. These models independently assess whether an antipattern is present and whether the suggested optimization is valid.”
6
LLMCheck rule-based validation
Validation
LLMCheck runs domain-specific rule-based validators to evaluate LLM responses and catch false positives.
▾ source quote
“PerfInsights employs a second layer of verification via LLMCheck by running through several domain-specific rule-based validators to evaluate LLM responses”
7
Confidence score output
Output
The final output includes a confidence score of the function's optimisability and suggested improvements.
▾ source quote
“The final output includes a confidence score of the function's optimisability and suggested improvements”
8
Optix downstream integration
Integration
Validated suggestions flow directly into Optix, Uber's continuous code optimization tool.
▾ source quote
“validated suggestions flowing directly into Optix, Uber's continuous code optimization tool”
9
LLMCheck accuracy feedback loop
Feedback loop
Dashboards powered by LLMCheck provide teams with visibility into detection accuracy, error patterns, and antipattern frequency.
▾ source quote
“Dashboards powered by LLMCheck provide teams with visibility into detection accuracy, error patterns, and antipattern frequency”
Reported outcome

PerfInsights reduced engineering time to detect and fix performance issues by 93.10%, achieved a 33.5% reduction in antipatterns over four months, and merged hundreds of diffs into Uber's Go monorepo.

Reported metrics
monthly compute spend (top 10 Go services)multi-million dollars
Task completion timeTasks that once required days now take hours
Diffs generated and mergedhundreds of diffs already generated and merged
average validated detections (February)265
Show all 20 reported metrics
monthly compute spend (top 10 Go services)multi-million dollars
task completion timeTasks that once required days now take hours
diffs generated and mergedhundreds of diffs already generated and merged
average validated detections (February)265
single-day detection high500
validated detections (June)176
antipattern reduction over four months33.5%
sustained antipattern reduction (Feb to June)89
manual effort for 5 antipatterns (historical baseline)∼320 hours
manual effort for 11 antipatterns (historical baseline)∼160 hours
manual effort for 6-month optimization project∼960 hours
total manual baseline hours (handful of cases)over 1,400 hours
projected annual antipattern reduction267
manual hours to address 267 antipatternsapproximately 3,800 hours
engineering time savings93.10%
detection time per issue (before vs after)from 14.5 hours to almost 1 hour
hallucination rate reductionmore than 80%
false positive rate improvement (dual-validation)from over 80% to the low teens
false positive reduction via prompt engineering80%
first cost-saving difffirst 5 digits saving diff
Reported stack
LLMLLMCheckOptix
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://www.uber.com/en-IN/blog/perfinsights/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

PerfInsights reduced engineering time to detect and fix performance issues by 93.10%, achieved a 33.5% reduction in antipatterns over four months, and merged hundreds of diffs into Uber's Go monorepo.

What tools did this team use?

LLM, LLMCheck, Optix.

What results were reported?

monthly compute spend (top 10 Go services): multi-million dollars; Task completion time: Tasks that once required days now take hours; Diffs generated and merged: hundreds of diffs already generated and merged; average validated detections (February): 265 (source-reported, not independently verified).

What failed first in this deployment?

Initial single-shot LLM-based antipattern detection produced inconsistent and unreliable results with hallucinations and often non-runnable code, and false positive rates exceeded 80%.

How is this quality assurance AI workflow structured?

Production profile collection → Hotpath function identification → Static noise filtering → LLM antipattern detection → LLM jury validation → LLMCheck rule-based validation → Confidence score output → Optix downstream integration → LLMCheck accuracy feedback loop.

Related quality assurance cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Quality assurance cases →