Quality assurance · Production

Rovo Dev Code Reviewer reduces PR cycle time by 30.8% at Atlassian

The problem

Manual code review at Atlassian was time-consuming, created deployment bottlenecks, and exposed code to human error, slowing engineering velocity as the product portfolio grew in complexity.

Workflow diagram · grounded in source

First-pass review triggered

trigger

“Rovo Dev works alongside our engineers, performing the initial "first pass" review”

Context-aware LLM comment generation

ai_action

“We rely on a zero-shot structured prompting approach that is augmented with readily-available contextual information (i.e., pull request and Jira issue information) in a structured design with persona, chain-of-thought, and review guidel…”

LLM-as-a-Judge factual check

validation

“a dedicated "LLM-as-a-Judge" component based on a cheaper model (gpt-4o-mini). This judge acts as a gatekeeper, reviewing every generated comment for factual correctness against the code change. If a comment is hallucinated or factually …”

ModernBERT actionability filter

validation

“a ModernBERT-based comment quality check on actionability”

Human engineer reviews suggestions

human_review

“the final decision to accept or decline a suggestion rests solely with the human reviewer”

Human feedback improves AI

feedback_loop

“Human feedback is used to refine and improve the AI over time”

Reported outcome

Rovo Dev Code Reviewer reduced median PR cycle time by 30.8% and cut human-written review comments by 35.6%, with 38.70% of its generated comments leading directly to code changes; engineering feedback was overwhelmingly positive.

Reported metrics

PR cycle time reduction30.8%

Human-written review comments reduced35.6%

Rovo Dev comment resolution rate38.70%

Human comment resolution rate (baseline)44.45%

Show all 5 reported metrics

PR cycle time reduction30.8%

human-written review comments reduced35.6%

Rovo Dev comment resolution rate38.70%

human comment resolution rate (baseline)44.45%

engineer satisfactionoverwhelmingly positive

Reported stack

Rovo DevClaude 3.5 Sonnetgpt-4o-miniModernBERTBitbucketJira

Source

https://www.atlassian.com/blog/artificial-intelligence/developer-productivity-improved-with-rovo-dev

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Rovo Dev Code Reviewer reduced median PR cycle time by 30.8% and cut human-written review comments by 35.6%, with 38.70% of its generated comments leading directly to code changes; engineering feedback was overwhelmin…

What tools did this team use?

Rovo Dev, Claude 3.5 Sonnet, gpt-4o-mini, ModernBERT, Bitbucket, Jira.

What results were reported?

PR cycle time reduction: 30.8%; Human-written review comments reduced: 35.6%; Rovo Dev comment resolution rate: 38.70%; Human comment resolution rate (baseline): 44.45% (source-reported, not independently verified).

How is this quality assurance AI workflow structured?

First-pass review triggered → Context-aware LLM comment generation → LLM-as-a-Judge factual check → ModernBERT actionability filter → Human engineer reviews suggestions → Human feedback improves AI.