PayPay builds GBB RiskBot: RAG-enhanced LLM code review system using historical incident data
PayPay's code review relied entirely on individual reviewer knowledge and ad-hoc knowledge sharing, with no automated system to systematically prevent recurring incidents across services. Knowledge silos, team turnover, and varying reviewer experience led to inconsistent risk assessment.
GBB RiskBot operates across 12 repositories with 380+ total bot runs, at a total cost of $0.59 USD for the measured month, described as very cost-effective compared to the potential cost of production incidents.
The system educates developers and democratizes knowledge across the organization.
Show all 5 reported metrics
Frequently asked questions
What did this team achieve with this AI workflow?
GBB RiskBot operates across 12 repositories with 380+ total bot runs, at a total cost of $0.59 USD for the measured month, described as very cost-effective compared to the potential cost of production incidents.
What tools did this team use?
GBB RiskBot, GitHub Actions, OpenAI embeddings, LangChain, ChromaDB, ChatGPT, gpt-4o-mini, text-embedding-ada-002, RAG, GitHub.
What results were reported?
Database initialization cost (47 incidents): $0.001852; per-PR analysis cost (1 file change): $0.000350; Monthly running cost (12 repositories): $0.59 USD; Total bot runs in measured month: 380+ (source-reported, not independently verified).
How is this quality assurance AI workflow structured?
PR opened triggers bot → Cron job ingests incident data → Incidents embedded into VectorDB → PR similarity search → RAG comment generated → Developer emoji feedback collected.