Quality assurance · Production

Adyen augments unit test generation with LLMs, AST parsing, and Knowledge Graphs

The problem

Writing unit tests is time-consuming and manual processes introduce variability; conventional LLMs and RAG systems with vector embeddings cannot capture the interconnected, context-dependent nature of large codebases, making automated unit test generation inaccurate.

First attempt

Conventional RAG systems using vector embeddings fell short for code generation because they capture only surface-level textual similarities, missing state management, dependency hierarchies, and the context-dependent meaning of code constructs.

Workflow diagram · grounded in source
1
Unit test need initiated
trigger
“augmenting the process of creating unit tests”
2
AST parsing of codebase
ai_action
“The AST offers a detailed structural representation of code, transcending the limitations of linear text representations. Each node in the AST corresponds to specific constructs in the source code. By breaking down code into its constitu…”
3
Knowledge Graph construction
ai_action
“Constructing a KG for a codebase involves parsing the code to identify these entities and their interrelations, illuminating dependencies and inheritance hierarchies”
4
Graph database storage and retrieval
integration
“Storing the processed code in a structured way, in a graph database, leads to a leap in our information retrieval capabilities”
5
LLM unit test generation
ai_action
“sending the correct code snippets allows the LLM to truly understand the code and assist the engineer in generating unit tests”
Reported outcome

Combining AST parsing and Knowledge Graphs with LLMs produced a tangible advancement in unit test generation, streamlining the process and bringing new depth and accuracy, and enhancing developer capabilities at Adyen.

Reported metrics
Unit test generation qualitytangible advancement in how we approach software testing
Test generation process efficiencystreamlines the test generation process and brings a new level of depth and accuracy
LLM unit test accuracyelevating the LLM's ability to generate unit tests accurately
Reported stack
LLMsASTKnowledge Graphsgraph databaseRAG
Source
https://www.adyen.com/knowledge-hub/elevating-code-quality-through-llm-integration
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Combining AST parsing and Knowledge Graphs with LLMs produced a tangible advancement in unit test generation, streamlining the process and bringing new depth and accuracy, and enhancing developer capabilities at Adyen.

What tools did this team use?

LLMs, AST, Knowledge Graphs, graph database, RAG.

What results were reported?

Unit test generation quality: tangible advancement in how we approach software testing; Test generation process efficiency: streamlines the test generation process and brings a new level of depth and accuracy; LLM unit test accuracy: elevating the LLM's ability to generate unit tests accurately (source-reported, not independently verified).

What failed first in this deployment?

Conventional RAG systems using vector embeddings fell short for code generation because they capture only surface-level textual similarities, missing state management, dependency hierarchies, and the context-dependent…

How is this quality assurance AI workflow structured?

Unit test need initiated → AST parsing of codebase → Knowledge Graph construction → Graph database storage and retrieval → LLM unit test generation.

Related quality assurance cases