Back office ops · Production

Loblaw Digital leverages LLMs to auto-generate dbt documentation across 3,000+ models

The problem

With thousands of dbt models across different lines of business, documentation was a manual, slow, and error-prone process that was frequently omitted, leading to 'documentation debt' and confusion for business users unfamiliar with the data.

Workflow diagram · grounded in source
1
dbt model compiled to JSON
trigger
“When dbt compiles a model, it will transform or compile the model into JSON format”
2
Schema ingested from manifest
integration
“the dbt documentor ingests the dbt model schema directly from a manifest.json file”
3
LLM generates documentation
ai_action
“the LLMs generate comprehensive documentation, detailing everything from describing the SQL queries to the intricate dependencies between models and columns. Importantly, this is achieved without reading actual data, thereby ensuring tha…”
4
AI-generated docs written to YAML
output
“All undocumented models will have [ai-gen] documentation in their YAML file”
Reported outcome

Automated dbt documentation generation using LLMs via dbt documentor increased productivity for analytics engineers, covering over 3,000 live models across dev, data, and business analytics teams at Loblaw.

Reported metrics
Models covered by automated documentationover 3,000 models
Analytics engineer productivityincrease productivity
Manual cross-referencing eliminatedeliminating the need to manually cross-reference multiple files and metadata
Reported stack
dbtdbt documentorVertex AILLMs
Source
https://medium.com/loblaw-digital/leveraging-llms-to-generate-ai-driven-dbt-documentation-c4735faa6ca5
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Automated dbt documentation generation using LLMs via dbt documentor increased productivity for analytics engineers, covering over 3,000 live models across dev, data, and business analytics teams at Loblaw.

What tools did this team use?

dbt, dbt documentor, Vertex AI, LLMs.

What results were reported?

Models covered by automated documentation: over 3,000 models; Analytics engineer productivity: increase productivity; Manual cross-referencing eliminated: eliminating the need to manually cross-reference multiple files and metadata (source-reported, not independently verified).

How is this back office ops AI workflow structured?

dbt model compiled to JSON → Schema ingested from manifest → LLM generates documentation → AI-generated docs written to YAML.

Related back office ops cases