back_office_ops · public · workflow
T-RAG: on-premise RAG with finetuned LLM and tree-based entity context for governance document QA
A large non-profit organization needed secure, on-premise question answering over its private governance documents but could not use API-based LLMs due to data leakage risks, faced limited computational resources, and required reliably correct responses to organizational queries.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · PDF document ingested
The original PDF governance document is parsed into text format using the LangChain library.
Tools used
LangChainLlama-2QLoRAvector database
Outcome
T-RAG, combining RAG with a finetuned open-source LLM and a tree structure for organizational entity hierarchies, performs better than a simple RAG or finetuning implementation alone.
Results
Volume1,614 question and answer pairs
Grounding & classification
Source type: technical build writeup
20 fields verified against source quotes, 3 dropped as unverifiable.
document aiknowledge searchragknowledge basepolicy documentfailure mode describedhuman review describedproduction runtime claimedworkflow describednonprofitaccuracy improvementtechnical build writeupback office opsrag answering