Back office ops · Production

Building a RAG system for internal engineering knowledge search from 1 TB of project documents

The problem

An engineering company needed an internal natural language chat tool to search across nearly a decade of project history totaling 1 TB of mixed technical documents—including OrcaFlex simulation files central to the offshore industry—without relying on external APIs for confidentiality reasons.

First attempt

LlamaIndex overflowed the laptop's RAM when processing large non-text files; a custom checkpoint system suffered data corruption and was too slow; the laptop's integrated GPU required 4-5 hours per 500 MB; and the production VM had only 100 GB of disk, far short of the full document corpus.

Workflow diagram · grounded in source
1
Document filtering by extension and pattern
Validation
A filtering system excluded files by extension and name patterns before indexing to prevent non-text assets from entering the pipeline.
▾ source quote
“I added a filtering system to the pipeline that excluded files by extension and by name patterns (simulation files, numerical results, etc.)”
2
Batch embedding and vector storage
Ai action
Documents are processed in batches of 150 files, embeddings are generated, and stored directly in ChromaDB.
▾ source quote
“a batch pipeline that processed 150 files at a time, generated their embeddings, and stored them directly in ChromaDB. This allowed indexing the 451GB of documents across multiple sessions, with checkpoints, without losing progress on interruptions”
3
User submits natural language question
Trigger
Users submit questions via Streamlit's native Q&A widget, in the style of any current chat interface for interacting with an AI.
▾ source quote
“It also has a native widget for Q&A, in the style of any current chat for interacting with an AI”
4
RAG query and LLM response generation
Ai action
LlamaIndex queries ChromaDB for relevant document vectors and Ollama generates the natural language response.
▾ source quote
“With Flask I built a simple API to access LlamaIndex, which in turn queried ChromaDB and Ollama”
5
Response with source document references
Output
The system returns a formatted response showing the information sources—the documents used to generate the answer.
▾ source quote
“the system must show the information sources, that is, the documents used to generate the answer”
6
Document download via SAS token from Azure
Integration
For each document cited in a response, the system generates a download link with a SAS token for direct retrieval from Azure Blob Storage.
▾ source quote
“For each document cited in a response, the system generates a download link with a SAS token that allows the user to download it directly from the cloud.”
Reported outcome

The RAG system reached production with 738,470 vectors and a 54 GB index in ChromaDB, achieved a 54% reduction in files to index through filtering, and is described as fast, reliable, and useful for colleagues.
The GPU indexing phase cost 184 euros on Hetzner.

Reported metrics
Files to index reduction54%
vectors in ChromaDB index738,470
ChromaDB index size54GB
GPU rental cost on Hetzner184 euros
Show all 10 reported metrics
files to index reduction54%
vectors in ChromaDB index738,470
ChromaDB index size54GB
GPU rental cost on Hetzner184 euros
CPU indexing speed before GPU4-5 hours per 500 MB
total GPU indexing durationbetween 2 and 3 weeks
document corpus size451GB
production VM disk space100 GB
original project data volume1 TB
system qualityfast, reliable, and above all useful
Reported stack
Ollamanomic-embed-textLlamaIndexChromaDBSQLiteFlaskStreamlitPythonAzure Blob StorageDocker ComposeNVIDIA Container ToolkitGunicornllama3.2:3b
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://en.andros.dev/blog/aa31d744/from-zero-to-a-rag-system-successes-and-failures/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The RAG system reached production with 738,470 vectors and a 54 GB index in ChromaDB, achieved a 54% reduction in files to index through filtering, and is described as fast, reliable, and useful for colleagues.

What tools did this team use?

Ollama, nomic-embed-text, LlamaIndex, ChromaDB, SQLite, Flask, Streamlit, Python, Azure Blob Storage, Docker Compose.

What results were reported?

Files to index reduction: 54%; vectors in ChromaDB index: 738,470; ChromaDB index size: 54GB; GPU rental cost on Hetzner: 184 euros (source-reported, not independently verified).

What failed first in this deployment?

LlamaIndex overflowed the laptop's RAM when processing large non-text files; a custom checkpoint system suffered data corruption and was too slow; the laptop's integrated GPU required 4-5 hours per 500 MB; and the pro…

How is this back office ops AI workflow structured?

Document filtering by extension and pattern → Batch embedding and vector storage → User submits natural language question → RAG query and LLM response generation → Response with source document references → Document download via SAS token from Azure.

Related back office ops cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Back office ops cases →