quality_assurance · workflow
DREAM: A blueprint for distributed RAG experimentation using Ray, LlamaIndex, Ragas, MLFlow and MinIO on Kubernetes
Evaluating which combination of LLMs, embedding models, and retrieval methods works best for a RAG use case is time-consuming when combinations must be explored sequentially.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Load and store unstructured data
PDFs are downloaded and pushed to MinIO S3 object storage using boto3.
Tools used
Ray TunePostgreSQLboto3gpt-3.5-turbogpt-4text-embedding-3-smalltext-embedding-3-large
Outcome
DREAM demonstrates how distributed RAG experimentation, evaluation, and tracking can be conducted using open-source technologies, enabling comparison and selection of the best-performing RAG parameter combination.
Grounding & classification
Source type: technical build writeup
14 fields verified against source quotes, 9 dropped as unverifiable.
data extractionragknowledge basesource backedworkflow describedaccuracy improvementtechnical build writeupquality assurancerag answering