customer_support · manufacturing · workflow
From RAG to Fabric: Lessons Learned from Building Real-World RAGs at GenAIIC — Part 1
AWS customers have high demand for RAG chatbots that can extract information from massive, heterogeneous knowledge bases, but building effective RAG solutions is difficult because retrieval quality is almost always the primary failure point and evaluation requires significant human effort.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · User query submitted
The workflow begins with a user's question submitted to the RAG system.
Tools used
OpenSearchOpenSearch ServerlessAmazon Bedrock Knowledge BasesAmazon S3Amazon TitanAmazon BedrockAnthropic's ClaudeAmazon TextractAWS LambdaTextractorCohere EmbedAmazon Titan Text Embeddings
Outcome
(not stated)
What failed first
Two main RAG failure modes are identified from real-world experience: relevant information is not retrieved (causing the FM to hallucinate or use its own knowledge), or relevant information is buried in excessive irrelevant data (causing the FM to mix up sources and produce wrong answers).
Results
Running sinceMay 2023
Grounding & classification
Source type: technical build writeup
29 fields verified against source quotes, 3 dropped as unverifiable.
chatbotknowledge searchragsummarizationknowledge basefailure mode describedhuman review describedproduction runtime claimedsource backedtools describedworkflow describedfinancial servicesmanufacturingtechnical build writeupcustomer supportit supportrag answering