How Reforge Built Their AI-Powered Browser Extension Using RAG and Chain of Thought
Reforge users consistently struggled to apply course content to their actual daily work, even after Reforge launched Artifacts to bridge the gap.
The initial simple RAG setup did not produce reliable outputs, and the LLM failed to accurately classify document types, resulting in irrelevant suggestions being shown to beta testers.
After upgrading from simple RAG to a Chain of Thought approach with explicit document classification, the distribution of course recommendations diversified and less than 50% of responses contained no Reforge references, with further increases in references seen after upgrading to version 4.0.
Frequently asked questions
What did this team achieve with this AI workflow?
After upgrading from simple RAG to a Chain of Thought approach with explicit document classification, the distribution of course recommendations diversified and less than 50% of responses contained no Reforge referenc…
What tools did this team use?
Pinecone, LaunchDarkly, Adaline, Segment, MetaBase, Amplitude, Snowflake, Retool, Google Docs, Notion.
What results were reported?
responses with no Reforge references: less than 50%; Reforge references per response after v4.0 upgrade: significant increase in references; RAG output reliability: did not produce reliable outputs consistently (source-reported, not independently verified).
What failed first in this deployment?
The initial simple RAG setup did not produce reliable outputs, and the LLM failed to accurately classify document types, resulting in irrelevant suggestions being shown to beta testers.
How is this back office ops AI workflow structured?
User requests document help → Embedding and similarity search → Document type classification → Parallel suggestion generation → Stream suggestions to extension → Track recommendation distribution.