Legal document review · Production

Harvey Builds Enterprise-Grade RAG Systems for Legal and Professional Services Using LanceDB and Postgres

The problem

Deploying enterprise RAG for legal and professional services requires solving for sparse vs. dense retrieval trade-offs, performance and accuracy at massive scale, complex domain-specific data structures, and strict data privacy requirements that prevent sensitive client documents from leaving customer-controlled environments.

Workflow diagram · grounded in source
1
Multi-source document ingestion
integration
“User-uploaded files in Assistant that persist for the duration of a thread (1-50 documents) - User-stored documents in Vault projects persisted for the duration of a long-term project (1,000-10,000 documents) - Private & public third par…”
2
Semantic embedding generation
ai_action
“RAG is powered by semantic search, which uses embedding vectors to capture semantic meaning of text or images and find relevant items for the user”
3
Vector database indexing and storage
integration
“We primarily use LanceDB Enterprise in production because of its strength across latency, accuracy, ingestion throughput, scalability, data privacy, and hosting. Meanwhile, we still appreciate the simplicity and feature-completeness of P…”
4
LLM enrichment via retrieval
ai_action
“enriches LLMs with relevant, up-to-date or user-inputed knowledge, ensuring that the product we provide is grounded and accurate”
5
Domain expert feedback and model tuning
feedback_loop
“continues with their usage and detailed feedback, which is then used to tune the models for further accuracy”
Reported outcome

Harvey now serves users in 45 countries, and its PwC Tax AI collaboration produced a system 91% preferred over off-the-shelf ChatGPT, bringing highly accurate answers to hundreds of professional service firms worldwide.

Reported metrics
Tax AI preference over ChatGPT91%
Postgres PGVector query latency at 500K embeddings<2s P50
LanceDB query latency at 15M rows with metadata filtering<2s P50
Countries served45 countries
Reported stack
LanceDB EnterprisePostgresPGVector
Source
https://www.harvey.ai/blog/enterprise-grade-rag-systems
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Harvey now serves users in 45 countries, and its PwC Tax AI collaboration produced a system 91% preferred over off-the-shelf ChatGPT, bringing highly accurate answers to hundreds of professional service firms worldwide.

What tools did this team use?

LanceDB Enterprise, Postgres, PGVector.

What results were reported?

Tax AI preference over ChatGPT: 91%; Postgres PGVector query latency at 500K embeddings: <2s P50; LanceDB query latency at 15M rows with metadata filtering: <2s P50; Countries served: 45 countries (source-reported, not independently verified).

How is this legal document review AI workflow structured?

Multi-source document ingestion → Semantic embedding generation → Vector database indexing and storage → LLM enrichment via retrieval → Domain expert feedback and model tuning.

Related legal document review cases