compliance_monitoring · finance · workflow
Ramp builds in-house RAG model to standardize industry classification on NAICS codes
Ramp's industry classification relied on a homegrown system stitched together from third-party data, Sales-entered data, and customer self-reporting, producing multiple non-auditable sources of truth and overly broad labels that made compliance, credit risk profiling, and sales targeting unreliable. Different teams used incompatible taxonomies, requiring many-to-many mappings between over 100 internal levels and thousands of NAICS or SIC codes.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · New business enters pipeline
Internal services handle embeddings for new businesses and LLM prompt evaluations.
Tools used
RAGLLMClickhouseKafka
Outcome
Migrating to NAICS codes via an in-house RAG model improved classification accuracy, simplified cross-team workflows, and made the system fully auditable and tunable. Hyperparameter optimization yielded performance boosts of up to 60% in acc@k and 5%-15% improvement in fuzzy accuracy.
Results
Volumeup to 60%
Grounding & classification
Source type: technical build writeup
27 fields verified against source quotes.
document classificationragknowledge basebuilder submittedfailure mode describedmetric backednamed customerproduction runtime claimedtools describedworkflow describedfinancial servicesaccuracy improvementemployee productivityerror reductiontechnical build writeupback office opscompliance monitoringsales opsextract classify routerag answering