Back office ops · Production

Ramp resolves merchant classification corrections in under 10 seconds using an AI agent with RAG and guardrails

The problem

Ramp's automatic transaction-to-merchant matching is imperfect, and manual remediation by support and engineering teams took hours per request. Teams could service only 3% of reports in 2023 and 1.5% in 2024, a share that shrinks further as transaction volume grows.

First attempt

The manual remediation process by customer support, finance, and engineering teams covered only a tiny and declining fraction of requests — 3% in 2023 and 1.5% in 2024 — and cost Ramp hundreds of dollars per resolution.

Workflow diagram · grounded in source
1
User submits reclassification request
trigger
“Ramp users are able to go to a transaction and request a merchant classification to be fixed by providing a new merchant name, website, and category.”
2
LLM builds transaction context
ai_action
“The LLM uses three parts of its context to guide this decision: Transaction card acceptor name and MCC. Extracted merchant names, addresses, and line items from related receipt images. User-provided memos for related transactions.”
3
RAG fetches related merchants
ai_action
“We pull merchants that are similar to the transaction using vector embedding similarity and merchants whose names roughly match the requested merchant name.”
4
LLM selects resolution action
ai_action
“If a request is reasonable, the LLM must choose an action: ➕ Create a new Ramp merchant. 🔧 Update an existing merchant. ↔️ Reassign the transaction to a more fitting merchant.”
5
Guardrails validate and retry
validation
“If the LLM hallucinates, we inform it of its mistake and have it retry until we get a valid response.”
6
Result delivered to user
output
“the user will observe a change on the front end in seconds”
Reported outcome

The AI agent now handles close to 100% of merchant reclassification requests in under 10 seconds at the cost of cents per request, and improves nearly 99% of transaction classifications according to LLM-judge evaluation.

Reported metrics
Repeat correction request ratefewer than 10%
Agent request rejection rate1 out of 4
Transaction classification improvement ratenearly 99%
Reasonable rejection ratenearly two thirds
Show all 11 reported metrics
repeat correction request ratefewer than 10%
agent request rejection rate1 out of 4
transaction classification improvement ratenearly 99%
reasonable rejection ratenearly two thirds
agent resolution timeless than 10 seconds
manual service rate 2023 (baseline)3%
manual service rate 2024 (baseline)1.5%
agent service rate (current)close to 100%
prior resolution costhundreds of dollars
current resolution costcents
prior resolution timehours
Reported stack
LLMembeddingsOLAPRAG
Source
https://builders.ramp.com/post/fixing-merchant-classifications-with-ai
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The AI agent now handles close to 100% of merchant reclassification requests in under 10 seconds at the cost of cents per request, and improves nearly 99% of transaction classifications according to LLM-judge evaluation.

What tools did this team use?

LLM, embeddings, OLAP, RAG.

What results were reported?

Repeat correction request rate: fewer than 10%; Agent request rejection rate: 1 out of 4; Transaction classification improvement rate: nearly 99%; Reasonable rejection rate: nearly two thirds (source-reported, not independently verified).

What failed first in this deployment?

The manual remediation process by customer support, finance, and engineering teams covered only a tiny and declining fraction of requests — 3% in 2023 and 1.5% in 2024 — and cost Ramp hundreds of dollars per resolution.

How is this back office ops AI workflow structured?

User submits reclassification request → LLM builds transaction context → RAG fetches related merchants → LLM selects resolution action → Guardrails validate and retry → Result delivered to user.

Related back office ops cases