Back office ops · Production

Ramp Research: In-house AI analyst agent answers 1,800+ data questions per month in Slack

The problem

Data questions at Ramp stacked up behind a single on-call analyst in a #help-data Slack channel, causing decisions to wait hours for answers and leading most questions to go unasked entirely as people hesitated to add to the queue.

First attempt

An initial human-in-the-loop system that pinged domain owners for every in-domain question didn't scale because effort still grew with request volume, reintroducing the core bottleneck.

Workflow diagram · grounded in source
1
Data question asked in Slack
trigger
“answer data* questions directly in Slack, 24/7, in minutes – not hours”
2
Metadata fetch from data stack
ai_action
“At Ramp, that context lives in dbt, Looker, and Snowflake. We aggregated and indexed this metadata, allowing the agent to fetch the right models and form precise queries.”
3
Domain documentation retrieval
ai_action
“These documents were then organized into a file system that Ramp Research can access as needed.”
4
Agentic data inspection
ai_action
“we gave the agent tools to inspect column values, branch, and backtrack – reasoning through the data the way a human analyst would”
5
In-thread CSV answer delivery
output
“We've added in-thread CSV previews so they can validate results without leaving Slack”
6
Stateful thread clarification
feedback_loop
“Making each thread stateful has allowed users to clarify intent, collaborate in threads, and reason through complex problems with Ramp Research as their data expert”
7
Python framework context testing
validation
“we built a Python mini-framework in our dbt project. It asserts not only on the final answer but also the intermediate steps, including expected tool calls, table references, and query shape. This suite enabled us to close the feedback l…”
Reported outcome

Ramp Research answered over 1,800 data questions across 1,200+ conversations with 300 different users and caused a 10-20x increase in questions asked — with 1,476 answered in the beta channel over 4 weeks versus 66 in the old #help-data channel.

Reported metrics
Data questions answeredover 1,800
Conversationsmore than 1,200
Unique users300
Increase in questions asked10-20x increase
Show all 8 reported metrics
data questions answeredover 1,800
conversationsmore than 1,200
unique users300
increase in questions asked10-20x increase
questions answered in beta channel (last 4 weeks)1,476
questions answered in #help-data channel (last 4 weeks)66
beta channel members500+
answer speedminutes – not hours
Reported stack
SlackLookerSnowflakedbtRedashPython
Source
https://engineering.ramp.com/post/meet-ramp-research
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Ramp Research answered over 1,800 data questions across 1,200+ conversations with 300 different users and caused a 10-20x increase in questions asked — with 1,476 answered in the beta channel over 4 weeks versus 66 in…

What tools did this team use?

Slack, Looker, Snowflake, dbt, Redash, Python.

What results were reported?

Data questions answered: over 1,800; Conversations: more than 1,200; Unique users: 300; Increase in questions asked: 10-20x increase (source-reported, not independently verified).

What failed first in this deployment?

An initial human-in-the-loop system that pinged domain owners for every in-domain question didn't scale because effort still grew with request volume, reintroducing the core bottleneck.

How is this back office ops AI workflow structured?

Data question asked in Slack → Metadata fetch from data stack → Domain documentation retrieval → Agentic data inspection → In-thread CSV answer delivery → Stateful thread clarification → Python framework context testing.

Related back office ops cases