Back office ops · Production

Hermes: Swiggy's in-house Text-to-SQL solution enables natural language data queries in Slack

The problem

Swiggy teams needing specific data had to either find an existing dashboard, write SQL themselves (requiring table knowledge, access, and SQL skills), or submit an analyst request — all taking minutes to days — causing important questions to go unasked or to be answered with proxy or incorrect information.

First attempt

The V1 implementation using GPT 3.5 variants with a kitchen-sink approach — treating all business needs and data as the same — did not perform well against the complexity and volume of Swiggy's tables, columns, and business-specific context.

Workflow diagram · grounded in source
1
User submits question in Slack
trigger
“We use Slack as the entry point, where the users can type in their prompts and receive the final SQL query and its output as response”
2
Lambda middleware formats input
integration
“Middleware (using AWS Lambda): This acts as the intermediary that facilitates communication between the UI and the Gen AI model. This also incorporates a bit of processing / formatting of the inputs before sending them to the Gen AI Model”
3
Metrics retrieval via RAG
ai_action
“This first stage retrieves relevant metrics to understand the user's question. This includes leveraging the knowledge base to fetch associated queries and historical SQL examples via embedding-based vector lookup.”
4
Table and column retrieval
ai_action
“The next stage identifies necessary tables and columns using metadata descriptions, combining LLM querying, filtering, and vector-based lookup. For tables with a large number of columns, multiple LLM calls are made to avoid token limits.…”
5
Structured prompt and SQL generation
ai_action
“All of the gathered information is compiled into a structured prompt. This includes querying the database and collecting data snapshots, which are then sent to the LLM for SQL generation.”
6
Query validation with retries
validation
“The generated SQL query is validated by running it on our database. Errors are relayed back to the LLM for correction with a set number of retries.”
7
Results returned to user
output
“Once an executable SQL is obtained, it is run and the results are relayed back to the user. If retries fail, the query is shared with users along with modification notes.”
8
User feedback collection
feedback_loop
“We collect feedback on the accuracy of the returned query from stakeholders directly within the Slack bot. This helps us evaluate model misses and come up with RCAs and fixes.”
Reported outcome

Hundreds of users across Swiggy answer several thousand queries with an average turnaround time of less than 2 minutes, and the first-shot acceptance rate for generated SQL increased dramatically after the V2 launch.

Reported metrics
Average query turnaround time<2 minutes
User adoptionHundreds of users across the company
Queries answeredseveral thousand queries
first-shot SQL acceptance rateincreased dramatically
Show all 5 reported metrics
average query turnaround time<2 minutes
user adoptionHundreds of users across the company
queries answeredseveral thousand queries
first-shot SQL acceptance rateincreased dramatically
V2 vs V1 performanceperformed significantly better
Reported stack
GPT 3.5GPT-4oSlackAWS LambdaDatabricksSnowflakeAlation
Source
https://bytes.swiggy.com/hermes-a-text-to-sql-solution-at-swiggy-81573fb4fb6e
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Hundreds of users across Swiggy answer several thousand queries with an average turnaround time of less than 2 minutes, and the first-shot acceptance rate for generated SQL increased dramatically after the V2 launch.

What tools did this team use?

GPT 3.5, GPT-4o, Slack, AWS Lambda, Databricks, Snowflake, Alation.

What results were reported?

Average query turnaround time: <2 minutes; User adoption: Hundreds of users across the company; Queries answered: several thousand queries; first-shot SQL acceptance rate: increased dramatically (source-reported, not independently verified).

What failed first in this deployment?

The V1 implementation using GPT 3.5 variants with a kitchen-sink approach — treating all business needs and data as the same — did not perform well against the complexity and volume of Swiggy's tables, columns, and bu…

How is this back office ops AI workflow structured?

User submits question in Slack → Lambda middleware formats input → Metrics retrieval via RAG → Table and column retrieval → Structured prompt and SQL generation → Query validation with retries → Results returned to user → User feedback collection.

Related back office ops cases