It support · Production

Benchling builds an LLM-powered RAG Slackbot for internal Terraform Cloud Q&A

The problem

Benchling engineers regularly asked Terraform Cloud questions in Slack, but finding answers required reading through a lengthy Confluence FAQ or spelunking through old Slack threads, making information retrieval slow and tedious.

Workflow diagram · grounded in source
1
Engineer asks in Slack
trigger
“only the Slack message that explicitly tags our @help-terraform-cloud user is made available”
2
Vector database search
ai_action
“the system runs a search against the vector database to find the most relevant sections of text across all the ingested data sources”
3
LLM synthesizes response
ai_action
“Those query results are then fed into an LLM prompt (we use Claude 3.5 Sonnet v2) to synthesize a helpful response based on the retrieved answers”
4
Answer delivered in Slack
output
“expose this to the user through a familiar Slack interface”
Reported outcome

Benchling built a working RAG-based Slackbot that lets engineers get synthesized answers to Terraform Cloud questions from multiple internal and public data sources, and found the knowledge base setup took minutes instead of days.

Reported metrics
Knowledge base setup timeminutes instead of days
Reported stack
Amazon BedrockOpenSearch ServerlessClaude 3.5 Sonnet v2Amazon Titan Text Embeddings v2AWS LambdaAWS API GatewaySlackConfluence
Source
https://benchling.engineering/building-an-llm-powered-slackbot-557a6241e993
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Benchling built a working RAG-based Slackbot that lets engineers get synthesized answers to Terraform Cloud questions from multiple internal and public data sources, and found the knowledge base setup took minutes ins…

What tools did this team use?

Amazon Bedrock, OpenSearch Serverless, Claude 3.5 Sonnet v2, Amazon Titan Text Embeddings v2, AWS Lambda, AWS API Gateway, Slack, Confluence.

What results were reported?

Knowledge base setup time: minutes instead of days (source-reported, not independently verified).

How is this it support AI workflow structured?

Engineer asks in Slack → Vector database search → LLM synthesizes response → Answer delivered in Slack.

Related it support cases