Ecommerce ops · Production

Rufus scales conversational shopping for 250M+ Amazon customers using Amazon Bedrock

The problem

Rufus was initially built on a custom in-house LLM optimized for the shopping domain, but training iterations took weeks or months, making it impossible to rapidly adopt advanced reasoning and larger context window capabilities as frontier models evolved.

First attempt

Off-the-shelf models evaluated before the custom LLM performed poorly in shopping domain evaluations, and larger third-party models added unacceptable latency and cost penalties.

Workflow diagram · grounded in source
1
Customer shopping query
trigger
“a customer may want to know something factual about the shoes they're considering and ask, "are these shoes waterproof?"”
2
Per-query model selection
routing
“we can evaluate and select the optimal model for each query type, balancing answer quality, latency, and engagement”
3
Context injection
ai_action
“if we know a customer is asking a question about their previous orders, we can provide their order history to the initial inference request of the model”
4
Web grounding for accuracy
ai_action
“Amazon Nova Web Grounding because it can interact with web browsers to retrieve and cite authoritative internet sources, resulting in significantly reduced answer defects and improved accuracy and customer trust”
5
Agentic tool execution
ai_action
“Rufus can dynamically call services as tools to provide personalized, real-time, accurate information or take actions on behalf of the user. When a customer asks Rufus about product availability, pricing, or specifications, Rufus goes fa…”
6
Conversational response delivered
output
“Rufus uses agentic AI capabilities to automatically add products to the cart for quick review and checkout”
7
Auto-buy price execution
output
“With the auto-buy feature, Rufus can complete purchases on your behalf within 30 minutes of when the desired price is met and finalize the order using your default payment and shipping details”
8
Account memory update
feedback_loop
“Rufus now has account memory, understanding customers based on their individual shopping activity. Rufus can use information you may have shared previously such as hobbies you enjoy, or a previous mention of a pet, to provide a much more…”
Reported outcome

Adopting Amazon Bedrock increased development velocity by over 6x.
Rufus now serves more than 250 million annual users, with monthly users up 140% YoY and interactions up 210% YoY. Customers using Rufus are 60% more likely to complete a purchase, and auto-buy users save an average of 20% per purchase.

Reported metrics
annual Rufus usersMore than 250 million
monthly users growth YoY140% YoY
interactions growth YoY210% YoY
purchase completion likelihood for Rufus users60% more likely
Show all 7 reported metrics
annual Rufus usersMore than 250 million
monthly users growth YoY140% YoY
interactions growth YoY210% YoY
purchase completion likelihood for Rufus users60% more likely
development velocity increase with Bedrockover 6x
auto-buy customer savings per purchaseaverage of 20%
answer defectssignificantly reduced answer defects
Reported stack
Amazon BedrockAmazon NovaClaude SonnetAmazon Nova Web Groundingprompt cachingLLM-as-a-judgeconverse API
Source
https://aws.amazon.com/blogs/machine-learning/how-rufus-scales-conversational-shopping-experiences-to-millions-of-amazon-customers-with-amazon-bedrock?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Adopting Amazon Bedrock increased development velocity by over 6x.

What tools did this team use?

Amazon Bedrock, Amazon Nova, Claude Sonnet, Amazon Nova Web Grounding, prompt caching, LLM-as-a-judge, converse API.

What results were reported?

annual Rufus users: More than 250 million; monthly users growth YoY: 140% YoY; interactions growth YoY: 210% YoY; purchase completion likelihood for Rufus users: 60% more likely (source-reported, not independently verified).

What failed first in this deployment?

Off-the-shelf models evaluated before the custom LLM performed poorly in shopping domain evaluations, and larger third-party models added unacceptable latency and cost penalties.

How is this ecommerce ops AI workflow structured?

Customer shopping query → Per-query model selection → Context injection → Web grounding for accuracy → Agentic tool execution → Conversational response delivered → Auto-buy price execution → Account memory update.

Related ecommerce ops cases