Ecommerce ops · Production

Amazon.com Catalog Team builds self-learning generative AI at scale with Amazon Bedrock

The problem

Amazon's catalog system processes millions of daily product submissions requiring structured attribute extraction and title generation, but the traditional approach of having applied scientists analyze failures, update prompts, and redeploy could not scale to the volume and variety. Large models delivered accuracy but could not scale cost-efficiently; smaller models struggled with complex ambiguous cases.

First attempt

The existing manual improvement loop—scientists analyzing failures, updating prompts, testing, and redeploying—was resource-intensive and could not keep pace with the volume and variety of real-world catalog submissions.

Workflow diagram · grounded in source
1
Product listing submission
trigger
“When a seller lists a new product, the catalog system must extract structured attributes—dimensions, materials, compatibility, and technical specifications—while generating content such as titles that match how customers search”
2
Generator-evaluator workers process
ai_action
“multiple lightweight worker models operating in parallel—some as generators extracting attributes, others as evaluators assessing those extractions. We explicitly prompt evaluators to be critical, instructing them to scrutinize extractio…”
3
Consensus or disagreement routing
routing
“When the generator and evaluator agree, we have high confidence in the result and process it at minimal computational cost. When they disagree, we've identified a case worth investigating—triggering the supervisor to resolve the dispute”
4
Supervisor agent resolves and extracts learnings
ai_action
“When workers disagree, we invoke a supervisor agent—a more capable model that resolves the dispute and investigates why it occurred. The supervisor determines what context or reasoning the workers lacked, and these insights become reusab…”
5
Post-inference feedback capture
feedback_loop
“Post-inference, sellers express disagreement through listing updates and appeals—signals that our original extraction might have missed important context. Customers disagree through returns and negative reviews”
6
Hierarchical knowledge base update
integration
“We use a hierarchical structure where a large language model (LLM)-based memory manager navigates the knowledge tree to place each learning. The learning aggregator and memory manager utilize Amazon DynamoDB for the knowledge base”
7
Learnings injected into worker prompts
feedback_loop
“During inference, workers receive relevant learnings in their prompts based on product category, automatically incorporating domain knowledge from past disagreements”
8
Human review queue
human_review
“Human review (Amazon Simple Queue Service (Amazon SQS))”
Reported outcome

The self-learning system continuously improved accuracy while reducing costs, with error rates falling continuously through accumulated learnings rather than retraining, and the system evolving from generic understanding to domain-specific expertise.

Reported metrics
Error rate trajectoryError rates fell continuously
System cost and qualitycosts decrease and quality increases
Reported stack
Amazon BedrockAmazon Nova LiteAnthropic Claude SonnetAmazon EC2Bedrock AgentCoreAmazon DynamoDBAmazon SQSAmazon CloudWatchAmazon Bedrock Runtime
Source
https://aws.amazon.com/blogs/machine-learning/how-the-amazon-com-catalog-team-built-self-learning-generative-ai-at-scale-with-amazon-bedrock?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The self-learning system continuously improved accuracy while reducing costs, with error rates falling continuously through accumulated learnings rather than retraining, and the system evolving from generic understand…

What tools did this team use?

Amazon Bedrock, Amazon Nova Lite, Anthropic Claude Sonnet, Amazon EC2, Bedrock AgentCore, Amazon DynamoDB, Amazon SQS, Amazon CloudWatch, Amazon Bedrock Runtime.

What results were reported?

Error rate trajectory: Error rates fell continuously; System cost and quality: costs decrease and quality increases (source-reported, not independently verified).

What failed first in this deployment?

The existing manual improvement loop—scientists analyzing failures, updating prompts, testing, and redeploying—was resource-intensive and could not keep pace with the volume and variety of real-world catalog submissions.

How is this ecommerce ops AI workflow structured?

Product listing submission → Generator-evaluator workers process → Consensus or disagreement routing → Supervisor agent resolves and extracts learnings → Post-inference feedback capture → Hierarchical knowledge base update → Learnings injected into worker prompts → Human review queue.

Related ecommerce ops cases