Customer support · Production

Amazon Pharmacy builds HIPAA-compliant LLM chatbot for customer care agents using Amazon SageMaker

The problem

Customer care agents at Amazon Pharmacy struggled to quickly find precise pharmacy information due to the diversity, volume, and complexity of healthcare processes, slowing down patient service.

Workflow diagram · grounded in source
1
Patient contacts via chat
trigger
“a patient contacting Amazon Pharmacy customer care via chat (Step 1)”
2
Agent queries internal chatbot
integration
“Agents use a separate internal customer care UI to ask questions to the LLM-based Q&A chatbot (Step 2)”
3
Query embedding and retrieval
ai_action
“the query is converted to an embedding and then used as a search key for a matching index (from the previous step). The matching criteria is based on a similarity model, such as FAISS or Amazon Open Search Service. When there are matches…”
4
LLM generates response
ai_action
“the prompt is sent to the LLM (generator foundation modal), which composes the final machine-generated response to the original question”
5
Agent reviews and responds
human_review
“the machine-generated response is returned to the agent, who can review the answer before providing it back to the end-customer (Step 4)”
6
Agent submits feedback
feedback_loop
“Agents also label the machine-generated response with their feedback (for example, positive or negative). This feedback is then used by the Amazon Pharmacy development team to improve the solution (through fine-tuning or data improvement…”
Reported outcome

Amazon Pharmacy deployed a HIPAA-compliant RAG-based chatbot enabling agents to assist patients more quickly with precise answers, while SageMaker JumpStart cut months of model development work.

Reported metrics
Patient service speedassist patients more quickly
Model development timecut months of work
Answer qualityprecise, informative, and concise answers
Reported stack
Amazon SageMakerAmazon SageMaker JumpStartAWS FargateAmazon S3FAISSAmazon OpenSearch ServiceAmazon ECSAWS CloudFormation
Source
https://aws.amazon.com/blogs/machine-learning/learn-how-amazon-pharmacy-created-their-llm-based-chat-bot-using-amazon-sagemaker?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Amazon Pharmacy deployed a HIPAA-compliant RAG-based chatbot enabling agents to assist patients more quickly with precise answers, while SageMaker JumpStart cut months of model development work.

What tools did this team use?

Amazon SageMaker, Amazon SageMaker JumpStart, AWS Fargate, Amazon S3, FAISS, Amazon OpenSearch Service, Amazon ECS, AWS CloudFormation.

What results were reported?

Patient service speed: assist patients more quickly; Model development time: cut months of work; Answer quality: precise, informative, and concise answers (source-reported, not independently verified).

How is this customer support AI workflow structured?

Patient contacts via chat → Agent queries internal chatbot → Query embedding and retrieval → LLM generates response → Agent reviews and responds → Agent submits feedback.

Related customer support cases