Customer support · Production

Sword Health: lessons learned shipping LLM-powered physical therapy AI agent Phoenix

The problem

Healthcare has long faced a dichotomy between quality and affordability. Shipping LLM-powered products in a highly regulated industry presents unique challenges in ensuring safety, consistency, and reliability, with inconsistency issues emerging once features reach production.

Workflow diagram · grounded in source
1
Patient session begins
trigger
“Phoenix creates a seamless support system for the patient during his rehabilitation program”
2
Phoenix delivers real-time feedback
ai_action
“Phoenix delivers a true one-on-one experience by providing real-time feedback to the member and being available to answer any question that the patient might have”
3
Guardrails filter input and output
validation
“we have built medical advice guardrails that constrain the types of tips that Phoenix is able to provide. These guidelines were devised in collaboration with our clinical team”
4
RAG retrieves knowledge articles
ai_action
“when the patient asks a question, we retrieve the top and most similar articles from the knowledge base or from the vector database, and then we include those articles in the prompt and generate an answer to the user”
5
PT reviews clinical recommendations
human_review
“For everything that's clinical related, we always have the physical therapist reviewing or accepting the recommendation or changing it if they do not agree with it”
6
Post-conversation sentiment analysis
feedback_loop
“sentiment analysis that we run after each conversation between the Phoenix and the patient”
7
User feedback feeds model improvement
feedback_loop
“By collecting this feedback, we can build high-quality datasets that can be used for things such as guardrails, evaluations, few-shot learning, and fine-tuning”
Reported outcome

Sword Health shipped and iterated on many LLM-powered features across its product portfolio over several years, establishing a systematic development practice with guardrails, evals, RAG, and feedback loops.
Switching from GPT-4.0 to Claude 3.5 Sonnet with minor prompt adjustments produced an increase in performance of around 10 percentage points.

Reported metrics
Latency increase from online guardrailsaround 30%
Performance improvement from model switcharound 10% points
Patient non-engagement ratearound 50%
Reported stack
PhoenixGondolaStreamlitGPT-4.0Claude 3.5 SonnetMySQLvector databaseLangfuseLangSmithRAGAS
Source
https://www.infoq.com/presentations/ai-healthcare-learnings/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Sword Health shipped and iterated on many LLM-powered features across its product portfolio over several years, establishing a systematic development practice with guardrails, evals, RAG, and feedback loops.

What tools did this team use?

Phoenix, Gondola, Streamlit, GPT-4.0, Claude 3.5 Sonnet, MySQL, vector database, Langfuse, LangSmith, RAGAS.

What results were reported?

Latency increase from online guardrails: around 30%; Performance improvement from model switch: around 10% points; Patient non-engagement rate: around 50% (source-reported, not independently verified).

How is this customer support AI workflow structured?

Patient session begins → Phoenix delivers real-time feedback → Guardrails filter input and output → RAG retrieves knowledge articles → PT reviews clinical recommendations → Post-conversation sentiment analysis → User feedback feeds model improvement.

Related customer support cases