Back office ops · Production

How Qualtrics built Socrates: An AI platform powered by Amazon SageMaker and Amazon Bedrock

The problem

Qualtrics needed an enterprise-level ML platform to enable its researchers, scientists, engineers, and knowledge workers to efficiently build, test, and deliver AI-powered capabilities across the XM product suite.

Workflow diagram · grounded in source
1
Data sourcing and exploration
integration
“Knowledge workers can source, explore, and analyze Qualtrics data using Socrates's ML workbenches and AI Data Infrastructure”
2
Model training and HPO
ai_action
“Scientists and researchers are enabled to conduct research, prototype, develop, and train models using a host of SageMaker features”
3
Model deployment to production
integration
“ML engineers can test, productionize, and monitor a heterogeneous set of ML models possessing a wide range of capabilities, inference modes, and production traffic patterns”
4
Unified GenAI Gateway
integration
“The Unified GenAI Gateway is an API that provides a common interface for consumers to interact with all of the platform-supported LLMs and embedding models, regardless of their underlying providers or hosting environments”
5
Agentic workflow orchestration
ai_action
“Socrates Agent Platform, built on top of LangGraph Platform providing a flexible orchestration framework to develop agents as graphs that expedite delivery of agentic features”
6
Product integration via inference API
output
“Partner application teams are provided with an abstracted model inference interface that makes the integration of an ML model into the Qualtrics product a seamless engineering experience”
Reported outcome

The Socrates platform enables the full ML lifecycle at Qualtrics using Amazon SageMaker and Amazon Bedrock, has reduced AI inference costs multiple folds for some use cases, and has boosted performance and accessibility of AI-driven features within the XM suite.

Reported metrics
AI inference cost reduction (Qualtrics, some use cases)reduced AI inference costs multiple folds for some of our use cases
FM deployment cost reduction (SageMaker inference components, average)50%
FM deployment latency reduction (SageMaker inference components, average)20%
Auto scaling time reductionup to 40%
Show all 8 reported metrics
AI inference cost reduction (Qualtrics, some use cases)reduced AI inference costs multiple folds for some of our use cases
FM deployment cost reduction (SageMaker inference components, average)50%
FM deployment latency reduction (SageMaker inference components, average)20%
Auto scaling time reductionup to 40%
Auto scaling detection speed improvement (Meta Llama 3 8B)six times faster
Generative AI inference throughput increase (optimization toolkit)two times higher throughput
Generative AI inference cost reduction (optimization toolkit)up to 50%
Multi-model endpoint inference cost reductionup to 90%
Reported stack
Amazon SageMakerAmazon BedrockSageMaker InferenceJupyterLabLangGraph PlatformOpenAI GPTMeta Llama 3
Source
https://aws.amazon.com/blogs/machine-learning/how-qualtrics-built-socrates-an-ai-platform-powered-by-amazon-sagemaker-and-amazon-bedrock?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The Socrates platform enables the full ML lifecycle at Qualtrics using Amazon SageMaker and Amazon Bedrock, has reduced AI inference costs multiple folds for some use cases, and has boosted performance and accessibili…

What tools did this team use?

Amazon SageMaker, Amazon Bedrock, SageMaker Inference, JupyterLab, LangGraph Platform, OpenAI GPT, Meta Llama 3.

What results were reported?

AI inference cost reduction (Qualtrics, some use cases): reduced AI inference costs multiple folds for some of our use cases; FM deployment cost reduction (SageMaker inference components, average): 50%; FM deployment latency reduction (SageMaker inference components, average): 20%; Auto scaling time reduction: up to 40% (source-reported, not independently verified).

How is this back office ops AI workflow structured?

Data sourcing and exploration → Model training and HPO → Model deployment to production → Unified GenAI Gateway → Agentic workflow orchestration → Product integration via inference API.

Related back office ops cases