Back office ops · Production

How Salesforce achieves high-performance model deployment with Amazon SageMaker AI

The problem

Salesforce's AI Model Serving team struggled to balance latency and throughput with cost-efficiency at scale, keep pace with fast-moving AI innovation requiring constant model evaluation and quick deployment, and maintain secure model hosting across environments.

Workflow diagram · grounded in source
1
Requirements and objectives gathering
trigger
“gathering requirements and performance objectives, hosting, optimizing, and scaling AI models, including LLMs, built by Salesforce's data science and research teams”
2
Multi-environment model evaluation
validation
“evaluation of ML models across multiple environments and extensive performance testing to achieve scalability and reliability for inferencing on AWS”
3
CI/CD pipeline regression testing
validation
“implemented continuous integration (CI) pipelines using a mix of internal and external tools such as Jenkins and Spinnaker to detect any unintended side effects early”
4
YAML configuration management
integration
“Configuration management using simple YAML files enabled rapid experimentation across optimizers and hyperparameters without altering the underlying code”
5
Automated multi-region deployment
output
“deploying them quickly through automated, self-service processes across multiple AWS Regions”
6
Real-time monitoring and autoscaling
feedback_loop
“SageMaker AI provides elastic load balancing, instance scaling, and real-time model monitoring”
Reported outcome

Salesforce achieved substantial improvements in deployment speed and cost-efficiency, with iteration cycles dropping from weeks to days or hours, and model deployment time reduced by as much as 50%.

Reported metrics
Model deployment time reductionas much as 50%
Iteration cycle timedays or even hours instead of weeks
Deployment speed and cost-efficiencysubstantial improvements
Reported stack
Amazon SageMaker AISageMaker AI Deep Learning ContainersJenkinsSpinnakerTensorRTvLLMDJL-ServingAWS TrainiumAWS InferentiaAWS Graviton
Source
https://aws.amazon.com/blogs/machine-learning/how-salesforce-achieves-high-performance-model-deployment-with-amazon-sagemaker-ai?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Salesforce achieved substantial improvements in deployment speed and cost-efficiency, with iteration cycles dropping from weeks to days or hours, and model deployment time reduced by as much as 50%.

What tools did this team use?

Amazon SageMaker AI, SageMaker AI Deep Learning Containers, Jenkins, Spinnaker, TensorRT, vLLM, DJL-Serving, AWS Trainium, AWS Inferentia, AWS Graviton.

What results were reported?

Model deployment time reduction: as much as 50%; Iteration cycle time: days or even hours instead of weeks; Deployment speed and cost-efficiency: substantial improvements (source-reported, not independently verified).

How is this back office ops AI workflow structured?

Requirements and objectives gathering → Multi-environment model evaluation → CI/CD pipeline regression testing → YAML configuration management → Automated multi-region deployment → Real-time monitoring and autoscaling.

Related back office ops cases