Back office ops · Production

How Amazon Bedrock Custom Model Import streamlined LLM deployment for Salesforce

The problem

Salesforce's AI platform team faced significant operational overhead deploying fine-tuned LLMs—spending months on instance selection, serving engine configuration, and GPU capacity reservations that were expensive, time-consuming, and difficult to maintain across frequent model releases.

Workflow diagram · grounded in source
1
CI/CD saves model artifacts to S3
trigger
“their continuous integration and continuous delivery (CI/CD) process saves model artifacts to their model store (an Amazon Simple Storage Service (Amazon S3) bucket)”
2
Register model via Bedrock import API
integration
“they now call the Amazon Bedrock Custom Model Import API to register the model with Amazon Bedrock. This control plane operation is lightweight because Amazon Bedrock pulls the model directly from Amazon S3, adding minimal overhead (5–7 …”
3
Client request preprocessing
integration
“Client requests flow through their established preprocessing layer for business logic like prompt formatting before reaching Amazon Bedrock”
4
SageMaker proxy routes to Bedrock
routing
“they deployed lightweight SageMaker CPU containers that act as intelligent proxies—running their custom model.py logic while forwarding the actual inference to Amazon Bedrock endpoints”
5
Bedrock serverless inference
ai_action
“Amazon Bedrock—where the service automatically spins up model copies on-demand and scales out under heavy load”
6
Postprocessing and response delivery
output
“with postprocessing applied to the raw model output”
Reported outcome

Salesforce achieved a 30% reduction in model deployment time and up to 40% cost savings by adopting Amazon Bedrock Custom Model Import, while maintaining backward-compatibility with existing applications through a hybrid architecture using SageMaker proxy containers alongside Amazon Bedrock serverless inference.

Reported metrics
Model deployment time reduction30%
Cost reductionup to 40%
latency improvement vs SageMaker baseline at low concurrency44%
deployment overhead added by Bedrock import step5–7 mins
Show all 6 reported metrics
model deployment time reduction30%
cost reductionup to 40%
latency improvement vs SageMaker baseline at low concurrency44%
deployment overhead added by Bedrock import step5–7 mins
overall model release process durationapproximately 1 hour
additional network hop latency from SageMaker proxy5–10 millisecond latency
Reported stack
Amazon Bedrock Custom Model ImportAmazon SageMaker InferenceAmazon S3AWS Identity and Access ManagementAmazon Bedrock Knowledge BasesAmazon Bedrock GuardrailsAmazon Bedrock AgentsLlamaQwenMistralvLLMTensorRT-LLMAmazon API GatewayAWS LambdaAgentforce
Source
https://aws.amazon.com/blogs/machine-learning/how-amazon-bedrock-custom-model-import-streamlined-llm-deployment-for-salesforce?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Salesforce achieved a 30% reduction in model deployment time and up to 40% cost savings by adopting Amazon Bedrock Custom Model Import, while maintaining backward-compatibility with existing applications through a hyb…

What tools did this team use?

Amazon Bedrock Custom Model Import, Amazon SageMaker Inference, Amazon S3, AWS Identity and Access Management, Amazon Bedrock Knowledge Bases, Amazon Bedrock Guardrails, Amazon Bedrock Agents, Llama, Qwen, Mistral.

What results were reported?

Model deployment time reduction: 30%; Cost reduction: up to 40%; latency improvement vs SageMaker baseline at low concurrency: 44%; deployment overhead added by Bedrock import step: 5–7 mins (source-reported, not independently verified).

How is this back office ops AI workflow structured?

CI/CD saves model artifacts to S3 → Register model via Bedrock import API → Client request preprocessing → SageMaker proxy routes to Bedrock → Bedrock serverless inference → Postprocessing and response delivery.

Related back office ops cases