Back office ops · Production

Deploying DeepSeek-R1 on AWS: Our Journey Through Performance, Cost, and Reality

The problem

The team wanted to evaluate whether self-hosted open-source LLMs could replace paid AI coding assistants, motivated by goals of control, privacy, and long-term cost savings, but faced uncertainty about operational viability at startup scale.

First attempt

The 16B model crashed under load even after aggressive quantization and batch-size reductions; performance degraded below usable token speeds with longer context windows; and tuning parameters introduced unpredictable latency and crashes.

Workflow diagram · grounded in source
1
Decide to evaluate self-hosted LLMs
trigger
“we explored deploying open-source DeepSeek-R1 models in-house to evaluate their viability as alternatives to paid services like ChatGPT”
2
Provision AWS EC2 GPU instances
integration
“We tested: g5g.xlarge (4 vCPU + 8 GB + 16 GB GPU) g5g.2xlarge (8 vCPU + 16 GB + 16 GB GPU)”
3
Deploy via Docker and Ollama stack
integration
“Docker & NVIDIA Container Toolkit: The backbone of our deployment. It ensured consistent environments with GPU access, saving us from the usual "it works on my machine" chaos.”
4
Benchmark four model sizes
ai_action
“We benchmarked four DeepSeek-R1 model sizes: DeepSeek-R1 1.5B, 7B,8B, 16B”
5
Validate cost versus SaaS alternatives
validation
“Keeping just one g5g.2xlarge instance running 24/7 clocks in at around $414/month. Equivalent SaaS services (e.g., ChatGPT Plus at $20/user/month) appeared drastically cheaper at our scale.”
6
Output evaluation conclusions
output
“For startups operating at a small-to-medium scale, today's SaaS LLM offerings deliver far better value for money. Until infra costs drop or usage scales dramatically, self-hosting simply doesn't add up.”
Reported outcome

Self-hosting was found not cost-effective for startups at small-to-medium scale: a single AWS instance cost around $414/month while comparable SaaS offerings were drastically cheaper, with hidden operational overhead further widening the gap.

Reported metrics
AWS g5g.2xlarge instance cost (24/7)around $414/month
ChatGPT Plus SaaS cost per user$20/user/month
Minimum acceptable token generation speed30–35 tokens/sec
Internal users supportedfewer than 100 internal users
Show all 5 reported metrics
AWS g5g.2xlarge instance cost (24/7)around $414/month
ChatGPT Plus SaaS cost per user$20/user/month
Minimum acceptable token generation speed30–35 tokens/sec
Internal users supportedfewer than 100 internal users
Operational costs versus expectationssignificantly exceeded expectations
Reported stack
DeepSeek-R1AWS EC2DockerNVIDIA Container ToolkitOllamaOpenWeb UI
Source
https://liftoffllc.medium.com/deploying-deepseek-r1-on-aws-our-journey-through-performance-cost-and-reality-48168c6125d5
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Self-hosting was found not cost-effective for startups at small-to-medium scale: a single AWS instance cost around $414/month while comparable SaaS offerings were drastically cheaper, with hidden operational overhead…

What tools did this team use?

DeepSeek-R1, AWS EC2, Docker, NVIDIA Container Toolkit, Ollama, OpenWeb UI.

What results were reported?

AWS g5g.2xlarge instance cost (24/7): around $414/month; ChatGPT Plus SaaS cost per user: $20/user/month; Minimum acceptable token generation speed: 30–35 tokens/sec; Internal users supported: fewer than 100 internal users (source-reported, not independently verified).

What failed first in this deployment?

The 16B model crashed under load even after aggressive quantization and batch-size reductions; performance degraded below usable token speeds with longer context windows; and tuning parameters introduced unpredictable…

How is this back office ops AI workflow structured?

Decide to evaluate self-hosted LLMs → Provision AWS EC2 GPU instances → Deploy via Docker and Ollama stack → Benchmark four model sizes → Validate cost versus SaaS alternatives → Output evaluation conclusions.

Related back office ops cases