Workflow · Production

OpenRouter: Founding story and architecture of a multi-model AI inference marketplace

The problem

The AI inference ecosystem grew into a fragmented, heterogeneous landscape of providers with incompatible features—different samplers, caching support, tool calling, and pricing—making it difficult for developers to choose, compare, or switch between models.

Workflow diagram · grounded in source
1
Developer API request
trigger
“It is an API that lets you access all language models”
2
Provider aggregation and routing
routing
“we aggregated all providers in one spot, and at different price points, it became a marketplace”
3
Plugin capability augmentation
integration
“searching the web for all models, PDF parsing for all models”
4
Streaming output with web annotations
output
“every language model can just kind of tap into this plugin and get web annotations as results are being fed back to users in real time”
Reported outcome

OpenRouter became a marketplace aggregating over 400 models and over 60 active providers, growing 10 to a hundred percent month over month for the last two years, and achieving about 30 milliseconds latency, while its data supports the conclusion that AI inference is not winner-take-all.

Reported metrics
Month-over-month growth rate10 to a hundred percent month over month for the last two years
Models available on platformover 400 models
Active providers on platformover 60 active providers
API latencyabout 30 milliseconds
Show all 5 reported metrics
month-over-month growth rate10 to a hundred percent month over month for the last two years
models available on platformover 400 models
active providers on platformover 60 active providers
API latencyabout 30 milliseconds
Google Gemini token share on platformgrown to 34, 30 5%
Reported stack
Window AIBedrockVertex
Source
https://libraries.thoth.art/aiewf2025/talk/fun-stories-from-building-openrouter-and-where-all-this-is-going
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

OpenRouter became a marketplace aggregating over 400 models and over 60 active providers, growing 10 to a hundred percent month over month for the last two years, and achieving about 30 milliseconds latency, while its…

What tools did this team use?

Window AI, Bedrock, Vertex.

What results were reported?

Month-over-month growth rate: 10 to a hundred percent month over month for the last two years; Models available on platform: over 400 models; Active providers on platform: over 60 active providers; API latency: about 30 milliseconds (source-reported, not independently verified).

How is this workflow AI workflow structured?

Developer API request → Provider aggregation and routing → Plugin capability augmentation → Streaming output with web annotations.