It support · Production

Solo.io uses agentgateway to govern, secure, and observe AI agent traffic to MCP servers and LLMs

The problem

As the number of internal MCP servers at Solo.io grew, the team needed a centralized way to aggregate, secure, and observe AI agent traffic without modifying individual servers or agents. LLM access through Vertex AI also provided no visibility into usage patterns or per-user, per-model costs.

First attempt

Before agentgateway, LLM access routed through Vertex AI provided no visibility into usage patterns or per-user, per-model costs.

Workflow diagram · grounded in source
1
Employee triggers support agent
trigger
“our support agent allows employees to ask questions in our corporate Slack by mentioning @Support”
2
Agent accesses MCP tools
ai_action
“The agent can access MCP tools to search our internal Slack conversations, query our codebase, interact with internal tools, and retrieve information from documentation and GitHub issues. It returns answers along with confidence estimates.”
3
MCP server multiplexing
integration
“Agentgateway enables this through configuration alone, without requiring code changes or redeployments”
4
Central auth and policy enforcement
validation
“With agentgateway, these concerns are handled centrally through configuration while MCP servers remain unchanged”
5
Observability metrics enrichment
output
“we enrich metrics to identify which user and product are associated with each tool call”
6
LLM traffic routing through gateway
routing
“By routing LLM traffic through agentgateway, we gained the ability to track usage and spending at both individual and organization levels”
7
Per-user cost and usage tracking
output
“we can correlate telemetry with each user's email to understand LLM model usage and cost per user”
Reported outcome

By routing traffic through agentgateway, Solo.io can now track LLM usage and spending at individual and organization levels, analyze tool usage distribution, and govern both MCP and LLM traffic without modifying agents or backend services.

Reported metrics
Operational efficiencyimproved operational efficiency with agentic AI
LLM usage and cost visibilitytrack usage and spending at both individual and organization levels
Tool usage analysisanalyze tool usage distribution and better understand how our internal agents use tools over time
Reported stack
agentgatewayClaude CodeCursorVertex AIGrafanaClaudeSlackGitHub
Source
https://aaif.io/blog/use-agentgateway-to-mediate-mcp-and-llm-traffic-at-solo-io/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

By routing traffic through agentgateway, Solo.io can now track LLM usage and spending at individual and organization levels, analyze tool usage distribution, and govern both MCP and LLM traffic without modifying agent…

What tools did this team use?

agentgateway, Claude Code, Cursor, Vertex AI, Grafana, Claude, Slack, GitHub.

What results were reported?

Operational efficiency: improved operational efficiency with agentic AI; LLM usage and cost visibility: track usage and spending at both individual and organization levels; Tool usage analysis: analyze tool usage distribution and better understand how our internal agents use tools over time (source-reported, not independently verified).

What failed first in this deployment?

Before agentgateway, LLM access routed through Vertex AI provided no visibility into usage patterns or per-user, per-model costs.

How is this it support AI workflow structured?

Employee triggers support agent → Agent accesses MCP tools → MCP server multiplexing → Central auth and policy enforcement → Observability metrics enrichment → LLM traffic routing through gateway → Per-user cost and usage tracking.

Related it support cases