It support · Production

Mercari builds LLM Key Server for secure, temporary LLM API access via OIDC and LiteLLM

The problem

LLM API keys at Mercari had no expiration date, proliferated without clear management across multiple providers, and required manual administrator registration for access, making regular access audits difficult and creating prolonged information-leakage risk if keys were compromised.

First attempt

The recommended approach of accessing LLM APIs through Google Cloud or Azure using Workload Identity and cross-cloud federation was too complex to configure broadly, and many external AI and LLM products do not support these methods, necessitating an alternative. Overly strict security policies also risked being bypassed by users.

Workflow diagram · grounded in source
1
Obtain OIDC identity token
trigger
“users or workloads who need LLM access for Claude Code or other applications obtain an OIDC ID token from Google APIs to prove their identity”
2
Key Server verifies token
validation
“when the OIDC ID token is sent to the LLM Key Server, the server verifies the token signature”
3
Temporary API key issued
output
“issues a temporary API key for accessing LiteLLM based on the information in the token. This API key has a short expiration period”
4
LLM access via LiteLLM
integration
“allowing users to access various LLM models through LiteLLM”
5
Automatic key renewal
feedback_loop
“we provide an automatic key renewal mechanism. This is implemented as a Go library that automatically renews keys, enabling continuous LLM API usage”
Reported outcome

Mercari deployed the LLM Key Server, enabling users to obtain temporary API keys through their internal accounts without manual requests, and accelerating LLM adoption in CI/CD pipelines including automated code reviews, as well as in internal tools for document summarization and translation.

Reported metrics
LLM adoption in CI/CD pipelinesaccelerated LLM adoption in CI/CD pipelines
LLM adoption for internal document workflowsaccelerated LLM adoption within the company
Information leakage risktime-limited keys reduce information leakage risks
Reported stack
LiteLLMGitHub ActionsGoogle Apps ScriptGoogle WorkspaceGoogle CloudClaude CodeOpenID Connect (OIDC)
Source
https://engineering.mercari.com/en/blog/entry/20251202-llm-key-server/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Mercari deployed the LLM Key Server, enabling users to obtain temporary API keys through their internal accounts without manual requests, and accelerating LLM adoption in CI/CD pipelines including automated code revie…

What tools did this team use?

LiteLLM, GitHub Actions, Google Apps Script, Google Workspace, Google Cloud, Claude Code, OpenID Connect (OIDC).

What results were reported?

LLM adoption in CI/CD pipelines: accelerated LLM adoption in CI/CD pipelines; LLM adoption for internal document workflows: accelerated LLM adoption within the company; Information leakage risk: time-limited keys reduce information leakage risks (source-reported, not independently verified).

What failed first in this deployment?

The recommended approach of accessing LLM APIs through Google Cloud or Azure using Workload Identity and cross-cloud federation was too complex to configure broadly, and many external AI and LLM products do not suppor…

How is this it support AI workflow structured?

Obtain OIDC identity token → Key Server verifies token → Temporary API key issued → LLM access via LiteLLM → Automatic key renewal.

Related it support cases