Mercari builds LLM Key Server for secure, temporary LLM API access via OIDC and LiteLLM
LLM API keys at Mercari had no expiration date, proliferated without clear management across multiple providers, and required manual administrator registration for access, making regular access audits difficult and creating prolonged information-leakage risk if keys were compromised.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Obtain OIDC identity token
Users or workloads that need LLM access obtain an OIDC ID token from Google APIs to prove their identity.
Mercari deployed the LLM Key Server, enabling users to obtain temporary API keys through their internal accounts without manual requests, and accelerating LLM adoption in CI/CD pipelines including automated code reviews, as well as in internal tools for document summarization and translation.
What failed first
The recommended approach of accessing LLM APIs through Google Cloud or Azure using Workload Identity and cross-cloud federation was too complex to configure broadly, and many external AI and LLM products do not support these methods, necessitating an alternative. Overly strict security policies also risked being bypassed by users.