Back office ops · Production

Woowa Brothers builds an AI API Gateway to centralize Gen AI service development

The problem

Woowa Brothers' Gen AI service development was fragmented: credentials and prompts were scattered across services without a unified hub, every team duplicated PII handling, logging, and rate-limiting logic, and any configuration change required a full code deployment.

Workflow diagram · grounded in source
1
Project registration and API key issuance
trigger
“Users will first request for the generation of a project, and use the gateway with the api_key issued”
2
API key validation per request
validation
“the concept of "project" is offered to validate api_key for each request”
3
PII detection and masking
validation
“requests containing sensitive data should be rejected or have the sensitive information masked before being passed to external Gen APIs”
4
Template-based prompt rendering
integration
“Templates use dynamic inputs and generate the final prompt based on these inputs (query). Values entered in dynamic templates are in {key} format. When requested through templates, values corresponding to {key} are entered to create the …”
5
Routing to Gen AI provider
routing
“If multiple credentials are registered while creating a proxy, the gateway will select and use an appropriate credential, which reduces users' concern”
6
Input and output logging
integration
“Input and output data from Gen AI processes can be used in various ways, including debugging of services, improvement loops for generated quality, and caching. To use these input and output data, each service is required to create and ma…”
7
Simplified response delivery
output
“The request format is simplified by defining these elements together, and a proxy function is provided to manage information needed to call Gen AI API independently from the requesting client server”
Reported outcome

The gateway centralizes credential and prompt management across multiple Gen AI providers, eliminates duplicated PII, logging, and rate-limiting code, and enables data scientists to modify configurations without engineer support or redeployment, with an expected significant improvement in development team productivity.

Reported metrics
Development team productivitysignificantly improve the productivity
System stability, security, and efficiencygreatly improve the overall system stability, security, and efficiency
Long-term cost vs commercial solutionsLong-term cost savings compared to commercial solutions
Reported stack
AI API GatewayML SDKLangChainAWS Secrets ManagerAWS BedrockAzure OpenAIGCP VertexAINCP
Source
https://tech.deliveryhero.com/generative-ai-services-easy-start-with-the-gateway/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The gateway centralizes credential and prompt management across multiple Gen AI providers, eliminates duplicated PII, logging, and rate-limiting code, and enables data scientists to modify configurations without engin…

What tools did this team use?

AI API Gateway, ML SDK, LangChain, AWS Secrets Manager, AWS Bedrock, Azure OpenAI, GCP VertexAI, NCP.

What results were reported?

Development team productivity: significantly improve the productivity; System stability, security, and efficiency: greatly improve the overall system stability, security, and efficiency; Long-term cost vs commercial solutions: Long-term cost savings compared to commercial solutions (source-reported, not independently verified).

How is this back office ops AI workflow structured?

Project registration and API key issuance → API key validation per request → PII detection and masking → Template-based prompt rendering → Routing to Gen AI provider → Input and output logging → Simplified response delivery.

Related back office ops cases