Back office ops · Production

Trendyol autooptimizer: autonomous AI agent achieves 4x inference serving performance on Gemma 4 26B in 18 experiments

The problem

Manual LLM inference serving configuration tuning requires hours of iterative flag-flipping, restarts, and benchmarking across a huge interacting parameter space, with results tracked only in half-remembered spreadsheets and no systematic search strategy.

Workflow diagram · grounded in source
1
User specifies model and framework
trigger
“You point it at a model and a serving framework. An agent takes it from there.”
2
Agent builds hypothesis backlog
ai_action
“It maintains a prioritized hypothesis backlog ordered by expected impact. It starts with the big levers — quantization, batching, memory utilization — and works its way down to the smaller knobs. Each hypothesis comes with a rationale, a…”
3
Agent applies hypothesis to config
ai_action
“The agent picks the most promising hypothesis from its backlog and applies it to the serving config.”
4
Server launched and benchmarked
ai_action
“It launches a fresh server with the new flags, runs a standardized benchmark, and reads the score.”
5
Keep or revert based on score
routing
“If things got better, the change sticks. If they got worse, it's rolled back.”
6
Loop to next hypothesis
feedback_loop
“Then it picks the next hypothesis and does it again. And again. And again.”
7
Optimized config delivered
output
“Come back an hour later — or the next morning — to a log of every experiment it tried and a serving config that's been through dozens of trials.”
Reported outcome

The autonomous agent ran 18 experiments on Gemma 4 26B and delivered a serving configuration scoring 4x higher than the defaults (baseline 167, final ~640) with zero human intervention, completing work that would otherwise take a human a full day.

Reported metrics
Serving performance score improvement4x higher than the defaults
Baseline serving score167
Score after first agent step524
First-step improvementA 3x improvement in one step
Show all 7 reported metrics
serving performance score improvement4x higher than the defaults
baseline serving score167
score after first agent step524
first-step improvementA 3x improvement in one step
final serving score~640
experiments run autonomously18
human interventionzero human intervention
Reported stack
autooptimizervLLMSGLang
Source
https://medium.com/trendyol-tech/4x-faster-inference-let-the-agent-do-the-tuning-b27c8afa9e86
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The autonomous agent ran 18 experiments on Gemma 4 26B and delivered a serving configuration scoring 4x higher than the defaults (baseline 167, final ~640) with zero human intervention, completing work that would othe…

What tools did this team use?

autooptimizer, vLLM, SGLang.

What results were reported?

Serving performance score improvement: 4x higher than the defaults; Baseline serving score: 167; Score after first agent step: 524; First-step improvement: A 3x improvement in one step (source-reported, not independently verified).

How is this back office ops AI workflow structured?

User specifies model and framework → Agent builds hypothesis backlog → Agent applies hypothesis to config → Server launched and benchmarked → Keep or revert based on score → Loop to next hypothesis → Optimized config delivered.

Related back office ops cases