Workflow · Production

Incrementally improving LLM nutritional estimation in Taralli using DSPy prompt optimization

The problem

LLM-based nutritional estimation is difficult to improve without rigorous measurement; relying on subjective 'vibes' makes it impossible to know whether a change helps or hurts the system.

Workflow diagram · grounded in source
1
User submits food description
trigger
“I created an endpoint that takes a food_description and returns these messages for on-device inference. The iOS app calls this endpoint”
2
LLM estimates nutritional content
ai_action
“The best performing model is Gemini 3 Flash with a set of 16 examples in the prompt. It achieves a score of around 60%.”
3
Accuracy evaluated at threshold
validation
“we want the predicted calories to be within 10% of the ground truth calories. Which means our metric will then be Accuracy@10%.”
4
Prompt optimization with DSPy
feedback_loop
“GEPA: The new kid on the block for prompt optimizers. The nice thing about GEPA is that you can also give it some textual feedback on the incorrect predictions, and it will use it.”
5
Production model update
output
“I decided to update Taralli to use Gemini 3 Flash with the few-shot approach. This approach is ~15% more accurate when compared to the old version”
Reported outcome

Switching Taralli to Gemini 3 Flash with a few-shot approach yielded ~15% accuracy improvement over the prior production model, and the system now supports fully offline on-device inference.

Reported metrics
NutriBench best model carb accuracy (Acc@7.5)66.82%
NutriBench best model prompt response rate~99%
Taralli best model Accuracy@10%around 60%
Accuracy improvement over previous production version~15%
Reported stack
DSPyMIPROv2GEPAOpenRouterNutriBenchHugging FaceGemini 2.5 FlashGemini 3 FlashDeepSeech v3.2
Source
https://duarteocarmo.com/blog/from-nutribench-to-taralli-how-far-can-you-take-a-prompt
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Switching Taralli to Gemini 3 Flash with a few-shot approach yielded ~15% accuracy improvement over the prior production model, and the system now supports fully offline on-device inference.

What tools did this team use?

DSPy, MIPROv2, GEPA, OpenRouter, NutriBench, Hugging Face, Gemini 2.5 Flash, Gemini 3 Flash, DeepSeech v3.2.

What results were reported?

NutriBench best model carb accuracy (Acc@7.5): 66.82%; NutriBench best model prompt response rate: ~99%; Taralli best model Accuracy@10%: around 60%; Accuracy improvement over previous production version: ~15% (source-reported, not independently verified).

How is this workflow AI workflow structured?

User submits food description → LLM estimates nutritional content → Accuracy evaluated at threshold → Prompt optimization with DSPy → Production model update.