Semantic multi-component embeddings with Redis reduce LLM token consumption by 91% in enterprise tool selection
An enterprise AI platform with 70+ automated tools was sending all tool definitions to the LLM on every query, consuming over 8,000 tokens per request, causing spiralling costs, slower responses, and irrelevant tool suggestions.
Keyword matching and category filtering failed to bridge the vocabulary gap between natural language queries and technical tool names, and could not disambiguate context-dependent queries.
Semantic tool selection with multi-component embeddings and Redis vector search reduced token consumption by 91.5% and cost per query by 49%, while improving Precision@3 to 95% and delivering 31% faster response times.
Show all 16 reported metrics
Frequently asked questions
What did this team achieve with this AI workflow?
Semantic tool selection with multi-component embeddings and Redis vector search reduced token consumption by 91.5% and cost per query by 49%, while improving Precision@3 to 95% and delivering 31% faster response times.
What tools did this team use?
Redis Stack, text-embedding-3-small.
What results were reported?
Token usage reduction: 91.5%; Cost reduction per query: 49%; Average input tokens per query (before): 7,244; Average input tokens per query (after): 198 (source-reported, not independently verified).
What failed first in this deployment?
Keyword matching and category filtering failed to bridge the vocabulary gap between natural language queries and technical tool names, and could not disambiguate context-dependent queries.
How is this back office ops AI workflow structured?
User query arrives → Query embedding generated → Multi-component vector search → Weighted relevance scoring → Adaptive tool selection → Minimal tool set sent to LLM.