Space-efficient ML feature stores using bloom filters: a Zalando engineering benchmark
Conventional key-value-store-based ML feature stores become very large and impose significant challenges: network calls add 2–10ms of latency per lookup, distributed databases with strict performance requirements are expensive to host, backfill operations become very costly, and multiple feature lookups per request become prohibitively expensive under strict latency budgets.
A bloom-filter-backed compressed feature store achieves the same click-prediction classification performance (AUC~=0.7997) as a conventional key-value store while using only 3% of the memory, with no detectable throughput overhead.
Show all 6 reported metrics
Frequently asked questions
What did this team achieve with this AI workflow?
A bloom-filter-backed compressed feature store achieves the same click-prediction classification performance (AUC~=0.7997) as a conventional key-value store while using only 3% of the memory, with no detectable throug…
What tools did this team use?
Redis, Bloom-Filter.
What results were reported?
Compressed vs uncompressed feature store memory footprint: 3%; click-prediction AUC at 3% memory: AUC~=0.7997; uplift retained at 40MB state size: 90%; State size at 90% uplift as fraction of uncompressed: 0.3% (source-reported, not independently verified).
How is this ecommerce ops AI workflow structured?
User request triggers lookup → In-process feature store query → Click prediction.