Ecommerce ops · Production

Zalando builds cross-lingual neural product search with deep learning

The problem

Zalando's symbolic IR system (Solr/Elasticsearch) had a fragile, multi-component NLP pipeline that did not scale to multiple languages without rewriting language-specific components, and could not understand synonyms or semantics without hard-coded rules and lexicons.

First attempt

The existing Solr/Elasticsearch symbolic approach required hard-coded, language-dependent synonym lexicons and had tightly coupled pipeline components where an upstream defect could break the entire system, and component-level improvements did not reliably translate to better end-user search quality.

Workflow diagram · grounded in source
1
User query submitted
trigger
“doing inference means serving search requests from users”
2
Query encoded by character-RNN
ai_action
“The character-RNN (e.g. LSTM, GRU, SRU) model is a good choice. By feeding RNN character by character, the model becomes resilient to misspelling such as adding/deleting/replacing characters. The misspelled queries would result in a simi…”
3
Product images encoded by ResNet CNN
ai_action
“The RGB image data of a product is fed into a multi-layer convolutional neural network based on the ResNet architecture, resulting in an image vector representation in 128-dimensions”
4
Product attributes encoded by DNN
ai_action
“The attributes of a product can be combined into a sparse one-hot encoded vector. It is then supplied to a four-layer, fully connected deep neural network with steadily diminishing layer size”
5
Similarity computed in metric layer
ai_action
“In the metric layer, we compute the similarity of a query vector with an image vector and an attribute vector, respectively. Finally, in the loss layer, we compute the difference of similarities between positive and negative pairs”
6
Results ranked and returned
output
“first represent user input as a vector using query encoder; then iterate over all available products and compute the metric between the query vector and each of them; finally, sort the results”
Reported outcome

The deep neural network-based search system handles misspellings, cross-lingual queries, and semantic synonymy without hard-coded rules, with an inference time of about two seconds per query on a quad-core CPU for 300,000 products.

Reported metrics
Inference time per queryabout two seconds per query on a quad-core CPU for 300,000 products
Reported stack
TensorFlowResNet
Source
https://engineering.zalando.com/posts/2018/02/search-deep-neural-network.html
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The deep neural network-based search system handles misspellings, cross-lingual queries, and semantic synonymy without hard-coded rules, with an inference time of about two seconds per query on a quad-core CPU for 300…

What tools did this team use?

TensorFlow, ResNet.

What results were reported?

Inference time per query: about two seconds per query on a quad-core CPU for 300,000 products (source-reported, not independently verified).

What failed first in this deployment?

The existing Solr/Elasticsearch symbolic approach required hard-coded, language-dependent synonym lexicons and had tightly coupled pipeline components where an upstream defect could break the entire system, and compon…

How is this ecommerce ops AI workflow structured?

User query submitted → Query encoded by character-RNN → Product images encoded by ResNet CNN → Product attributes encoded by DNN → Similarity computed in metric layer → Results ranked and returned.

Related ecommerce ops cases