Ecommerce ops · Production

eBay develops e-Llama: continued pretraining of Llama 3.1 for e-commerce domain adaptation

The problem

General-purpose LLMs like GPT-4 and Claude are too costly and introduce data security risks for eBay's scale; they also lack e-commerce domain knowledge, while training a new LLM from scratch is prohibitively time- and resource-intensive.

First attempt

Third-party LLMs such as GPT-4 and Claude were found impractical for eBay's needs due to cost, data security risks, and limited fine-tuning on proprietary data.

Workflow diagram · grounded in source
1
Identify LLM adaptation need
trigger
“these services come with considerable costs, making them impractical for businesses like eBay that need fine-tuned, scalable and cost-effective solutions”
2
Collect and filter e-commerce data
integration
“we gather data from public listings and product reviews from the eBay website. This data is then thoroughly filtered and serialized to fit the task of autoregressive language modeling”
3
Train e-commerce domain classifier
ai_action
“we train an e-commerce classifier and use it to extract e-commerce specific examples from a larger open-source dataset”
4
Continued pretraining on domain data
ai_action
“we continue training the Llama base models on a large amount of e-commerce data in order to infuse domain specific knowledge into the model. This technique is known as "continued pretraining"”
5
Optimize training setup via experiments
validation
“We determine the optimal training setup through a series of experiments at a smaller scale. We find that, for our use case, a maximum learning rate of 10% of the original maximum learning rate, and a general-to-e-commerce data sampling r…”
6
Instruction-tune with human feedback
feedback_loop
“we further instruction-tuned the models, aligning them with human feedback to ensure they generated safe and contextually appropriate content”
7
Deploy e-Llama for AI initiatives
output
“the model training is enabling eBay to leverage proprietary and open LLMs to drive new AI initiatives across the company”
Reported outcome

The e-Llama models achieve approximately 25% improvement in e-commerce benchmarks for English and about 30% for non-English, while retaining general-domain performance with only 1% degradation on NLU benchmarks for the 70B model.

Reported metrics
e-commerce benchmark improvement (English)approximately 25%
e-commerce benchmark improvement (non-English)about 30%
general domain NLU benchmark degradation (e-Llama 70B)1%
Total training tokens1 trillion tokens
Show all 6 reported metrics
e-commerce benchmark improvement (English)approximately 25%
e-commerce benchmark improvement (non-English)about 30%
general domain NLU benchmark degradation (e-Llama 70B)1%
total training tokens1 trillion tokens
70B model training durationaround one month
70B model GPU-hoursabout 340k GPU-hours
Reported stack
Llama 3.1Megatron-LMflash-attention-2NVIDIA H100NVLinkInfiniBand
Source
https://innovation.ebayinc.com/stories/scaling-large-language-models-for-e-commerce-the-development-of-a-llama-based-customized-llm-for-e-commerce/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The e-Llama models achieve approximately 25% improvement in e-commerce benchmarks for English and about 30% for non-English, while retaining general-domain performance with only 1% degradation on NLU benchmarks for th…

What tools did this team use?

Llama 3.1, Megatron-LM, flash-attention-2, NVIDIA H100, NVLink, InfiniBand.

What results were reported?

e-commerce benchmark improvement (English): approximately 25%; e-commerce benchmark improvement (non-English): about 30%; general domain NLU benchmark degradation (e-Llama 70B): 1%; Total training tokens: 1 trillion tokens (source-reported, not independently verified).

What failed first in this deployment?

Third-party LLMs such as GPT-4 and Claude were found impractical for eBay's needs due to cost, data security risks, and limited fine-tuning on proprietary data.

How is this ecommerce ops AI workflow structured?

Identify LLM adaptation need → Collect and filter e-commerce data → Train e-commerce domain classifier → Continued pretraining on domain data → Optimize training setup via experiments → Instruction-tune with human feedback → Deploy e-Llama for AI initiatives.

Related ecommerce ops cases