Ecommerce ops · Production

Mercari fine-tunes a 2B-parameter LLM with QLoRA to extract listing attributes, outperforming GPT-3.5 Turbo at 14× lower cost

The problem

Extracting structured attributes from Mercari's diverse user-generated listing descriptions was difficult due to wide seller variability, category-specific attribute definitions, and continuously evolving content. Commercial LLM APIs were prohibitively expensive at production scale, and conventional ML models required continuous retraining as attribute definitions changed.

Workflow diagram · grounded in source

Listing description input

trigger

“Input: A text description of the listing and a list of attribute keys to extract.”

Historical dataset collection

integration

“we gathered historical descriptions along with their attributes. Since attribute keys can vary across item categories, we started by focusing on the 20 categories with the highest listings on our platform.”

QLoRA fine-tuning

ai_action

“we used a single A100 GPU with an 80 GB memory VM instance (a2-ultragpu-1g ) from GCP to fine-tune a large language model using QLoRA”

Post-training quantization

output

“We used the llama.cpp library—an open-source tool that enables post-training model quantization and faster inference using LLMs in C/C++”

BLEU score evaluation

validation

“BLEU Score: This score provided insights into the quality of extracted attribute values compared to the actual values.”

Attribute values extracted

output

“Output: The extracted attributes and their values.”

Reported outcome

The QLoRA-fine-tuned gemma-2b-it model is approximately 95% smaller than the original base model, achieves a BLEU score slightly more than five percentage points higher than GPT-3.5 turbo, and is estimated to cost more than 14 times less to run in production.

Reported metrics

Model size reduction vs base modelapproximately 95% smaller

BLEU score improvement over GPT-3.5 turboslightly more than five percentage points higher

cost reduction vs GPT-3.5 turbomore than 14 times

Reported stack

QLoRAgemma-2b-itGPT-3.5 turbollama.cppHuggingFaceW&BSFTTrainerGCP

Source

https://engineering.mercari.com/en/blog/entry/20240913-fine-tuning-an-llm-to-extract-dynamically-specified-attributes/

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The QLoRA-fine-tuned gemma-2b-it model is approximately 95% smaller than the original base model, achieves a BLEU score slightly more than five percentage points higher than GPT-3.5 turbo, and is estimated to cost mor…

What tools did this team use?

QLoRA, gemma-2b-it, GPT-3.5 turbo, llama.cpp, HuggingFace, W&B, SFTTrainer, GCP.

What results were reported?

Model size reduction vs base model: approximately 95% smaller; BLEU score improvement over GPT-3.5 turbo: slightly more than five percentage points higher; cost reduction vs GPT-3.5 turbo: more than 14 times (source-reported, not independently verified).

How is this ecommerce ops AI workflow structured?

Listing description input → Historical dataset collection → QLoRA fine-tuning → Post-training quantization → BLEU score evaluation → Attribute values extracted.