Back office ops · Production

Square builds RoBERTa-based merchant categorization model achieving ~30% accuracy improvement

The problem

Square's merchant categorization relied heavily on self-selection during onboarding, which was highly susceptible to miscategorization. Historical ML approaches used older methods and over-indexed on self-selected data as ground truth, leading to inefficiencies and missed opportunities.

First attempt

Previous ML attempts at Square used older methods, over-indexed on self-selected onboarding data as labels, and narrowly focused on new seller onboarding without leveraging post-onboarding signals.

Workflow diagram · grounded in source
1
Daily inference trigger
trigger
“This model performed inference daily and, given the vast number of sellers on the platform (in the tens of millions!)”
2
Remove auto-created services
validation
“we remove all auto-created services that were not later manually modified by the seller”
3
Rank catalog by purchase frequency
ai_action
“we rank their catalog by the amount of times that item was purchased, and subsequently cut off any items beyond the token threshold”
4
RoBERTa merchant classification
ai_action
“the RoBERTa model can make highly accurate predictions about a merchant's category, significantly improving upon previous methods”
5
Incremental prediction check
validation
“We avoided running predictions for sellers whose inputs remained unchanged from the previous day”
6
Output prediction tables
output
“The model produced two key output tables to serve distinct purposes”
7
Power business metrics
integration
“model output, which refreshes on a daily basis, is now utilized by Square to power all business metrics that require segmentation by merchant categories/audiences”
Reported outcome

The RoBERTa model achieved an absolute ~30% improvement in categorization accuracy over existing methods, with per-category accuracy gains including 13% for food_and_drink and 38% for retail, and now powers all Square business metrics requiring merchant category segmentation.

Reported metrics
Overall categorization accuracy improvement~30%
Accuracy increase - beauty and personal care32%
Accuracy increase - food and drink13%
Accuracy increase - health care and fitness22%
Show all 9 reported metrics
overall categorization accuracy improvement~30%
accuracy increase - beauty_and_personal_care32%
accuracy increase - food_and_drink13%
accuracy increase - health_care_and_fitness22%
accuracy increase - home_and_repair35%
accuracy increase - leisure_and_entertainment18%
accuracy increase - professional_services17%
accuracy increase - retail38%
training data size (manually reviewed sellers)over 20,000
Reported stack
RoBERTaDatabricksPySparkHuggingface
Source
https://developer.squareup.com/blog/roberta-model-for-merchant-categorization-at-square/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The RoBERTa model achieved an absolute ~30% improvement in categorization accuracy over existing methods, with per-category accuracy gains including 13% for food_and_drink and 38% for retail, and now powers all Square…

What tools did this team use?

RoBERTa, Databricks, PySpark, Huggingface.

What results were reported?

Overall categorization accuracy improvement: ~30%; Accuracy increase - beauty and personal care: 32%; Accuracy increase - food and drink: 13%; Accuracy increase - health care and fitness: 22% (source-reported, not independently verified).

What failed first in this deployment?

Previous ML attempts at Square used older methods, over-indexed on self-selected onboarding data as labels, and narrowly focused on new seller onboarding without leveraging post-onboarding signals.

How is this back office ops AI workflow structured?

Daily inference trigger → Remove auto-created services → Rank catalog by purchase frequency → RoBERTa merchant classification → Incremental prediction check → Output prediction tables → Power business metrics.

Related back office ops cases