Ticket triage · Production

Wayfair automates supplier ticket triage with a hybrid LLM-agent workflow (Wilma)

The problem

Wayfair's supplier associates performed manual, rote ticket triage — reviewing unstructured ticket data, looking up supplier IDs in a database, and entering structured fields by hand — causing slower response times and accuracy problems in classification.

First attempt

A fully agentic design using a supervisor coordinating multiple sub-agents failed due to inter-agent communication errors and unnecessary repeated calls; in one case an agent incorrectly decided not to invoke the JIRA API to update the ticket.

Workflow diagram · grounded in source
1
Supplier email creates ticket
Trigger
When a supplier emails Wayfair, a new ticket is created in SupportHub and a Pub/Sub event invokes the LangGraph orchestration graph.
▾ source quote
“When a supplier emails Wayfair a new ticket is created in SupportHub, and a webhook publishes an event to a Pub/Sub topic. Our Pub/Sub consumer listens for these events and when it sees one corresponding to the creation of a …”
2
LLM intent classification
Ai action
An LLM identifies the question type via intent classification from the ticket content.
▾ source quote
“Use an LLM to identify the question type (i.e., intent classification)”
3
LLM language identification
Ai action
An LLM identifies the supplier's preferred language from the ticket.
▾ source quote
“Use an LLM to identify the supplier's preferred language”
4
ReAct agent supplier ID lookup
Ai action
A ReAct LLM agent with BigQuery access iteratively determines the supplier ID by reasoning over ticket data and adapting its queries based on results.
▾ source quote
“the reasoning+acting ReAct LLM agent is given a high level task ("Identify the supplier ID given the information on the ticket") and access to a BigQuery tool. A typical chain of reasoning and action steps might look like this: - …”
5
JIRA API ticket update
Output
The JIRA API is used to update the ticket with the identified structured information.
▾ source quote
“Use the JIRA API to update the ticket information”
Reported outcome

The hybrid Wilma workflow achieved better-than-human performance with 93% accuracy on question type (vs.
75% for humans), 98% on language, and 88% on supplier ID, while reducing ticket processing time and freeing associates for high-value work.

Reported metrics
Question type identification accuracy93%
Language identification accuracy98%
supplier ID identification accuracy88%
Human question type accuracy (baseline)75%
Show all 5 reported metrics
question type identification accuracy93%
language identification accuracy98%
supplier ID identification accuracy88%
human question type accuracy (baseline)75%
ticket processing timereduces the time it takes to process a ticket
Reported stack
WilmaLangGraphPub/SubBigQueryJIRA APIArizeSupportHubReAct
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://www.aboutwayfair.com/careers/tech-blog/automating-supplier-ticket-management-with-llm-agents-lessons-from-the-field
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The hybrid Wilma workflow achieved better-than-human performance with 93% accuracy on question type (vs.

What tools did this team use?

Wilma, LangGraph, Pub/Sub, BigQuery, JIRA API, Arize, SupportHub, ReAct.

What results were reported?

Question type identification accuracy: 93%; Language identification accuracy: 98%; supplier ID identification accuracy: 88%; Human question type accuracy (baseline): 75% (source-reported, not independently verified).

What failed first in this deployment?

A fully agentic design using a supervisor coordinating multiple sub-agents failed due to inter-agent communication errors and unnecessary repeated calls; in one case an agent incorrectly decided not to invoke the JIRA…

How is this ticket triage AI workflow structured?

Supplier email creates ticket → LLM intent classification → LLM language identification → ReAct agent supplier ID lookup → JIRA API ticket update.

Related ticket triage cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Ticket triage cases →