Ticket triage · Production

Wayfair automates supplier ticket triage with a hybrid LLM-agent workflow (Wilma)

The problem

Wayfair's supplier associates performed manual, rote ticket triage — reviewing unstructured ticket data, looking up supplier IDs in a database, and entering structured fields by hand — causing slower response times and accuracy problems in classification.

First attempt

A fully agentic design using a supervisor coordinating multiple sub-agents failed due to inter-agent communication errors and unnecessary repeated calls; in one case an agent incorrectly decided not to invoke the JIRA API to update the ticket.

Workflow diagram · grounded in source
1
Supplier email creates ticket
trigger
“When a supplier emails Wayfair a new ticket is created in SupportHub, and a webhook publishes an event to a Pub/Sub topic. Our Pub/Sub consumer listens for these events and when it sees one corresponding to the creation of a new ticket, …”
2
LLM intent classification
ai_action
“Use an LLM to identify the question type (i.e., intent classification)”
3
LLM language identification
ai_action
“Use an LLM to identify the supplier's preferred language”
4
ReAct agent supplier ID lookup
ai_action
“the reasoning+acting ReAct LLM agent is given a high level task ("Identify the supplier ID given the information on the ticket") and access to a BigQuery tool. A typical chain of reasoning and action steps might look like this: - "I don'…”
5
JIRA API ticket update
output
“Use the JIRA API to update the ticket information”
Reported outcome

The hybrid Wilma workflow achieved better-than-human performance with 93% accuracy on question type (vs.
75% for humans), 98% on language, and 88% on supplier ID, while reducing ticket processing time and freeing associates for high-value work.

Reported metrics
Question type identification accuracy93%
Language identification accuracy98%
supplier ID identification accuracy88%
Human question type accuracy (baseline)75%
Show all 5 reported metrics
question type identification accuracy93%
language identification accuracy98%
supplier ID identification accuracy88%
human question type accuracy (baseline)75%
ticket processing timereduces the time it takes to process a ticket
Reported stack
WilmaLangGraphPub/SubBigQueryJIRA APIArizeSupportHubReAct
Source
https://www.aboutwayfair.com/careers/tech-blog/automating-supplier-ticket-management-with-llm-agents-lessons-from-the-field
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The hybrid Wilma workflow achieved better-than-human performance with 93% accuracy on question type (vs.

What tools did this team use?

Wilma, LangGraph, Pub/Sub, BigQuery, JIRA API, Arize, SupportHub, ReAct.

What results were reported?

Question type identification accuracy: 93%; Language identification accuracy: 98%; supplier ID identification accuracy: 88%; Human question type accuracy (baseline): 75% (source-reported, not independently verified).

What failed first in this deployment?

A fully agentic design using a supervisor coordinating multiple sub-agents failed due to inter-agent communication errors and unnecessary repeated calls; in one case an agent incorrectly decided not to invoke the JIRA…

How is this ticket triage AI workflow structured?

Supplier email creates ticket → LLM intent classification → LLM language identification → ReAct agent supplier ID lookup → JIRA API ticket update.

Related ticket triage cases