Marketing ops · Production

Klaviyo Segments AI: automating customer segment creation with prompt chaining and few-shot LLM agents

The problem

Klaviyo users at opposite ends of the experience spectrum both failed to get the most from segmentation: less experienced users could not translate their concepts into segment logic, while highly experienced users spent hours a week on repetitive segment building.

Workflow diagram · grounded in source
1
User describes desired segment
trigger
“users build segments of their customers using logical conditions — usually based on things that customers have/have not done or properties about those customers.”
2
Parallel atomic LLM requests
ai_action
“Segments AI makes extensive use of prompt chaining internally. That means when we generate content with an LLM, we try to make each request as small, atomic, and simple as possible.”
3
Few-shot subtask agent specialists
ai_action
“this process can be done implicitly by seeding training examples into the system instruction. We find best performance when these training examples are annotated with input, ideal output, and the generalizable lesson to be learned.”
4
Segment JSON assembled
output
“paste together these results into a final composition”
5
LLM and manual validation
validation
“we decided to use a mix of LLMs evaluating LLMs and hand-designed test cases. The validation suite we created was packaged such that it could be run before major changes to the codebase.”
Reported outcome

Segments AI was released in November 2023 to Klaviyo's 130,000 customers, giving inexperienced users a simpler way to interact with segments and giving experienced users a way to automate creating complex segments.

Reported metrics
customers reached by Segments AI130,000
Time experienced users spend building repetitive segmentshours a week
Reported stack
OpenAIChatGPT
Source
https://klaviyo.tech/building-segments-ai-fe33d9cab822
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Segments AI was released in November 2023 to Klaviyo's 130,000 customers, giving inexperienced users a simpler way to interact with segments and giving experienced users a way to automate creating complex segments.

What tools did this team use?

OpenAI, ChatGPT.

What results were reported?

customers reached by Segments AI: 130,000; Time experienced users spend building repetitive segments: hours a week (source-reported, not independently verified).

How is this marketing ops AI workflow structured?

User describes desired segment → Parallel atomic LLM requests → Few-shot subtask agent specialists → Segment JSON assembled → LLM and manual validation.

Related marketing ops cases