Marketing ops · Production

Acxiom uses LangSmith to scale AI-driven audience segmentation with LLM observability

The problem

Acxiom needed to build a scalable generative AI application for dynamic audience segmentation from natural language input, but their initial lightweight logging solution could not scale as the user base grew, and complex multi-step LLM reasoning produced hallucinated or incorrect results.

First attempt

Acxiom's initial prompt input/output logging system was too lightweight for scale, their RAG-based agentic architecture suffered from cascading debugging failures, and hallucinations arose from LLM reasoning gaps.

Workflow diagram · grounded in source
1
Natural language audience request
trigger
“a user might request: "Identify an audience of men over thirty who rock climb or hike but aren't married"”
2
RAG retrieves data catalog
ai_action
“LangChain's Retrieval-Augmented Generation (RAG) tools with custom agentic code. The RAG workflow would only use metadata and the data dictionary of Acxiom's core data products with detailed descriptions”
3
Overseer and researcher agents
ai_action
“new agents, such as an overseer and researcher agent, were added to the architecture for more nuanced decision-making related to audience-building”
4
LangSmith trace and observe
validation
“the tree-structured trace visualization and metadata tracking tools in LangSmith were particularly helpful. These helped the Acxiom team identify bottlenecks in requests that involved more than 60 LLM calls and 200k tokens for a single u…”
5
Audience JSON output
output
“deliver a JSON structure containing curated IDs and values from Acxiom's transactional and predictive data products”
Reported outcome

By adopting LangSmith, Acxiom achieved streamlined debugging of complex LLM chains, more accurate and dynamic audience segment creation, scalable growth without reengineering the observability layer, and optimized token usage through better cost visibility.

Reported metrics
LLM calls per single user interactionmore than 60
Tokens per single user interaction200k
Application improvementssignificant improvements across their application
Reported stack
LangSmithLangChainRAGvLLMClaudeAWS BedrockDatabricks
Source
https://blog.langchain.dev/customers-acxiom/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

By adopting LangSmith, Acxiom achieved streamlined debugging of complex LLM chains, more accurate and dynamic audience segment creation, scalable growth without reengineering the observability layer, and optimized tok…

What tools did this team use?

LangSmith, LangChain, RAG, vLLM, Claude, AWS Bedrock, Databricks.

What results were reported?

LLM calls per single user interaction: more than 60; Tokens per single user interaction: 200k; Application improvements: significant improvements across their application (source-reported, not independently verified).

What failed first in this deployment?

Acxiom's initial prompt input/output logging system was too lightweight for scale, their RAG-based agentic architecture suffered from cascading debugging failures, and hallucinations arose from LLM reasoning gaps.

How is this marketing ops AI workflow structured?

Natural language audience request → RAG retrieves data catalog → Overseer and researcher agents → LangSmith trace and observe → Audience JSON output.

Related marketing ops cases