Compliance monitoring · Production

Clio: Anthropic's privacy-preserving system for analyzing real-world Claude usage at scale

The problem

AI providers lacked a scalable way to understand how their models were actually being used in practice while rigorously protecting user privacy; traditional top-down safety approaches required knowing what to look for in advance and could not discover unknown usage patterns or coordinated misuse.

First attempt

Anthropic's pre-existing Trust and Safety classifiers produced both false negatives (failing to flag policy violations in translation requests) and false positives (incorrectly flagging job seekers' resumes, security programming questions, and Dungeons & Dragons content as harmful).

Workflow diagram · grounded in source
1
Conversation intake trigger
trigger
“We used Clio to analyze 1 million conversations with Claude on claude.ai (both the Free and Pro tiers)”
2
Facet extraction
ai_action
“Extracting facets: For each conversation, Clio extracts multiple "facets"—specific attributes or metadata such as the conversation topic, number of back-and-forth turns in the conversation, or the language used.”
3
Semantic clustering
ai_action
“Semantic clustering: Similar conversations are automatically grouped together by theme or general topic.”
4
Cluster description generation
ai_action
“Cluster description: Each cluster receives a descriptive title and summary that captures common themes from the raw data while excluding private information.”
5
Privacy verification
validation
“Claude verifies that cluster summaries don't contain any overly specific or identifying information before they're displayed to the human user.”
6
Hierarchy and interface output
output
“Building hierarchies: Clusters are organized into a multi-level hierarchy for easier exploration. They can then be presented in an interactive interface that analysts at Anthropic can use to explore patterns across different dimensions (…”
7
Trust and Safety human review
human_review
“Our Trust and Safety team is able to review topic clusters for areas that indicate likely violations of our Usage Policy.”
8
Enforcement or safety action
output
“Our Trust and Safety teams can use this bottom-up review approach to identify individual accounts for further review and, if appropriate, take action in accordance with our terms and policies.”
Reported outcome

Clio identified a coordinated SEO spam network that evaded individual-conversation review, monitored for AI misuse during the 2024 US General Election, and helped reduce both false positives and false negatives in Anthropic's existing Trust and Safety classifiers.

Reported metrics
Conversations analyzed1 million
Share of conversations: web and mobile app developmentover 10%
Share of conversations: teaching and learningmore than 7%
Share of conversations: business strategy and operationsnearly 6%
Reported stack
Claude
Source
https://www.anthropic.com/research/clio
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Clio identified a coordinated SEO spam network that evaded individual-conversation review, monitored for AI misuse during the 2024 US General Election, and helped reduce both false positives and false negatives in Ant…

What tools did this team use?

Claude.

What results were reported?

Conversations analyzed: 1 million; Share of conversations: web and mobile app development: over 10%; Share of conversations: teaching and learning: more than 7%; Share of conversations: business strategy and operations: nearly 6% (source-reported, not independently verified).

What failed first in this deployment?

Anthropic's pre-existing Trust and Safety classifiers produced both false negatives (failing to flag policy violations in translation requests) and false positives (incorrectly flagging job seekers' resumes, security…

How is this compliance monitoring AI workflow structured?

Conversation intake trigger → Facet extraction → Semantic clustering → Cluster description generation → Privacy verification → Hierarchy and interface output → Trust and Safety human review → Enforcement or safety action.

Related compliance monitoring cases