Back office ops · Production

ART·E: Building an Email Research Agent with Reinforcement Learning That Beats o3

The problem

Searching an email inbox to answer natural-language questions required manually opening the inbox, devising keywords, and reading through search results — a process the author wanted AI to replace.

Workflow diagram · grounded in source
1
User submits email query
trigger
“The agent is given an initial prompt defining the task, which is to answer the user's question based on the contents of their inbox.”
2
Search inbox for emails
ai_action
“search_emails(keywords, sent_after, sent_before): finds up to 10 emails matching the given keywords with date filters applied, and returns the message IDs as well as matching snippets”
3
Read full email body
ai_action
“read_email(message_id): returns the full email body for the given message ID”
4
Return answer with sources
output
“return_final_answer(answer: str, sources: list[str]): returns the final answer to the user's question, and the list of message IDs that supported the answer”
5
Score trajectories
validation
“Score all 4 trajectories with our reward function”
6
GRPO model update
feedback_loop
“Use all 12 groups of 4 trajectories (so 48 trajectories total) along with their rewards to calculate the loss using the GRPO formula and update the model”
Reported outcome

ART·E is faster, cheaper, and more accurate than o3 on the email search task, taking almost a full turn less per query while hallucinating fewer wrong answers.
Training cost approximately $80 and completed in just under a day.

Reported metrics
Accuracy vs o3faster, cheaper, and more accurate than o3
Turns taken vs o3almost a full turn less than o3
Hallucinations vs o3hallucinated fewer wrong ones
Training costabout $80
Show all 5 reported metrics
accuracy vs o3faster, cheaper, and more accurate than o3
turns taken vs o3almost a full turn less than o3
hallucinations vs o3hallucinated fewer wrong ones
training costabout $80
training durationjust under a day
Reported stack
ARTGRPOsqliteFTS5vLLMUnslothSkypilotRunpodWeights & Biases
Source
https://openpipe.ai/blog/art-e-mail-agent
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

ART·E is faster, cheaper, and more accurate than o3 on the email search task, taking almost a full turn less per query while hallucinating fewer wrong answers.

What tools did this team use?

ART, GRPO, sqlite, FTS5, vLLM, Unsloth, Skypilot, Runpod, Weights & Biases.

What results were reported?

Accuracy vs o3: faster, cheaper, and more accurate than o3; Turns taken vs o3: almost a full turn less than o3; Hallucinations vs o3: hallucinated fewer wrong ones; Training cost: about $80 (source-reported, not independently verified).

How is this back office ops AI workflow structured?

User submits email query → Search inbox for emails → Read full email body → Return answer with sources → Score trajectories → GRPO model update.

Related back office ops cases