Finance ops ·

OpenAI builds real-time access engine combining rate limits and credits for Codex and Sora

The problem

Users of Codex and Sora were hitting hard rate limits after finding real value, while neither pure rate limiting nor asynchronous usage-based billing alone could balance fairness, capacity, and user experience.

First attempt

Third-party usage billing and metering platforms were evaluated but failed to support real-time credit decisions and full observability into why requests were allowed or blocked.

Workflow diagram · grounded in source
1
User request arrives
trigger
“Every request passes through a single evaluation path that makes a real‑time decision about how much usage is allowed”
2
Rate limit consumption
validation
“Enforce rate limits until they're reached”
3
Credit balance verification
routing
“synchronously consuming from rate limits and, if needed, verifying sufficient credits”
4
Definitive outcome returned
output
“it then returns one definitive outcome while settling any credit debits asynchronously”
5
Async credit debit
integration
“We decrease the Credit Balance and insert a Balance Update record in a single atomic database transaction. Balance updates are serialized per account, so concurrent requests can never race to spend the same credits.”
6
Audit and reconciliation
feedback_loop
“These datasets aren't a casual by-product; they actually drive the system, with each dataset triggering the next. Separating what occurred, any associated charges, and what we debited lets us independently audit, replay, and reconcile ev…”
Reported outcome

OpenAI built an in-house distributed system that fuses rate limits and credits into a provably correct, real-time access model, enabling users to work longer without hard stops or billing inaccuracies.

Reported metrics
User continuity of accesspeople can work longer, explore more deeply, and take projects further without facing hard stops
Billing correctnessprovably correct
Double-charging preventionatomic consumption prevents double-charging
Reported stack
CodexSora
Source
https://openai.com/index/beyond-rate-limits/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

OpenAI built an in-house distributed system that fuses rate limits and credits into a provably correct, real-time access model, enabling users to work longer without hard stops or billing inaccuracies.

What tools did this team use?

Codex, Sora.

What results were reported?

User continuity of access: people can work longer, explore more deeply, and take projects further without facing hard stops; Billing correctness: provably correct; Double-charging prevention: atomic consumption prevents double-charging (source-reported, not independently verified).

What failed first in this deployment?

Third-party usage billing and metering platforms were evaluated but failed to support real-time credit decisions and full observability into why requests were allowed or blocked.

How is this finance ops AI workflow structured?

User request arrives → Rate limit consumption → Credit balance verification → Definitive outcome returned → Async credit debit → Audit and reconciliation.

Related finance ops cases