Finance ops · Production

Dropbox applies gradient-boosted ML to optimize subscription payment retry timing

The problem

Dropbox's payment retry system relied on about ten static hardcoded rule sets that decayed in performance over time and required complex, time-consuming manual customer segmentation and A/B testing to maintain.

First attempt

An initial ML design using a separate model per payment attempt created unnecessary complexity and contradicted the goal of simplifying the billing system. Loading and running the model inside the Payments Platform directly caused significant bloat and prediction latencies averaging around two minutes.

Workflow diagram · grounded in source

Payment attempt fails

trigger

“When a payment attempt for a customer fails, the payments platform makes a request to the predict module to get the next best time to charge the customer”

Retrieve customer signals

integration

“The predict module collects the most recent usage and payments signals for customers, as well as information about the previous failure. This data is stored in Edgestore(the primary metadata storage system at Dropbox) using a daily sched…”

Generate charge-time prediction

ai_action

“The collected signals are sent to Predict Service via a gRPC call, which encodes the signals into a feature dataframe and then sends them to the model. The model returns the best ranked time for when to charge the customer.”

Log prediction results

output

“The predict module also logs the model's prediction, along with other relevant information that can be used for troubleshooting and analysis”

Schedule next charge attempt

output

“Once the payments service receives the best time to charge, it then uses it to schedule the next payment attempt, and stores that in Edgestore”

Monitor business and model metrics

feedback_loop

“We use Grafana dashboards and Vortex for monitoring our model, and infrastructure metrics. For business metrics we use Superset. All these live metrics, and dashboards help us proactively track the expected behavior of the model, enablin…”

Reported outcome

The ML system outperforms the rule-based approach and the individual-customer model is deployed in production.
Migrating to Predict Service reduced prediction latency from several minutes to under 300ms for 99% of predictions, and the system delivers an overall increase in payment charge success rates and reduction of collection time.

Reported metrics

prediction latency after Predict Service migrationunder 300ms for 99 percent of them

prediction latency before Predict Service migrationaround two minutes on average

Payment charge success ratesoverall increase in payment charge success rates

Collection timereduction of collection time

Reported stack

gradient boosted ranking modelPredict ServiceStormcrowEdgestoreGrafanaVortexSuperset

Source

https://dropbox.tech/machine-learning/optimizing-payments-with-machine-learning

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The ML system outperforms the rule-based approach and the individual-customer model is deployed in production.

What tools did this team use?

gradient boosted ranking model, Predict Service, Stormcrow, Edgestore, Grafana, Vortex, Superset.

What results were reported?

prediction latency after Predict Service migration: under 300ms for 99 percent of them; prediction latency before Predict Service migration: around two minutes on average; Payment charge success rates: overall increase in payment charge success rates; Collection time: reduction of collection time (source-reported, not independently verified).

What failed first in this deployment?

An initial ML design using a separate model per payment attempt created unnecessary complexity and contradicted the goal of simplifying the billing system.

How is this finance ops AI workflow structured?

Payment attempt fails → Retrieve customer signals → Generate charge-time prediction → Log prediction results → Schedule next charge attempt → Monitor business and model metrics.