Dropbox builds in-house deep learning OCR pipeline for mobile document scanner
The problem
Dropbox's commercial off-the-shelf OCR SDK was expensive (charged per scan) and tuned for flat-bed scanners, not mobile photos with crinkled documents, shadows, and uneven lighting; Dropbox also lacked control over future innovation.
First attempt
When the Word Detector and Word Deep Net were first chained end-to-end, accuracy dropped to around 44%—far below the competition—due to spacing errors and spurious garbage text from image noise.
Workflow diagram · grounded in source
1
Mobile document upload
Trigger
Mobile clients upload scanned document images to an in-house asynchronous work queue.
▾ source quote
“mobile clients upload scanned document images to our in-house asynchronous work queue. When the upload is finished, we then send the image via a Remote Procedure Call (RPC) to a cluster of servers running the OCR service”
2
Orientation detection
Ai action
A deep net based on Inception Resnet v2 predicts the image orientation and rotates it upright before OCR processing.
▾ source quote
“We built an orientation predictor using another deep net based on the Inception Resnet v2 architecture, changed the final layer to predict orientation, collected an orientation training and validation data set, and fine-tuned from an ImageNet-trained model biased towards our …”
3
Word Detector segments image
Ai action
The Word Detector uses MSER features via OpenCV to detect and segment words and lines from the document image.
▾ source quote
“Our Word Detector first detects MSER features in an image, then strings these together into word and line detections”
4
Word Deep Net recognizes text
Ai action
A CNN followed by a Bidirectional LSTM and a CTC layer converts each word image into predicted text.
▾ source quote
“Convolutional Neural Net (CNN)with several convolutional layers. The visual features that are output by the CNN are then fed as a sequence to a Bidirectional LSTM ( Long Short Term Memory ) — common in speech recognition systems — which …”
5
Confidence scoring and lexicon validation
Validation
Each prediction is bucketed by confidence: high-confidence kept as-is, low-confidence filtered out, mid-confidence validated against an Oxford English Dictionary lexicon.
▾ source quote
“If the confidence was high, we kept the prediction as is. If the confidence was low, we simply filtered them out, making a bet that these were noise predictions. If the confidence was somewhere in the middle, we then ran …”
6
Wordinator post-processing
Output
The Wordinator produces discrete bounding boxes and OCRed text for each individual word.
▾ source quote
“we run these outputs along with the original outputs from the Word Detector through a module we call the Wordinator, which gives discrete bounding boxes for each individual OCRed word. This results in individual word coordinates along with their OCRed …”
7
PDF hidden layer and search index
Output
OCRed text is merged into the PDF as a hidden layer and added to Dropbox's search index.
▾ source quote
“we merge them back into the original PDF produced by the mobile document scanner as an OCR hidden layer. The OCRed text is also added to Dropbox's search index. The user can now highlight and copy-paste text from the PDF, …”
Reported outcome
After about 8 months of research, productionization, and refinement, Dropbox deployed a state-of-the-art OCR pipeline to millions of Dropbox Business users, achieving mid-90s Single Word Accuracy and replacing the commercial SDK entirely.
Reported metrics
Single Word Accuracy after initial synthetic trainingaround 79%
Single Word Accuracy after fine-tuning on real imagesmid-90s
Initial end-to-end accuracy44%
deployment coverage of Dropbox Business users100%
Show all 6 reported metrics
Single Word Accuracy after initial synthetic trainingaround 79%
Single Word Accuracy after fine-tuning on real imagesmid-90s
Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.
After about 8 months of research, productionization, and refinement, Dropbox deployed a state-of-the-art OCR pipeline to millions of Dropbox Business users, achieving mid-90s Single Word Accuracy and replacing the com…
Single Word Accuracy after initial synthetic training: around 79%; Single Word Accuracy after fine-tuning on real images: mid-90s; Initial end-to-end accuracy: 44%; deployment coverage of Dropbox Business users: 100% (source-reported, not independently verified).
What failed first in this deployment?
When the Word Detector and Word Deep Net were first chained end-to-end, accuracy dropped to around 44%—far below the competition—due to spacing errors and spurious garbage text from image noise.
How is this data entry ops AI workflow structured?
Mobile document upload → Orientation detection → Word Detector segments image → Word Deep Net recognizes text → Confidence scoring and lexicon validation → Wordinator post-processing → PDF hidden layer and search index.
This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.