Back office ops · Production

Dropbox brings AI-powered summarization and Q&A to web file previews using Riviera and LLMs

The problem

Knowledge workers suffer from information overload and need to understand large documents, videos, and other files without reading them in full or remembering exactly how content was worded or where it was located within a file.

First attempt

An earlier summary-of-summaries (map-reduce) approach repeated information across individual summaries, caused significant loss of overall file content, and compounded hallucination risk with each additional LLM call. The initial version also did not cache embeddings, resulting in redundant LLM calls for the same document.

Workflow diagram · grounded in source
1
File preview triggers AI request
trigger
“Dropbox can generate summaries and answer questions about files when those files are previewed on the web”
2
Riviera extracts text from file
integration
“the conversions we are interested are those that convert any file type to raw text. In the case of video, text extraction might looks something like: Video (.mp4) -> Audio (.aac) -> Transcript (.txt)”
3
Paragraph-level embedding generation
ai_action
“we split the text into paragraph-sized chunks and calculate an embedding for each chunk”
4
K-means clustering for summarization
ai_action
“Our summarization plugin takes the chunks and associated embeddings from the embeddings plugin and uses k-means clustering to group the text chunks from the file into clusters in this multi-dimensional embedding space. With this method, …”
5
Similarity-based chunk retrieval for Q&A
ai_action
“The Q&A plugin takes in the embeddings and text chunks from the embedding plugin and generates a new embedding for the user question. Then for each chunk of file text it computes the distance to the query text embedding. By calculating t…”
6
LLM generates response with follow-up questions
ai_action
“we also request context-relevant follow-up questions from the LLM. In testing we found that follow-up questions allow the user to more naturally learn about a file and the topic they are interested in. To gather these follow-up questions…”
7
Output returned with source references
output
“The relevant chunk locations are then returned to the user as sources, allowing them to reference the specific parts of the file that contributed to the answer”
Reported outcome

After optimization, cost-per-summary dropped by 93% and cost-per-query dropped by 64%.
P75 latency for summaries decreased from 115 seconds to 4 seconds, and p75 latency for queries decreased from 25 seconds to 5 seconds, making the feature more affordable and more responsive for users.

Reported metrics
Cost-per-summary reduction93%
Cost-per-query reduction64%
P75 summary latencyfrom 115 seconds to 4 seconds
P75 query latencyfrom 25 seconds to 5 seconds
Show all 7 reported metrics
cost-per-summary reduction93%
cost-per-query reduction64%
p75 summary latencyfrom 115 seconds to 4 seconds
p75 query latencyfrom 25 seconds to 5 seconds
topic diversity vs map-reduceapproximately 50% more
Riviera daily requestsabout 2.5 billion requests per day
supported file type conversionsabout 300 file types
Reported stack
RivieraLLMsk-means clustering
Source
https://dropbox.tech/machine-learning/bringing-ai-powered-answers-and-summaries-to-file-previews-on-the-web
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

After optimization, cost-per-summary dropped by 93% and cost-per-query dropped by 64%.

What tools did this team use?

Riviera, LLMs, k-means clustering.

What results were reported?

Cost-per-summary reduction: 93%; Cost-per-query reduction: 64%; P75 summary latency: from 115 seconds to 4 seconds; P75 query latency: from 25 seconds to 5 seconds (source-reported, not independently verified).

What failed first in this deployment?

An earlier summary-of-summaries (map-reduce) approach repeated information across individual summaries, caused significant loss of overall file content, and compounded hallucination risk with each additional LLM call.

How is this back office ops AI workflow structured?

File preview triggers AI request → Riviera extracts text from file → Paragraph-level embedding generation → K-means clustering for summarization → Similarity-based chunk retrieval for Q&A → LLM generates response with follow-up questions → Output returned with source references.

Related back office ops cases