Data entry ops · Production

MediaRadar | Vivvix achieves 150% increase in hourly ad throughput with Databricks Mosaic AI and Spark Structured Streaming

The problem

MediaRadar | Vivvix's ad classification of over 6 million unique products relied on manual operators and was bottlenecked by Amazon SQS polling constraints, leaving the company unable to keep pace with rapidly growing ad volumes or meet SLAs, while fragmented infrastructure across multiple pods made model monitoring nearly impossible.

First attempt

MediaRadar | Vivvix's existing ML models and in-house fine-tuned model were insufficient for the scale and diversity of over 6 million unique products due to lack of training data, and their SQS-based polling setup could not meet SLAs.

Workflow diagram · grounded in source
1
Real-time ad ingestion
trigger
“With the adoption of Spark Structured Streaming, data processing is now automated, enabling continuous and real-time data ingestion without manual intervention”
2
Ad preprocessing pipeline
ai_action
“preprocessing pipelines that included fingerprinting to identify duplicates, transcription and translation using models like Whisper and optical character recognition (OCR) to extract textual information from the ads”
3
GenAI product identification
ai_action
“using GenAI to identify products in ads”
4
Dual-layer classification and match selection
ai_action
“This innovative, dual-layer approach involved using GenAI to identify products in ads and then comparing the results with MediaRadar | Vivvix's own classification models to select the best matches, ensuring higher accuracy in identifying…”
5
Insights delivered to customers
output
“streamlined workflow that gives MediaRadar | Vivvix the confidence that they will deliver precise and timely insights to customers ranging from small brands to major industry leaders”
Reported outcome

After adopting Databricks, MediaRadar | Vivvix increased ad classification throughput from 800 to 2,000 creatives per hour — a 150% improvement — and reduced model experimentation from two days to half a day, while eliminating SLA concerns.

Reported metrics
Hourly ad throughput increase150%
creatives classified per hour before Databricks800 creatives an hour
creatives classified per hour after Databricks2,000 an hour
Model experimentation time reductiontwo days' worth of work now takes maybe half a day
Show all 5 reported metrics
hourly ad throughput increase150%
creatives classified per hour before Databricks800 creatives an hour
creatives classified per hour after Databricks2,000 an hour
model experimentation time reductiontwo days' worth of work now takes maybe half a day
unique products in classification scope6 million unique products
Reported stack
DatabricksDatabricks Mosaic AISpark Structured StreamingRay clusterWhisperOCRDatabricks Model ServingUnity CatalogAmazon Simple Queue Service (SQS)OpenAI's GPT-3.5
Source
https://www.databricks.com/customers/mediaradar-vivvix
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

After adopting Databricks, MediaRadar | Vivvix increased ad classification throughput from 800 to 2,000 creatives per hour — a 150% improvement — and reduced model experimentation from two days to half a day, while el…

What tools did this team use?

Databricks, Databricks Mosaic AI, Spark Structured Streaming, Ray cluster, Whisper, OCR, Databricks Model Serving, Unity Catalog, Amazon Simple Queue Service (SQS), OpenAI's GPT-3.5.

What results were reported?

Hourly ad throughput increase: 150%; creatives classified per hour before Databricks: 800 creatives an hour; creatives classified per hour after Databricks: 2,000 an hour; Model experimentation time reduction: two days' worth of work now takes maybe half a day (source-reported, not independently verified).

What failed first in this deployment?

MediaRadar | Vivvix's existing ML models and in-house fine-tuned model were insufficient for the scale and diversity of over 6 million unique products due to lack of training data, and their SQS-based polling setup co…

How is this data entry ops AI workflow structured?

Real-time ad ingestion → Ad preprocessing pipeline → GenAI product identification → Dual-layer classification and match selection → Insights delivered to customers.

Related data entry ops cases