Compliance monitoring · Production

Grab automates PetaByte-scale data classification with LLM, scanning 20,000+ entities per month

The problem

Grab managed PetaByte-level data across countless schemas, but manual PII campaigns were inconsistent — developers interpreted classification policies differently — and the data volume made table-level manual tagging infeasible. The existing third-party classification tool could not be customised and its regex classifiers generated too many false positives.

First attempt

The initial Gemini service built around a third-party classification tool had two blocking gaps: its ML classifiers could not be customised to Grab's internal taxonomy, and regex patterns produced excessive false positives. Building a bespoke in-house model was equally impractical due to the labelling and data-science investment required.

Workflow diagram · grounded in source
1
Data platform scan request
Trigger
Data platforms trigger scan requests to the Gemini service to initiate the tag classification process.
▾ source quote
“Data platforms trigger scan requests to the Gemini service to initiate the tag classification process”
2
Request batching and rate limiting
Routing
Gemini aggregates incoming requests into mini-batches via a message queue at fixed intervals, with a rate limiter attached at the workflow level.
▾ source quote
“it helps aggregate the requests into reasonable mini-batches. Aggregation is achievable through the message queue at fixed intervals. In addition, a rate limiter is attached at the workflow level”
3
LLM column-level tag classification
Ai action
The language model acts as a column tag generator and assigns the most appropriate tag to each column based on table name and column name.
▾ source quote
“We ask the language model to be a column tag generator and to assign the most appropriate tag to each column”
4
Predictions published to Kafka
Output
Classified tag predictions are published to the Kafka queue for downstream data platforms to consume.
▾ source quote
“The predictions are published to the Kafka queue to downstream data platforms”
5
Data owner tag verification
Human review
Data owners are notified weekly to verify the classified tags.
▾ source quote
“The platforms inform respective users weekly to verify the classified tags to improve the model's correctness”
6
Iterative prompt improvement
Feedback loop
User verification feedback is used to enable iterative prompt improvement.
▾ source quote
“to improve the model's correctness and to enable iterative prompt improvement”
Reported outcome

Within a month of rollout the LLM-powered system scanned over 20,000 data entities at 300–400 per day, saving an estimated 360 man-days per year.
Eighty percent of data owners reported the new tagging process helped them, and acknowledged tables required fewer than one tag change on average.

Reported metrics
Data entities scanned within one month of rolloutmore than 20,000
Daily classification throughput300-400 entities per day
Annual time savedapproximately 360 man-days per year
Data owner satisfaction with new tagging process80%
Show all 5 reported metrics
data entities scanned within one month of rolloutmore than 20,000
daily classification throughput300-400 entities per day
annual time savedapproximately 360 man-days per year
data owner satisfaction with new tagging process80%
average tags changed per acknowledged tableless than one tag
Reported stack
GeminiGPT3.5Azure OpenAIKafka
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://engineering.grab.com/llm-powered-data-classification
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Within a month of rollout the LLM-powered system scanned over 20,000 data entities at 300–400 per day, saving an estimated 360 man-days per year.

What tools did this team use?

Gemini, GPT3.5, Azure OpenAI, Kafka.

What results were reported?

Data entities scanned within one month of rollout: more than 20,000; Daily classification throughput: 300-400 entities per day; Annual time saved: approximately 360 man-days per year; Data owner satisfaction with new tagging process: 80% (source-reported, not independently verified).

What failed first in this deployment?

The initial Gemini service built around a third-party classification tool had two blocking gaps: its ML classifiers could not be customised to Grab's internal taxonomy, and regex patterns produced excessive false posi…

How is this compliance monitoring AI workflow structured?

Data platform scan request → Request batching and rate limiting → LLM column-level tag classification → Predictions published to Kafka → Data owner tag verification → Iterative prompt improvement.

Related compliance monitoring cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Compliance monitoring cases →