Compliance monitoring · Production

Metasense V2: Grab enhances LLM-powered data governance to classify an entire data lake

The problem

Grab's internal metadata generation service relied on a third-party data classification tool with limited ML customisation options, leaving the growing data lake dependent on manual tagging by Grabbers and making it impossible to scale data governance efficiently.

First attempt

The first LLM model was overwhelmed by mixed PII/non-PII data: 13 of 21 tags targeted non-PII distinctions, consuming capacity needed for PII detection, while overly long prompts and large table inputs further degraded accuracy on edge cases such as business emails containing personal names and nested JSON with hidden PII.

Workflow diagram · grounded in source
1
Data table submitted for classification
trigger
“Instead of manually classifying all new or existing tables, Grabbers can now rely on our model to assign the appropriate classification tier accurately”
2
Large tables split into subtasks
ai_action
“Splitting tables with more than 150 columns into smaller tables. Fewer table rows means that the LLM has sufficient capacity to focus on each column”
3
PII tag classification
ai_action
“One part for adding PII tags. Reducing the number of tags for the first part from 21 to 8 by removing all non-PII tags. This simplifies the task of differentiating types of data.”
4
Non-PII tag classification
ai_action
“Another part for adding all other types of tags”
5
Human verification by table owners
human_review
“the data pipeline still requires owners to manually perform verification to prevent any misclassifications”
6
LangSmith prompt iteration
feedback_loop
“Data scientists can create, update, and evaluate prompts directly on the LangSmith user interface and save them in commit mode. For rapid deployment, the prompt identifier in service configurations can be easily adjusted.”
7
Misclassification rate monitoring
feedback_loop
“we have set up alerts to monitor misclassification rates periodically, sounding an internal alarm if the rate crosses a defined threshold. A model improvement protocol has also been set in place for such alarms”
Reported outcome

Metasense V2 now covers the vast majority of Grab's data lake tables, has significantly reduced the manual classification workload for Grabbers, and achieves exceptionally low misclassification rates supported by automated threshold alerts.

Reported metrics
Data entries scanned by initial modelmore than 20,000
Entities processed per day300-400
non-PII tags in original model13
Total tags in original model21
Show all 8 reported metrics
data entries scanned by initial modelmore than 20,000
entities processed per day300-400
non-PII tags in original model13
total tags in original model21
PII tags in revised model8
prompt word count reduction1,254 to 737 words
manual classification workloadsignificantly reduced the workload for Grabbers
misclassification rateexceptionally low misclassification rates
Reported stack
LangChainLangSmithLarge Language Models
Source
https://engineering.grab.com/metasense-v2
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Metasense V2 now covers the vast majority of Grab's data lake tables, has significantly reduced the manual classification workload for Grabbers, and achieves exceptionally low misclassification rates supported by auto…

What tools did this team use?

LangChain, LangSmith, Large Language Models.

What results were reported?

Data entries scanned by initial model: more than 20,000; Entities processed per day: 300-400; non-PII tags in original model: 13; Total tags in original model: 21 (source-reported, not independently verified).

What failed first in this deployment?

The first LLM model was overwhelmed by mixed PII/non-PII data: 13 of 21 tags targeted non-PII distinctions, consuming capacity needed for PII detection, while overly long prompts and large table inputs further degrade…

How is this compliance monitoring AI workflow structured?

Data table submitted for classification → Large tables split into subtasks → PII tag classification → Non-PII tag classification → Human verification by table owners → LangSmith prompt iteration → Misclassification rate monitoring.

Related compliance monitoring cases