quality_assurance · saas · workflow
Meta's Automated Compliance Hardening (ACH) tool uses LLMs to enable scalable mutation testing for compliance
Compliance at Meta relied on manual processes that were error-prone and hard to scale, while mutation testing — described as the most powerful form of software testing — faced five major barriers (scalability, unrealistic mutants, equivalent mutants, computational cost, and overstretching) that prevented its deployment at scale in large industrial codebases.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Engineer describes mutant
Engineers use plain-text prompts to describe the mutant they want to test.
Tools used
ACHLLMsKotlin
Outcome
ACH was deployed for privacy testing across Facebook, Instagram, WhatsApp, and Meta's wearables platforms. Privacy engineers accepted 73% of generated tests. The equivalence detector achieved precision of 0.95 and recall of 0.96 with simple preprocessing, transforming historically time-consuming compliance processes into systems that save engineer and developer time.
Results
Time savedsave engineer and developer time while also enhancing compliance
Volume73%
Running sinceOctober 2024
Grounding & classification
Source type: technical build writeup
29 fields verified against source quotes.
agentic workflowcode generationquality inspectioncode diff prhuman review describedmetric backednamed customerproduction runtime claimedtools describedworkflow describedmediasoftwareaccuracy improvementautomation rateemployee productivitytechnical build writeupcompliance monitoringquality assuranceai draft human approval