Marketing ops · Production

Microsoft ISE develops multi-comparator pipeline to validate AI-generated ad image integrity

The problem

An advertising customer needed to scale 1:1 ad personalization using AI-generated (inpainting) backgrounds but had no way to verify that the product image remained unmodified in the generated output.

Workflow diagram · grounded in source
1
AI inpainting generates ad background
ai_action
“With inpainting, the inputs are: - An image of the product - A mask representing the area of the background to be generated - A textual prompt describing the background to be generated. The output is an image where the masked area has be…”
2
Template matching locates product
validation
“Template matching serves to provide a baseline of which to compare the images. By finding where the product is in the generated image, we can scope the comparison of only the relevant pixels. This helps when running MSE and PSNR”
3
MSE and PSNR pixel comparison
validation
“With MSE and PSNR, the key benefit is in determining color differences and disproportionate scaling”
4
Cosine similarity feature comparison
validation
“Cosine similarity performed well in detecting changes in the edges and curves that define the product. This means that it has less need for a template matcher. So, if the image was translated or proportionately scaled, it will accurately…”
5
Combined integrity determination
output
“With the combination of template matching, MSE, PSNR, and Cosine Similarity, we can achieve a strong system of image comparison to determine whether the product has been edited in the AI-generated image output”
Reported outcome

Combining template matching with MSE (or PSNR) and Cosine Similarity produced a strong system that can determine whether the product was edited in the AI-generated image, with each comparator covering the others' shortcomings.

Reported metrics
Combined image comparison system qualitystrong system of image comparison
MSE pixel change detection capabilityperformed well in accurately detecting changes in exact pixel differences
Cosine Similarity product edge detection capabilityperformed well in detecting changes in the edges and curves that define the product
Reported stack
OpenCVVGG16OpenAIChatGPTGimpmatplotlibnumpy
Source
https://devblogs.microsoft.com/ise/ai-ad-image-differential/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Combining template matching with MSE (or PSNR) and Cosine Similarity produced a strong system that can determine whether the product was edited in the AI-generated image, with each comparator covering the others' shor…

What tools did this team use?

OpenCV, VGG16, OpenAI, ChatGPT, Gimp, matplotlib, numpy.

What results were reported?

Combined image comparison system quality: strong system of image comparison; MSE pixel change detection capability: performed well in accurately detecting changes in exact pixel differences; Cosine Similarity product edge detection capability: performed well in detecting changes in the edges and curves that define the product (source-reported, not independently verified).

How is this marketing ops AI workflow structured?

AI inpainting generates ad background → Template matching locates product → MSE and PSNR pixel comparison → Cosine similarity feature comparison → Combined integrity determination.

Related marketing ops cases