Creative diagnostics: how computer vision analyzes what’s actually driving creative performance

TLDR: Creative diagnostics uses computer vision to analyze the physical components of ads: static images, video, and sound. It tells you not just which ad performs better, but which specific attributes of that ad are driving the difference. Connected to an ad server, it can use that intelligence to modify and re-serve optimized creative in real time during a live campaign. The differences in any individual element may be small, but the cumulative impact on cost and performance over a campaign can be significant.

Creative diagnostics uses a different branch of AI called computer vision, which is trained on static images and video, including sound. What that means in practice is that we can take any ad, including a TV ad, and the computer vision model will understand the components that make it up: its physical attributes, the way sound is used, where call to action buttons are placed, which objects appear, whether people are in it or not, which color references are used. All of it.

We do two things with that. The first is the diagnosis itself: understanding what's making up a set of ads for a given campaign. The second, if we're able to get access to the ad server where the ad is physically served to the consumer, is using that intelligence to modify the ad in real time. So if we know that a happy restaurant scene for a given product performs better than a marina scene for the same product, it's computer vision that gives us that data. We then use machine learning to rebuild the ad in real time and serve the optimized version during the course of the campaign.

The differences in any individual element may be small. But when you add up the performance over time, the difference in terms of cost and performance can be significant.

This is really interesting to look at. If you have a tracking server, you can see where individual consumers actually interact with an ad, and whether the call to action button is actually in the right place or not. That kind of detail is genuinely fascinating.

The distinction from standard creative testing is worth being clear about. If you take a traditional A/B test, what you know is that Creative A does better than Creative B in the north of England versus the south of England. That's interesting. What you don't know is what the difference between Creative A and Creative B is that's actually driven that difference. That's what creative diagnostics and optimization seeks to understand: which physical attributes of the ad in version A are actually driving the better performance than version B.

You can also combine this with segmentation. Different segments respond to different creative attributes, and once the model understands which physical elements are driving performance for which customer type, you can start making much more deliberate creative decisions rather than relying on intuition or broad testing.

At the moment creative diagnostics is a separate product, but the orchestration layer considers its output alongside the other models. So if we know a hundred things about Creative A, we can look at the relative importance of those hundred factors for segment one, segment two, and segment three. It's a lot of data, multidimensional again, and you need sophisticated models to make sense of it. That's where KAI Assistant comes in as the natural language layer on top, making it possible for a brand or creative team who wouldn't normally think in analytics terms to query what's actually working in their creative and why, without needing to dig through model outputs themselves.

It's one of the areas I find most genuinely interesting about KAI Analytics, and the creative diagnostics model specifically. The ability to understand not just that something worked, but exactly what it was about it that worked, and then act on that in real time during a live campaign, is a pretty different proposition from what most creative teams have had access to before.