AI vs human ad creative: 26,013 creatives

2026-09-14 · original measurements

The figures in this article are listed in our data index, with what each one was measured from and when.

Some links on this site are affiliate links. If you sign up through them we may earn a commission at no extra cost to you.

Over twelve weeks, one advertiser account ran 26,013 AI-generated creatives against 25,080 human-made ones, and on cost per conversion the two came out level. AI creative received about a seventh of the budget per creative. The edge did not hold: across four-week blocks it went from roughly 4% cheaper per conversion to roughly 6% more expensive as output scaled. The conversions it bought were also lower quality, with worse day-1 retention, fewer key actions and a higher cost per key action.

In this article, “creative” means the ad asset itself: the video that runs.

The result

One advertiser account, twelve weeks, two kinds of creative: 26,013 from batch AI generation and 25,080 from human editors. Same account, same buying system, same conversion event.

Measured on cost per conversion, they came out level.

The gap between those two share figures is 0.03 percentage points. It measures how much AI creative over- or under-delivered relative to the money it absorbed. In plain terms: AI creative took about 13% of the budget and returned about 13% of the conversions.

Cost figures below are indexed to human creative at 100. An index of 106 means 6% more expensive, and 96 means 4% cheaper.

Why a buying verdict beats a production-cost number

The number most often quoted about AI video is a production cost: a price per second of finished video, or per finished item. That number is real, and we have published our own: $0.51 for a five-shot 38.7-second ad, and about ¥1 per approved finished item. It does not tell you whether the output is worth buying, so what this article measures is the other half of that question.

An ad platform moves budget toward whatever converts. Over twelve weeks this one put 12.98% of spend into AI creative and got 12.96% of conversions back, which is as close to a market verdict as you get outside a controlled experiment.

Why the 1:7 budget gap matters

AI creative received 14.4% of the budget per creative that human creative received, about one seventh.

These are not two versions of the same product. AI creative was run as a high-volume test surface, with a small budget on each item. Human creative was run as a scaling asset, with a large budget on each item. A creative that gets a seventh of the budget produces a much noisier cost-per-conversion estimate, and averaging over a whole set does not fix that.

So this result compares portfolio against portfolio: averages over whole sets, not creative against creative. It shows the buying system would pay the same effective price for AI creative. It does not show that any individual AI creative is as good as any individual human one.

The edge reversed

Split the twelve weeks into three blocks of four, and index human creative to 100 inside each block:

In the first block, AI creative was roughly 4% cheaper per conversion. By the third it was roughly 6% more expensive. Over the same span, weekly AI output rose from about 150 creatives to about 5,000.

An aggregate index of 100 hides both halves of that movement. A headline that says AI matches human would be true of the twelve weeks in total, and wrong about the last month of them.

We do not know what caused the reversal. These are the candidates we can name but cannot yet separate:

• Creative fatigue. Reuse and recombination put the same visual ideas in front of the same audiences repeatedly. Fatigue would surface as a rising cost per conversion on roughly this timescale.
• Mix effects. The low per-creative budget on the AI side inflates the variance of its cost estimate, and the composition of the pool changed enormously as volume scaled.
• Platform and seasonal effects landing differently on the two pools.

Two of those three are properties of our own production rather than of the model: how often shots are reused, and how budget is allocated across a pool that grew thirtyfold. The workflow was itself changing while we measured it. Separating the causes needs creative-level similarity data joined to delivery results, plus a controlled test, and we have neither yet.

Where AI creative was worse

For one week in the period, the report carried downstream behavioural metrics rather than conversions alone. Against human creative in the same week:

More AI creative also received no spend at all. On a month-level cut, 72% of AI creatives recorded any spend, against 83% of human creatives.

Together, that means AI creative bought conversions at the same price and of lower quality. Parity on cost per conversion is not a craft verdict, and the downstream metrics are where the difference shows.

How does this compare with published research?

Three studies published in 2026 asked a question that sounds like ours, and they did not agree with each other. We read all three. The first thing worth saying is that they are not measuring the same thing we are, so they are context rather than corroboration. Ours is a fourth data point, and it does not slot neatly beside any of them.

Three gaps explain most of the disagreement, and none of them is a case of someone measuring it wrong.

1. They measure conversion; we measure the cost of conversion. Those are different quantities. A higher click-through rate lowers the cost of each click, and a lower conversion rate raises the cost of each conversion. The two can cancel, which leaves cost per conversion unchanged while the underlying rates move in opposite directions. That is the pattern the HKBU result describes: more attention, fewer conversions. It is arithmetically compatible with the parity we measured. We did not record click-through rate in this dataset, so we cannot confirm the mechanism. It is the first thing we would add.

2. They test different categories of advertising, and the results do not transfer between them. The Ipsos study tested brand advertising, including Cheerios, Visa and Ray-Ban, where the work is storytelling, emotion and point of view, and the outcome is modelled sales impact. This dataset is direct-response app activation, where the brief is product-driven and the outcome is a conversion event. Those are different crafts with different success criteria. A brand-advertising result, in either direction, is not evidence about direct-response creative, and ours is not evidence about brand work. We are not using the Ipsos finding to support our conclusion. We are reporting that the boundary between the two categories is one reason nobody should average these studies together.

3. “Retention” means two different things. The HKBU study’s viewing retention is measured inside the ad, as how long someone keeps watching. Our day-1 retention is measured after the install, as whether the acquired user comes back to the app. A model can win the first and lose the second. Our data covers only the second.

One asymmetry is worth stating plainly. All three studies use controlled designs, either randomized assignment or matched pairs, while ours is an observational portfolio comparison in a single account, which is weaker evidence about cause. What it adds is scale in a live direct-response account and a cost-per-conversion metric that none of the three used.

What this does not say

It does not say AI creative is as good as human creative. Portfolio parity is what a budget-allocation system produced, not a judgement about craft, and the downstream metrics above point the other way.

It does not generalise beyond this account. This is one direct-response advertiser, one product category, one platform group’s delivery system, twelve weeks. It is where we had the data, not where the question ends. Nothing here should be read as a statement about AI creative in general, and least of all about brand creative, where the studies above point elsewhere. One client establishes that the question is worth asking, and it does not establish the answer.

It is not a verdict on a model, or on the workflow around it either. The creative came from an earlier, lower-end generator, and not MiniMax H3, which is what we run now. It was not a cheaper model, either: per generated second it cost more than H3 does. Whatever cost advantage our production had did not come from the model’s price. The other variable is how the work is divided between the model and the people around it: who writes the prompts, what the review loop looks for, and how readily shots get reused inside new items. Our own numbers show that second variable moving. 28% of AI creatives never received any spend, and the cost advantage turned negative as output scaled. Both are production outcomes rather than statements about what a model can render, and both have clear room to improve.

Method

• Setup: one advertiser account; weekly and monthly aggregate reporting; batch AI-generated creative against human-edited creative over a twelve-week window.
• Model: an earlier, lower-end video generation model, unnamed here, and not MiniMax H3, which we run now. It was not cheaper per generated second than H3. The generator is one of the two variables under test; the other is the workflow around it.
• Conversion: the account’s own activation event; cost per conversion is spend divided by activations, summed over the period.
• Comparison level: portfolio against portfolio. Per-creative budgets differ by roughly 7×, so per-creative comparisons are not supported.
• Currency: none published. Costs and budgets are indexed to human creative = 100. Creative counts are exact.
• Rounding: the share figures in the results section are rounded to two decimals. The 0.03-point gap is computed from the unrounded values, 12.9836% of spend against 12.9553% of conversions, a difference of 0.0282 points. Subtracting the two displayed figures gives 0.02, so the sentence above and the table will disagree if you check them against each other. The unrounded difference is the one we report.
• Anonymity: client, product, platform and the campaign period are withheld by agreement. Ratios and indices are unchanged.
• Non-concurrent baseline, and a moving one: the AI and human creative sets were not tested over the same weeks. The human figures in the tables above come from the human test period as a whole, rather than being matched week by week to the AI output. This does not drive the headline result: the period-matched month-level cuts put AI at 0.92 and 0.96 of human cost per conversion, which is level or slightly ahead and consistent with the aggregate. It does bear on the reversal. The human baseline improved by about 19% per conversion between the first and second four-week blocks, against about 13% for AI. Part of the movement in that chart is therefore the human side getting better rather than the AI side getting worse. Which of the three causes listed in that section accounts for the remainder cannot be settled with this data.
• Not measured here: click-through rate, conversion rate per creative, and any per-creative attribution. All three would sharpen the comparison against the published studies above.

What we are working on next

Two of these are production problems rather than measurement problems, and they are the two we are actively working on.

The model. Everything in this dataset came from an earlier, lower-end generator, and as the Method section notes, it was not a cheaper one: per generated second it cost more than the model we run now. Re-running the same measurement on a newer model is the cheapest way to separate “AI creative” from “this particular model”, and it is the comparison readers will want most. That work is under way.

The split of work between the model and the people around it. The downstream quality gap, and the 28% of creatives that never received spend, are release-and-review problems as much as generation problems. Tighter prompt structure, a review loop that filters before release rather than after, and deliberate rather than opportunistic reuse would all move those numbers. This is the variable we expect to matter most, and the hardest one to measure honestly.

On the measurement side:

1. Click-through rate by creative source. It is the missing link between our parity result and the “attention without conversion” finding. If AI creative earns a higher CTR and a lower conversion rate at equal cost per conversion, the two results are one result.
2. Whether the reversal is fatigue. Reuse is the mechanism we believe creates the cost advantage, and fatigue is the mechanism we believe caps it. Testing it needs creative-similarity data joined to delivery results.
3. A controlled A/B. Two creative sets, matched content, one campaign, one budget rule. Everything above is portfolio-level, and only a controlled test supports a per-creative claim.

All of it needs to run on more than one account, because a single performance client is a starting point rather than a result. The conclusions above stay provisional until they survive a second category and a second buying setup, and we will publish what changes, including if the parity does not hold.

What stands so far is narrow. On cost per conversion, AI creative matched human creative at about a seventh of the per-creative budget, and the edge reversed as output scaled.