“Am I just paying for traffic I would have gotten anyway?”
“Is this new campaign actually driving new results?”
“What happens if we just…turn it off?”
Welcome to the chaos and clarity of incrementality testing.
For marketers focused on performance, incrementality is more than a metric. It’s the ultimate measure of what’s actually working. Done right, these tests separate the tactics that drive new value from the ones just soaking up credit.
Why Incrementality Matters More Than Ever
In a world of smart bidding, multi-touch attribution, and siloed dashboards, it’s easy to confuse performance with impact. But here's the uncomfortable truth:
A campaign can look efficient and still not drive any net new results.
That’s where incrementality comes in. By designing clean, structured tests and analyzing changes in total conversions, you can finally answer the question: "Is this channel or tactic actually adding value—or just taking credit?"
How to Run an Incrementality Test
1. Define Your Test Period & Structure
Not all incrementality tests are created equal. Pick the structure that fits your business:
Pro Tip: Use a geographic split if possible. It isolates impact more cleanly and avoids confounding variables.
2. Use Total Conversions (Not Just Paid Platform Data)
The key to incrementality is measuring what wouldn’t have happened without this campaign. That means:
- Pulling conversions from your source of truth (GA4, Adobe, CRM)
- Looking at total performance across all channels
- Comparing test vs. control over time
If your test looks bad in Google Ads, but total conversions stay flat, the campaign wasn't incremental.
3. Analyze the Funnel (Including CRM)
Don’t stop at form fills or MQLs. Look deeper:
- Lead quality (SQLs, revenue)
- Retention & LTV
- Pipeline velocity
If pausing a campaign hurts your long-term value? It was probably driving more than you thought.
4. Watch Competitor Behavior
When you pull back on spend, keep your eye on:
- Branded search impression share
- Direct traffic drops
- Competitor ad activity
We’ve seen slow erosion over months, where paid traffic loss isn’t covered by organic—and the damage accumulates.
The risk isn’t always immediate. Sometimes, it takes months to realize you gave your funnel away.
5. Phrase Your Hypothesis Clearly
This helps frame the outcome and align expectations. For example:
"If we pause branded search in CA, organic traffic will make up the difference with no drop in total conversions."
Then, test that hypothesis. And when the answer surprises you? That’s the point.
Why We Built a GPT for This
We initially started building a Looker Studio dashboard. But with ChatGPT-4o’s new image-generation capabilities, we realized we could do more—and faster.
So we built a custom GPT to:
- Generate clean incrementality charts
- Calculate % lift, CPA, and ROI
- Summarize performance and trends
- Provide a visual snapshot you can share with clients or teams
You upload a CSV with your test data. It does the math.
Try it here: Incrementality Test Assistant GPT
What It Works Best For
- Geo-based holdout tests
- Branded search pullbacks
- Paid social or CTV awareness tests
- Upper funnel channel validation
Not meant for:
- Multi-year MMM
- Granular keyword attribution
What You’ll Need to Run a Clean Test
Required:
- 30+ days of pre-test data (for expected conversion modeling)
- Test start and end dates
- Total conversion counts
Optional (but helpful):
- Spend and CPA
- CRM impact data
- Brand search volume trends
Final Thought: Test. Learn. Prove It.
A good incrementality test gives you the confidence to:
- Justify budgets
- Optimize smarter
- Scale winning channels
- Defend your spend with data-backed results
And when you can visualize and share that lift clearly? That’s power.
Did you test it out? Jump in the conversation and share the insights you discovered.
