Figuring out what drives growth can be tricky for marketers. But that’s where incrementality testing comes in. This methodology helps businesses measure the real impact of their marketing initiatives and separates the noise from what drives actual results.
While measuring incrementality can be challenging, there is no need to worry. Darya Kislova, our marketing analyst, has put together a clear and simple guide to help you understand what incrementality is, why it matters, and how to use it properly to make smarter marketing decisions and get the most out of your marketing efforts.
Let’s begin!
Table of Contents
- What is Incrementality?
- Why Is Measuring Incrementality Important?
- What Can Be Measured with Incrementality?
- MMM vs. Incrementality Tests
- Foundations of Incrementality Measurement in Advertising
- Challenges in Measuring Incrementality
- Approaches to Measuring Incrementality
- Best Practices for Incrementality Testing
- The Bottomline
What is Incrementality?
Incrementality refers to the additional outcomes — such as conversions, revenue, or profit — directly caused by a specific marketing action, beyond what would have occurred naturally.
Some of the key metrics used to measure incrementality include:
- Incremental ROAS (Return on Ad Spend). This metric measures how much additional revenue your marketing generates relative to the amount spent. For example, if a campaign spends $5,000 and generates $25,000 in incremental sales, the incremental ROAS is 5 ($25,000 ÷ $5,000). This means that for every dollar spent, you’re getting $5 in additional sales.
Formula: Incremental ROAS = Incremental Revenue / Marketing Spend
- Incremental Profit. This metric accounts for the cost of your campaign. If the sales from a campaign bring in $30,000 but the campaign costs $7,000, then your incremental profit is $23,000.
Formula: Incremental Profit = Incremental Revenue − Campaign Cost
- Incremental Sales Lift (or Conversions Lift). The increase in sales (or conversions) that is directly attributed to a marketing initiative. For example, if a flash sale boosts conversions from 1,000 to 1,300, that is a 300-unit increase in conversions — an incremental sales lift.
Formula: Incremental Sales Lift = Test Group Sales − Control Group Sales
Why Is Measuring Incrementality Important?
- By measuring what drives new sales, you can optimize your budget and avoid spending on ineffective channels.
- Measuring incrementality takes the guesswork out of the equation and lets you make decisions based on real data instead of assumptions.
- When you know what works, you can cut down on spending that doesn’t generate new value.
- Once you have evidence of what generates new value, you can scale your marketing efforts without fear of overspending.
What Can Be Measured with Incrementality?
You can measure the incremental impact of nearly any marketing action.
Some examples include:
- Channel performance (GAds, Meta Ads, etc.)
- Strategy (retargeting, branded search, etc.)
- Specific campaigns or ads
- Spend increases or decreases
- New product launches or promotional initiatives
- Website or platform feature changes
Note: Incrementality can apply to various business decisions, but for now, we’ll focus on marketing initiatives such as ad campaigns and channel strategies.
MMM vs. Incrementality Tests
While Marketing Mix Modeling (MMM) uses historical data to estimate how each channel contributes to sales over time, incrementality tests focus on measuring the real-time effects of marketing actions.
Combining the insights from both approaches, though, can give you a better view of both long-term strategy and short-term decisions.
Foundations of Incrementality Measurement in Advertising
- Control vs. Test — Provides a baseline for measuring net-new impact.
- Statistical Rigor — Highlights the importance of proper sample sizes, test duration, and confidence levels.
- Incrementality vs. Attribution — Clarifies the difference between assigning credit across channels (attribution) and identifying conversions that wouldn’t have happened otherwise (incrementality).
Challenges in Measuring Incrementality
Below are some of the challenges that you should consider before measuring incrementality.
- Budget Constraints. Holdout groups might reduce immediate sales, which makes them harder to justify financially.
- External Factors. Things like seasonality or competitor actions can affect results.
- Complex Customer Journeys. With many touchpoints, it’s tough to pinpoint the exact impact of a single marketing action.
- Privacy and Data Limitations. New regulations can limit the data you can collect and affect your ability to measure incrementality.
Approaches to Measuring Incrementality
There are several ways to measure incrementality, depending on what you’re testing.
Our experts have identified the four most common methods and provided detailed insights along with examples for each one.
Let’s take a look.
Conversion Lift Studies
What are they? These are studies where a segment of your audience is randomly withheld from ads ( “control” group) while the rest see the ads ( “test” group). The goal is to measure the difference in conversions between the two groups.
When to use: Implement these studies when using a specific advertising platform that provides built-in lift study tools. This metric is also great for testing if a specific campaign or strategy is driving real net new conversions.
Pros: Simple to set up using platform tools and offers direct measurement of incremental conversions.
Cons: Restricted to the platform’s ecosystem and may overlook cross-channel interactions.
Example
Scenario: A brand runs a social media ad campaign, withholding ads from 10% of the audience (control) while showing them to the remaining 90% (test).
Steps:
- Identify the total audience
- Split the audience: 10% control, 90% test
- Run ads for four weeks
- Measure conversions in both groups
Results: The test group generates 20% more conversions, with 15% attributed to incremental lift.
Decision: The brand increases the campaign budget and expands targeting based on the confirmed uplift.
A/B Testing with Holdout Groups
What are they? This classic test splits your audience into two groups: one sees the new campaign (test) and the other sees the old version (control), which helps measure the impact of the new campaign.
When to use: This method works well for any marketing channel or on-site experiment where a control group can be excluded from exposure.
Pros: Simple to set up and provides a clear measurement of the net new impact from the tested variation.
Cons: Needs sufficient traffic or time to achieve statistically significant results.
Example
Scenario: An eCommerce site tests a new paid search ad creative.
Steps:
- Split search traffic equally into two groups
- Group A sees the new creative; Group B sees the original
- Track conversions and revenue for two weeks
- Compare the performance of both groups to measure the impact
Results: Group A shows 12% higher conversions, with a 10% iROAS increase from incremental revenue.
Decision: The new creative is rolled out across all search campaigns and the team tests other variations to further improve messaging.
Geo Experiments
What are they? This involves splitting geographic regions into test groups. Some receive more intense marketing while others maintain a baseline.
When to use: This metric is great for large-scale experiments in real-world settings and is commonly used when platform-level or individual-level holdouts aren’t an option.
Pros: Accurately captures market-level impact and considers external factors at a regional level.
Cons: Needs enough geographic regions for reliable results and regional variations can influence outcomes.
Example
Scenario: A brand splits the U.S. into two regions with similar historical conversion data. Group A gets a 50% increase in paid social spending while Group B keeps the same budget.
Steps:
- Split the U.S. into two geo groups with similar past performance
- Increase Group A’s paid social spend by 50% while Group B stays the same
- Track revenue, conversion rates, and order volumes by state over six weeks
- Compare results between the two groups
Results: Group A’s extra spending resulted in an iROAS of 80%, which means that the increased budget didn’t produce enough additional sales to justify the cost.
Decision: The brand reverts Group A’s budget and will analyze the negative results further (e.g., targeting, creative, seasonality) before testing again.
Matched Market Tests
What are they? They are similar to geo-experiments but you pair markets that are similar in terms of demographics and past performance to create the control and test groups.
When to use: When there are fewer markets and you want to minimize differences between the test and control regions.
Pros: Accounts for market differences by pairing locations for more precise comparisons.
Cons: Demands detailed data for accurate matching and can be challenging if markets are highly distinct.
Example
Scenario: A brand tests a new paid search strategy in two metropolitan areas with similar historical sales trends, order values, and conversion rates.
Steps:
- Choose two similar cities based on past performance and demographics
- Apply the new strategy in City A (test) and keep the current strategy in City B (control)
- Track conversion volume, revenue, and ROI in both cities for one month
- Compare the results between the cities
Results: City A sees a 15% increase in online orders while City B remains steady. 10% of the orders in City A are attributed to the incremental impact of the new strategy.
Decision: The retailer rolls out the new strategy nationwide and plans further testing for optimizations.
Best Practices for Incrementality Testing
To ensure your incrementality testing is successful, consider applying the following best practices before and throughout the process:
- Define clear hypotheses and KPIs
- Use appropriate sample sizes and test durations
- Avoid test interference by keeping experiments separate
- Adopt an iterative approach for continuous improvements
The Bottomline
Incrementality testing is vital for understanding the real impact of your marketing efforts. Whether it’s through conversion lift studies, A/B tests, geo-experiments, or matched market tests, it gives you the insights you need to fine-tune your marketing spend and see better results.
If you’re looking to dive into incrementality testing or want to supercharge other marketing strategies, book a quick 15-minute call with our experts. We’re here to help you make smarter decisions that will boost your ROI and fuel growth.
Get in touch
Got a question? We'd love to hear from you. Send us a message and we'll respond as soon as possible.
By clicking submit, you agree to our Privacy Policy
Latest Insights
Get the tips from our experts to optimize and scale your campaigns