Media Mix Modeling: Technical Guideline to Overcome Pitfalls | by Hajime Takeda | Mar, 2023

By Jessie Hobb On Mar 31, 2023

How to Understand Outputs and Maximize its Practical Application

TLDR: Media Mix Modeling (MMM) is a statistical approach for marketing measurement, but it’s not a “One size fits all” solution like other techniques. I will cover essential checkpoints for data scientists to enhance their modeling techniques and achieve improved results, differences between advertising reports and MMM, as well as the distinctions between Multi-Touch Attribution (MTA) and MMM.

Introduction

MMM is a statistical method to understand & measure Return on Investment (ROI) and optimize their media budget.

Data scientists at companies with huge advertising budgets in the beverage, consumer goods, auto, and fashion industries have been working on improving MMM. Also, ad tech companies, such as Google and Meta, have been focusing on MMM actively these days since privacy regulations such as GDPR and Apple’s IDFA deprecation have been impacting the tracking accuracy of the digital world.

For more information, please refer to this article I posted previously.

After publishing the above article, I received numerous positive feedback and some questions from Towards Data Science readers. Some common questions included:

(1) What are the checkpoints for a successful MMM project?
(2) How do advertising reports and MMM differ?
(3) What are the differences between Multi-Touch Attribution(MTA) and MMM?

In this article, I will address these questions and provide insights to help you better understand these concepts.

(1) What are the checkpoints for a successful MMM project?

To ensure the reliability and accuracy of MMM, data scientists should consider the following checkpoints before trusting the results: Some of the ideas are inspired by Google’s research paper: Chan, D., & Perry, M. (2017). Challenges and Opportunities in Media Mix Modeling.

✔️ Sufficient data points:

Ensure that there are enough data points available to model the ad channels and their relationships.
Typically, MMMs often require a minimum of two years of weekly-level data. However, if you don’t have that much data, daily data is also acceptable, but in that case, you will need to be more careful in reviewing the outliers.

2. ✔️ Select appropriate input variables:

Variability in input data is crucial. For example, if TV spending remains unchanged during the entire time period, the model may struggle to determine its influence on sales. In such instances, it could be more beneficial to exclude TV from the input variables.
Keep factors that affect sales. For instance, if the expenditure on magazine X is significantly smaller compared to other ads and only temporary, it might be more appropriate to exclude it.
Input variables can be selected during both the pre-modeling stage and the model selection stage.

3. ✔️ Address correlated input variables:

Check for potential multicollinearity between input variables, which can lead to high-variance coefficient estimates and poor attribution of sales to specific ad channels.
If you constantly allocate the same budget for the Meta and Tiktok ads, the model’s reliability may be compromised, making it difficult to assess the impact of these media channels.
Use methods to address correlated variables, such as regularization techniques or variable selection methods.

4. ✔️ Control for selection bias:

Be mindful of potential selection bias issues, such as seasonality.
For instance, increased sales in November may not be a result of advertising but rather due to consumer demand or Black Friday sales.
In my experience, internal promotional data or pricing data can serve as valuable control factors for unobservable demand variables.

5. ✔️ Validate extrapolation assumptions:

Extrapolation assumptions involve making predictions or drawing conclusions about a model’s behavior beyond the range of available data.
Please be careful when trying to answer business questions such as “What if I multiply my ad spend to X by tenfold” or “What happens if I suddenly stop spending on ads X.”
Test the model’s performance on different scenarios, and be cautious when interpreting results that require significant extrapolation.

6. ✔️ Test model performance:

Validate the MMM by comparing its predictions to actual sales and marketing data.
R-squared and MAPE, mean absolute percentage error, are well-known measures for MMM. Generally speaking, R2 is considered good if it is more than 0.8. Also, for MAPE, the goal is for it to be 20% or below.

7. ✔️ Conducting randomized experiments

When using MMM to estimate the impact of changes in the media budget, it’s advisable to conduct randomized experiments to validate the findings.
For instance, by dividing regions into test and control groups and adjusting ad spend, you can measure the uplift and obtain more accurate results.

By addressing these checkpoints, data scientists can increase their confidence in the validity and reliability of MMM results and make more informed decisions based on the insights provided by the model.

(2) How do advertising reports and MMM differ? 🤔

To get straight to the point, MMM provides a broader perspective on marketing performance, helping businesses to allocate resources and optimize their overall marketing mix from a long-term perspective. In contrast, advertising reports from platforms like Google/Meta Ads give more granular insights into specific ads or campaigns, allowing for adjustments to improve short-term performance.

Features of Media Mix Modeling (MMM):

MMM focuses on helping businesses optimize their overall marketing mix and resource allocation.
It takes into account the interplay between different media channels, both online and offline.
Can be used even when the conversion point is offline.

Features of Advertising Reports:

Advertising reports offer more detailed insights into the performance of specific ads or campaigns such as click-through rates (CTR) and cost per click (CPC).
These reports primarily focus on short-term performance, helping advertisers quickly grasp the progress of individual campaigns.
Actual customers have multiple media touchpoints. Intertwined effects of multiple media cannot be considered in each media’s advertising report.

Image by Author: Advertising reports on Google Ads

(3) What are the differences between Multi-Touch Attribution(MTA) and MMM? 🤔

Multi-Touch Attribution (MTA) is another approach to marketing measurement that aims to assign credit to different marketing touchpoints along the customer journey using their access logs.

Each method has its pros and cons. Unlike MMM, which examines the aggregated cost of each marketing channel, MTA offers a more granular and reliable perspective on the contributions of individual touchpoints.

Another difference is the tracking of non-digital customer behavior. MTA focus on digital touchpoints and digital conversion, while MMM takes into account the costs of offline media spending and the results of offline conversions.

Image by Author: MTA does not track offline activities

MMM is more suitable when

A brand invests in offline media such as TV and magazines and wants to assess its impact.
A brand sells items through both offline and online channels/ A brand primarily sells items offline
A financial perspective is of greater importance.
A brand possesses historical advertising and sales data from the past 2–3 years.

MTA, on the other hand, is more suitable when

A brand that spends a lot on digital media such as Google ads and Meta ads.
A brand mainly sells items online.
Understanding the customer journey is prioritized.
Real-time assessment is needed.

Lastly

Thank you for reading! If you have any questions/suggestions, feel free to contact me on Linkedin! Also, I would be happy if you follow me on Towards Data Science.

Reference