What is Marketing Data Science
What is Data Science for Marketing? I get this question all the time when I say my occupation is Marketing Data Scientist. This question comes up even more often from fellow data scientists who do not work in the marketing field.
You may not have realized it, but, likely, you have already experienced the results of marketing data science.
When you search for a product on Google, you might have noticed the top listings have an ad tag next to them. That is paid marketing and the companies have bid on the keyword that you are searching.
When you were doing Christmas shopping on Amazon, there were sponsored listings among others. These listings were also paid marketing and were recommended to you by a recommendation algorithm that determined you were likely to click through and purchase.
When you view Instagram and see a promotional coupon for a service, you might be in the treatment group of marketing experimentation to test how well Instagram ad works, and yes lucky you because the control group does not get the promotion coupon.
All of the above are enabled by marketing data science. It utilizes data science methodologies and marketing-related data to make both marketers’ and consumers’ life easier.
I’m sure so far you have got a sense of marketing data science. Keep reading and you will find 7 more detailed areas and examples to deepen your understanding of how data science helps marketing.
I list this first because I’ve worked on this for the longest time and it is essential because without measuring how well what was done, there is no way to know how to improve. Measurement is the compass.
The key to marketing measurement is to use some Northstar metric(s), for example, the ROI (return on investment) of advertisement investments (ROAS — Return on Ad Spend), to gauge the efficiency of previous marketing activities towards driving business KPIs. Based on the measured performance, the marketing team can decide to change the budget allocation, targeting strategy, bidding strategy, or other aspects to improve future efficiency.
How does data science help marketing measurement? Here are a few examples:
a) Experimentation
To spend or not to spend, that is the question
Marketing DS utilizes A/B testing and causal inference methodologies such as difference-in-difference, regression discontinuity, etc to understand whether marketing activities cause KPI changes.
Let’s use Google Search as an example. Companies spend millions of dollars buying paid searches on Google, but is it worth it? Often people search on Google with a good idea of what company to choose and ad money could be a waste if they are converting anyways. A/B testing is to the rescue. We can use Google’s Conversion Lift user-level testing platform for measurement. Google randomly split users into control and treatment groups and serves treatment ads only to the treatment group. With rigorous statistical procedures, we can know the incremental conversions driven by ads with high confidence. Although tests are rarely perfect, we now have a better educated guess with good confidence thanks to marketing data science. When A/B testing is not possible, causal inference can also help us understand what would have happened if there is no ad spend and tell us whether and how much marketing has caused the changes in KPIs.
For more comparisons of different ways of marketing measurement, please check my article here.
b) Marketing Mix Models
How much and where to spend, that is also the question
With experimentations we have a good understanding of the efficiency of campaigns and channels, what about budget allocation? Where do you spend ad money and how much?
Imagine yourself as a marketing manager with a 100 million dollar marketing budget, where and how much do you spend them? You can do 50 million on TV because all your families and friends watch TV. You can put the rest on Instagram because Instagram seems to always recommend relevant ads to you. This could be a great approach based on personal experience, but there are more scientific ways to do this — Marketing Mix Models (MMM) to the rescue.
MMM uses time series techniques to attribute business KPIs to different drivers (such as advertising activities, promotions, etc) while controlling for other contributing factors (such as seasonality, trend, economy, competitor activities, etc). The model can tell you the efficiency and contribution of the KPI of each marketing channel at the current level of spending. It can also simulate if the spend level changes, what the KPI could be with everything else controlled. Given future budget levels and constraints, the model can forecast KPIs and inform the optimization of marketing spend allocation on different channels to reach maximum KPIs or ROAS.
For a more in-depth understanding of MMM, please check my articles: Marketing Mix Modeling 101, Marketing Mix Modeling 102
One of the projects I worked on at Doordash is a bidding optimization and automation platform that optimizes bidding amounts and updates the bidding automatically to 1) maximize interested KPIs and 2) save time and cost of labor.
Many channels, especially search, employ ads auction when determining what ads to show when people search on Google or other websites that show ads. It is like real-life auctions but also considers other factors in addition to the highest bidding.
In real-life auctions, when an object is put up, it may or may not be as valuable as the bidding price. You don’t want to bid too much and overpay, or bid too little and miss opportunities. Ad actions are also like that, a search may or may not lead to a conversion. You don’t want to waste money on low-quality and low-relevancy searches, but you also don’t want to lose potential customers to your competitors. What’s more complicated for digital ad auctions is that they can happen anytime and happen very fast. It is hard and not scientific for humans to determine reasonable bidding amounts on certain types of products or keywords and when to update the bid amount.
Marketing data science comes to the rescue. Marketing DS can use historical marketing data, seasonality, trend, and conversion data to establish a relationship between the bid levers and conversions. Using optimization algorithms and business constraints, we can know what would be ideal bidding amounts and timing to maximize conversions (or other business KPIs). This takes the heavy lifting and guesswork away from setting bids to meet business goals.
Customer lifetime value (CLV or LTV) is the total monetary value that a customer generates over the entire lifetime of being a customer with a company.
Use me as an oversimplified example, I found a local restaurant and continued to go there for 6 months and then I grew tired of their food. My lifetime with this restaurant is 6 months and my total CLV is $210 (summing up all my spending for the 6 months. (This can be calculated in other ways, e.g. adding my referral value, subtracting cost, etc. It is up to your definition and business use case.)
Why is CLV important?
It takes into account not just the initial purchase, but also their future repeated purchases, and even potential referrals, etc. It helps businesses make more informed decisions about strategies for both marketing and business operations.
For marketing, by knowing the value of a customer over time, businesses will be in a better position in determining how much they should invest in acquiring new customers, retaining existing customers, and what types of marketing and promotional activities are likely to generate the best returns. Imagine the local restaurant is also targeting another person Tommy. The cost of acquiring him and acquiring someone like me is the same, but the restaurant has a limited budget. Based on historical data, people like Tommy is unlikely to come back to the restaurant after the first month and in total, he will only generate 50 dollars in revenue, as opposed to 210 dollars. The restaurant might decide to use that budget on someone like me instead of Tommy.
There are many ways of calculating and predicting CLV. A common practice is to
- define a starting point — when do you start predicting CLV, upon signing up, or making the first purchase, etc?
- define a time frame — how long is a typical customer’s lifetime, it could be a month, a year, a few years, etc
- define prediction frequency — predict by day or week or month, etc
- define how to calculate CLV for your business — total revenue, or revenue minus certain cost, or revenue plus referrals from this customer
- define drivers for CLV and acquire the data— factors that impact your CLV.
- Choose the right model, predict, evaluate, validate, iterate, and productionalize — the industry loves Gradient Boosting models such as XGB and GBM for CLV prediction because these models are more versatile, require no assumptions of the model form, and more accurate than other types of models.
Alternatively, you can also make simple predictions just based on historical average CLV if you are only looking for a quick and dirty solution.
Churning means a customer stops using the services or purchasing the products from a company. Any successful business does not want to lose its valuable customers, as that can lead to the loss of revenue and future growth opportunities. Sometimes churning is inevitable, however many times it can be prevented or pushed back by using retention marketing strategies or customer relationship management (CRM) strategies. The business can proactively reach out to customers who are about to churn to gauge what additional support can be provided, determine whether to run a targeted retention or re-engagement marketing campaign and so on.
How do you know who is at high risk of churning? Some customers might not be vocal about it and just secretly leave. This is where churn prediction models come in handy: predicting churn to prevent churn.
A typical churn prediction model predicts the likelihood of someone churning in a time frame. The modeling steps can be similar to CLV modeling, however, the target is not a monetary value but a binary value of leaving or not leaving. The prediction will give us the likelihood of churning, which will be between 0 to 1. The features of the model can be historical data relevant to customer churn behaviors. Take the local restaurant as an example, last month’s order value might be a good feature to predict whether the customer will come back in the next month. Churn prediction can also be used in combination with CLV to calculate the expected future values of customers.
Models as simple as logistic regression, more complex models like random forest or boosting models, or even neural networks can be used for churn predictions. The model selection and model structure should be based on data availability, sample balance, model explainability, and business requirements.
Recommendation algorithms are so widely used by many businesses. Amazon uses it to recommend relevant products to buyers, Netflix uses it to recommend relevant TV shows and movies to watchers, and UberEats uses it to recommend restaurants and dishes to eaters. A recommender gets better and more powerful over time when it has learned about the user’s behaviors, likes, and dislikes.
Recommendation systems can be both content-based or collaborative-filtering based. As the name suggests, content-based recommendation is based on similarities between the content of products. For example, if I have purchased Harry Potter books, I might also be recommended to buy The Chronicles of Narnia. Collaborative filtering is more based on the similarities between user attributes and behaviors, and/or item attributes. I will be recommended to purchase something because someone similar has made the purchase.
Recommending products or services itself already serves as part of marketing. There is also ads recommendation system. If you take Uber rides as often as I do, you will notice that now the Uber app shows ads for other businesses. One time I took an Uber to the gym, and I was shown an ad for sports clothing for women, and the other time on my way to the movie theater, I was shown an upcoming movie preview for the next month.
Sentiment analysis is believed to be one of the most effective ways to measure marketing, especially brand marketing campaigns to analyze customers’ sentiment around your brand. Sentiment analysis identifies feelings and emotions expressed in words, which helps businesses to understand their strengths and weaknesses and spot future growth and improvement opportunities.
Marketing DS mine opinions and text data to extract information. It falls into the board category of text classification. Classifiers like Naive Bayesian can help with the job and it is also a hot topic for NLP (natural language processing). The text classifier helps tell whether the sentiment behind certain texts is positive, negative, or neutral.
For example, if I have three sentences from a survey on an imaginary company:
- Imaginary Service Co. provides awesome services.
- Imaginary Service Co. sucks.
- Imaginary Service Co. provides services.
The model should tell me that the sentiment is positive, negative, or neutral with a decent accuracy score.
I haven’t personally built sentiment analysis models, but I found this article informative for a starter.
Last but not least, data science can also help segment customers into different sub-groups based on shared features. Why do we need to segment the customers? From a marketing perspective, this can help marketers to target each meaningful segmentation differently and develop more effective and efficient marketing strategies.
For example, customers have different needs and preferences for the products. Some customers are more price-sensitive and others make purchases regardless, some customers prefer luxury goods and some care more about affordability more frugal. When we want to run a discount promotion campaign of 20% off to encourage more purchases, we can target customers that are more price sensitive since customers who have more rigid demand is likely to purchase without the discount. When a new high-end product line launches, we can target customers with a more luxurious lifestyle to increase conversion rate and ROI.
Marketing Data Science helps us segment customers using machine learning algorithms and data. The nature of the problem makes it an unsupervised machine learning problem (typically) that can be solved with clustering algorithms like K-means clustering. Take e-commerce as an example, features like customers’ past purchasing habits, demographic info, spending patterns, income, etc could be good features to use.
To sum up, in this story, I have written about 7 uses of Marketing Data Science that help businesses improve marketing efficiency and in return achieve higher growth:
- Marketing measurement and budget allocation — to measure the efficiency of marketing activities in the past and point out directions for future marketing strategy.
- Bidding optimization and automation — optimize ads bidding strategy and automate the bidding process to help increase conversions and reduce cost.
- Customer Lifetime Value prediction — to account for long-term customer value and optimize for long-term success.
- Churn prediction — to predict valuable customers at high risk of churning so that companies could focus resources to prevent it from happening.
- Recommendation systems — to recommend relevant ads to customers and increase conversion rate and ROI.
- Sentiment analysis — to analyze customer sentiment behind texts for better brand marketing strategies.
- Customer segmentation — to segment customers into meaning for segmentations for more accurate targeting.
Thank you for reading so far and congratulations you have a deeper understanding of Marketing Data Science.
I hope you have enjoyed my article and it has helped you in some way. I write articles about data science, business, working experiences, and many other topics. Follow me for more and subscribe to my email if you are interested to read more content like this and get free useful resources!
What is Marketing Data Science
What is Data Science for Marketing? I get this question all the time when I say my occupation is Marketing Data Scientist. This question comes up even more often from fellow data scientists who do not work in the marketing field.
You may not have realized it, but, likely, you have already experienced the results of marketing data science.
When you search for a product on Google, you might have noticed the top listings have an ad tag next to them. That is paid marketing and the companies have bid on the keyword that you are searching.
When you were doing Christmas shopping on Amazon, there were sponsored listings among others. These listings were also paid marketing and were recommended to you by a recommendation algorithm that determined you were likely to click through and purchase.
When you view Instagram and see a promotional coupon for a service, you might be in the treatment group of marketing experimentation to test how well Instagram ad works, and yes lucky you because the control group does not get the promotion coupon.
All of the above are enabled by marketing data science. It utilizes data science methodologies and marketing-related data to make both marketers’ and consumers’ life easier.
I’m sure so far you have got a sense of marketing data science. Keep reading and you will find 7 more detailed areas and examples to deepen your understanding of how data science helps marketing.
I list this first because I’ve worked on this for the longest time and it is essential because without measuring how well what was done, there is no way to know how to improve. Measurement is the compass.
The key to marketing measurement is to use some Northstar metric(s), for example, the ROI (return on investment) of advertisement investments (ROAS — Return on Ad Spend), to gauge the efficiency of previous marketing activities towards driving business KPIs. Based on the measured performance, the marketing team can decide to change the budget allocation, targeting strategy, bidding strategy, or other aspects to improve future efficiency.
How does data science help marketing measurement? Here are a few examples:
a) Experimentation
To spend or not to spend, that is the question
Marketing DS utilizes A/B testing and causal inference methodologies such as difference-in-difference, regression discontinuity, etc to understand whether marketing activities cause KPI changes.
Let’s use Google Search as an example. Companies spend millions of dollars buying paid searches on Google, but is it worth it? Often people search on Google with a good idea of what company to choose and ad money could be a waste if they are converting anyways. A/B testing is to the rescue. We can use Google’s Conversion Lift user-level testing platform for measurement. Google randomly split users into control and treatment groups and serves treatment ads only to the treatment group. With rigorous statistical procedures, we can know the incremental conversions driven by ads with high confidence. Although tests are rarely perfect, we now have a better educated guess with good confidence thanks to marketing data science. When A/B testing is not possible, causal inference can also help us understand what would have happened if there is no ad spend and tell us whether and how much marketing has caused the changes in KPIs.
For more comparisons of different ways of marketing measurement, please check my article here.
b) Marketing Mix Models
How much and where to spend, that is also the question
With experimentations we have a good understanding of the efficiency of campaigns and channels, what about budget allocation? Where do you spend ad money and how much?
Imagine yourself as a marketing manager with a 100 million dollar marketing budget, where and how much do you spend them? You can do 50 million on TV because all your families and friends watch TV. You can put the rest on Instagram because Instagram seems to always recommend relevant ads to you. This could be a great approach based on personal experience, but there are more scientific ways to do this — Marketing Mix Models (MMM) to the rescue.
MMM uses time series techniques to attribute business KPIs to different drivers (such as advertising activities, promotions, etc) while controlling for other contributing factors (such as seasonality, trend, economy, competitor activities, etc). The model can tell you the efficiency and contribution of the KPI of each marketing channel at the current level of spending. It can also simulate if the spend level changes, what the KPI could be with everything else controlled. Given future budget levels and constraints, the model can forecast KPIs and inform the optimization of marketing spend allocation on different channels to reach maximum KPIs or ROAS.
For a more in-depth understanding of MMM, please check my articles: Marketing Mix Modeling 101, Marketing Mix Modeling 102
One of the projects I worked on at Doordash is a bidding optimization and automation platform that optimizes bidding amounts and updates the bidding automatically to 1) maximize interested KPIs and 2) save time and cost of labor.
Many channels, especially search, employ ads auction when determining what ads to show when people search on Google or other websites that show ads. It is like real-life auctions but also considers other factors in addition to the highest bidding.
In real-life auctions, when an object is put up, it may or may not be as valuable as the bidding price. You don’t want to bid too much and overpay, or bid too little and miss opportunities. Ad actions are also like that, a search may or may not lead to a conversion. You don’t want to waste money on low-quality and low-relevancy searches, but you also don’t want to lose potential customers to your competitors. What’s more complicated for digital ad auctions is that they can happen anytime and happen very fast. It is hard and not scientific for humans to determine reasonable bidding amounts on certain types of products or keywords and when to update the bid amount.
Marketing data science comes to the rescue. Marketing DS can use historical marketing data, seasonality, trend, and conversion data to establish a relationship between the bid levers and conversions. Using optimization algorithms and business constraints, we can know what would be ideal bidding amounts and timing to maximize conversions (or other business KPIs). This takes the heavy lifting and guesswork away from setting bids to meet business goals.
Customer lifetime value (CLV or LTV) is the total monetary value that a customer generates over the entire lifetime of being a customer with a company.
Use me as an oversimplified example, I found a local restaurant and continued to go there for 6 months and then I grew tired of their food. My lifetime with this restaurant is 6 months and my total CLV is $210 (summing up all my spending for the 6 months. (This can be calculated in other ways, e.g. adding my referral value, subtracting cost, etc. It is up to your definition and business use case.)
Why is CLV important?
It takes into account not just the initial purchase, but also their future repeated purchases, and even potential referrals, etc. It helps businesses make more informed decisions about strategies for both marketing and business operations.
For marketing, by knowing the value of a customer over time, businesses will be in a better position in determining how much they should invest in acquiring new customers, retaining existing customers, and what types of marketing and promotional activities are likely to generate the best returns. Imagine the local restaurant is also targeting another person Tommy. The cost of acquiring him and acquiring someone like me is the same, but the restaurant has a limited budget. Based on historical data, people like Tommy is unlikely to come back to the restaurant after the first month and in total, he will only generate 50 dollars in revenue, as opposed to 210 dollars. The restaurant might decide to use that budget on someone like me instead of Tommy.
There are many ways of calculating and predicting CLV. A common practice is to
- define a starting point — when do you start predicting CLV, upon signing up, or making the first purchase, etc?
- define a time frame — how long is a typical customer’s lifetime, it could be a month, a year, a few years, etc
- define prediction frequency — predict by day or week or month, etc
- define how to calculate CLV for your business — total revenue, or revenue minus certain cost, or revenue plus referrals from this customer
- define drivers for CLV and acquire the data— factors that impact your CLV.
- Choose the right model, predict, evaluate, validate, iterate, and productionalize — the industry loves Gradient Boosting models such as XGB and GBM for CLV prediction because these models are more versatile, require no assumptions of the model form, and more accurate than other types of models.
Alternatively, you can also make simple predictions just based on historical average CLV if you are only looking for a quick and dirty solution.
Churning means a customer stops using the services or purchasing the products from a company. Any successful business does not want to lose its valuable customers, as that can lead to the loss of revenue and future growth opportunities. Sometimes churning is inevitable, however many times it can be prevented or pushed back by using retention marketing strategies or customer relationship management (CRM) strategies. The business can proactively reach out to customers who are about to churn to gauge what additional support can be provided, determine whether to run a targeted retention or re-engagement marketing campaign and so on.
How do you know who is at high risk of churning? Some customers might not be vocal about it and just secretly leave. This is where churn prediction models come in handy: predicting churn to prevent churn.
A typical churn prediction model predicts the likelihood of someone churning in a time frame. The modeling steps can be similar to CLV modeling, however, the target is not a monetary value but a binary value of leaving or not leaving. The prediction will give us the likelihood of churning, which will be between 0 to 1. The features of the model can be historical data relevant to customer churn behaviors. Take the local restaurant as an example, last month’s order value might be a good feature to predict whether the customer will come back in the next month. Churn prediction can also be used in combination with CLV to calculate the expected future values of customers.
Models as simple as logistic regression, more complex models like random forest or boosting models, or even neural networks can be used for churn predictions. The model selection and model structure should be based on data availability, sample balance, model explainability, and business requirements.
Recommendation algorithms are so widely used by many businesses. Amazon uses it to recommend relevant products to buyers, Netflix uses it to recommend relevant TV shows and movies to watchers, and UberEats uses it to recommend restaurants and dishes to eaters. A recommender gets better and more powerful over time when it has learned about the user’s behaviors, likes, and dislikes.
Recommendation systems can be both content-based or collaborative-filtering based. As the name suggests, content-based recommendation is based on similarities between the content of products. For example, if I have purchased Harry Potter books, I might also be recommended to buy The Chronicles of Narnia. Collaborative filtering is more based on the similarities between user attributes and behaviors, and/or item attributes. I will be recommended to purchase something because someone similar has made the purchase.
Recommending products or services itself already serves as part of marketing. There is also ads recommendation system. If you take Uber rides as often as I do, you will notice that now the Uber app shows ads for other businesses. One time I took an Uber to the gym, and I was shown an ad for sports clothing for women, and the other time on my way to the movie theater, I was shown an upcoming movie preview for the next month.
Sentiment analysis is believed to be one of the most effective ways to measure marketing, especially brand marketing campaigns to analyze customers’ sentiment around your brand. Sentiment analysis identifies feelings and emotions expressed in words, which helps businesses to understand their strengths and weaknesses and spot future growth and improvement opportunities.
Marketing DS mine opinions and text data to extract information. It falls into the board category of text classification. Classifiers like Naive Bayesian can help with the job and it is also a hot topic for NLP (natural language processing). The text classifier helps tell whether the sentiment behind certain texts is positive, negative, or neutral.
For example, if I have three sentences from a survey on an imaginary company:
- Imaginary Service Co. provides awesome services.
- Imaginary Service Co. sucks.
- Imaginary Service Co. provides services.
The model should tell me that the sentiment is positive, negative, or neutral with a decent accuracy score.
I haven’t personally built sentiment analysis models, but I found this article informative for a starter.
Last but not least, data science can also help segment customers into different sub-groups based on shared features. Why do we need to segment the customers? From a marketing perspective, this can help marketers to target each meaningful segmentation differently and develop more effective and efficient marketing strategies.
For example, customers have different needs and preferences for the products. Some customers are more price-sensitive and others make purchases regardless, some customers prefer luxury goods and some care more about affordability more frugal. When we want to run a discount promotion campaign of 20% off to encourage more purchases, we can target customers that are more price sensitive since customers who have more rigid demand is likely to purchase without the discount. When a new high-end product line launches, we can target customers with a more luxurious lifestyle to increase conversion rate and ROI.
Marketing Data Science helps us segment customers using machine learning algorithms and data. The nature of the problem makes it an unsupervised machine learning problem (typically) that can be solved with clustering algorithms like K-means clustering. Take e-commerce as an example, features like customers’ past purchasing habits, demographic info, spending patterns, income, etc could be good features to use.
To sum up, in this story, I have written about 7 uses of Marketing Data Science that help businesses improve marketing efficiency and in return achieve higher growth:
- Marketing measurement and budget allocation — to measure the efficiency of marketing activities in the past and point out directions for future marketing strategy.
- Bidding optimization and automation — optimize ads bidding strategy and automate the bidding process to help increase conversions and reduce cost.
- Customer Lifetime Value prediction — to account for long-term customer value and optimize for long-term success.
- Churn prediction — to predict valuable customers at high risk of churning so that companies could focus resources to prevent it from happening.
- Recommendation systems — to recommend relevant ads to customers and increase conversion rate and ROI.
- Sentiment analysis — to analyze customer sentiment behind texts for better brand marketing strategies.
- Customer segmentation — to segment customers into meaning for segmentations for more accurate targeting.
Thank you for reading so far and congratulations you have a deeper understanding of Marketing Data Science.
I hope you have enjoyed my article and it has helped you in some way. I write articles about data science, business, working experiences, and many other topics. Follow me for more and subscribe to my email if you are interested to read more content like this and get free useful resources!