Unlocking User Activation with Root Cause Analysis | by Jordan Gomes | Mar, 2023
A Step-by-Step guide on how to conduct a structured root cause analysis
In my previous article, I discussed how to define activation metrics for your business — those metrics that you can move in the short term and have an impact on the long term.
Building those is the very first step of a great journey toward product growth, and once you have identified those — this is when the fun begins. Your next endeavor is now to understand what are the best ways and the most effective levers to move those.
And this is where the root cause analysis can help. RCA is a structured approach to problem-solving that can provide a better understanding of which levers to use and how much they’ll contribute to your desired outcome. It will help you gain a more in-depth understanding of the problem at hand and identify its root causes.
I have a particular fondness for this type of analysis as it lies at the intersection of quantitative and qualitative methods. This makes it a highly cross-functional and comprehensive study, where utilizing qualitative data is necessary to identify potential causes and test assumptions.
To make this more digestible, we’ll be re-using the same example I was using in my previous article:
- You run a fitness app.
- You just discovered that users that upload a workout within 7d of downloading the app are far more likely to stay engaged with your app than users who don’t.
- You are now thinking about what would be the best way to move this metric.
Root Cause Analysis (RCA) is a structured approach to problem-solving that helps you identify the underlying cause or causes of a problem, rather than simply addressing its symptoms. It allows you to really get to the WHY — why is something happening the way it is happening. The output can take multiple forms — but usually can be represented as a ‘decision tree’ or ‘failure tree’, something easy to comprehend, very visual, that business stakeholders particularly appreciate.
Digressing a bit — this methodology can be applied to virtually any problem,
and in its simplest form, it doesn’t require any particular data knowledge.
If properly used, it can be a great tool to make you think about complex
systemic interactions and allow you to move away from a symptom to its cause.
It typically involves ~5 steps: defining the problem, gathering data, identifying and evaluating the different causes, identifying the root cause(s), and developing a plan of action to address them. Additionally to allowing you to understand the “WHY”, it can allow you to have rough estimates of how much you’ll be able to move the needle on your problem (and from these estimates you can build prioritization models, OKRs, etc.)
#1: Defining the Problem: The first step in RCA is to clearly define the problem you’re trying to solve. In our example (and at this stage), it is pretty straightforward: “How might we increase the number of new users who upload a workout within 7 days of signing up for our mobile fitness app in order to improve user retention rates?”
#2: Gathering the Data: The next step is to gather data about the problem. This involves gathering information about the issue, such as when it occurs / what happens / who is impacted / how they are impacted. It is important to get both a quant and qual vision of the issue and to talk with people who are either impacted and/or are subject matter experts in the topic.
#3: Identifying & Evaluating Possible Causes: That’s where the real fun begins! Once you have the data, the next step is to identify the possible causes of the problem. This involves brainstorming with some subject matter experts and talking to your users to be able to have a good understanding of all the factors that could be contributing to the issue — the very visible ones, but also the underlying causes that may be less obvious.
During this step you can use Regression analysis to get some inspiration. Regression analysis can be useful when you have 0 idea about where to look at. It can help you find features highly correlated with your issue, and from there, manually deep dive to get a better understanding of the problem.
In our example, let’s imagine we find that certain countries or certain devices are highly negatively correlated with the likelihood of becoming activated. In that case, there might be something worth looking into:
- maybe there is an issue with the translation?
- maybe there is a display bug at some point on certain devices?
- etc.
Note that as always, correlation is not causation — this method can help you
find subsets of users worth looking into, but the results shouldn’t be taken
directly (i.e. without further checking) for your root cause analysis.
Once you have identified all the different possible causes, and you started building your tree, it is important to evaluate those and understand how much they are contributing to your problem. For some of those causes, it will be easy as you will have the data, for some others, a bit more complicated, and you might have to gather some more data and/or use some imagination to produce some realistic estimates — but by having these weights, it will make things easier when you will start writing your recommendations and decide which levers to move.
#4: Iterate until you have identified the root causes: Now that you have identified the first nodes of your decision tree, you can re-iterate, all the way until you get to the actual causes. As a rule of thumb, you should try to iterate this as much as 5 times (the so-called “5 Whys” technique) — this is a good forcing function to challenge you to really think about the second/third order effect, and not just the obvious causes.
If we take our example back, this would look like something along those lines:
#5: Develop an Action plan: Now that you have a clear picture of the potential root causes, the final step is to develop a plan of action to address them. I wrote an article about “how to prioritize which data science project to work on”: the idea is similar here: you want to prioritize which lever you are going to be pulling based on different parameters, which can be (but are not limited to): the size of the opportunity, ability to action it, confidence in the outcome, time to market, etc.
Not only RCA allows you to get a better sense of what is driving the issue, but it also allows you to have a better understanding of how to fix it, and how your fix might affect the final metric.
The work you did here, can actually also be useful in setting targets / OKRs for your company: in the example above, you can use the different weights (maybe discounted using the parameters you used for prioritization, like confidence in outcome) to inform the different targets the company can set for itself.
In the end, RCA is an easy-to-use, beautiful tool, that can help your team takes better decisions. It doesn’t require a high level of technical skills but requires to be organized and to mix of quantitative and qualitative data. It can have an incredible impact — if properly done.
Hope you enjoyed reading this piece! Do you have any tips you’d want to share? Let everyone know in the comment section!
And If you want to read more of me, here are a few other articles you might like:
A Step-by-Step guide on how to conduct a structured root cause analysis
In my previous article, I discussed how to define activation metrics for your business — those metrics that you can move in the short term and have an impact on the long term.
Building those is the very first step of a great journey toward product growth, and once you have identified those — this is when the fun begins. Your next endeavor is now to understand what are the best ways and the most effective levers to move those.
And this is where the root cause analysis can help. RCA is a structured approach to problem-solving that can provide a better understanding of which levers to use and how much they’ll contribute to your desired outcome. It will help you gain a more in-depth understanding of the problem at hand and identify its root causes.
I have a particular fondness for this type of analysis as it lies at the intersection of quantitative and qualitative methods. This makes it a highly cross-functional and comprehensive study, where utilizing qualitative data is necessary to identify potential causes and test assumptions.
To make this more digestible, we’ll be re-using the same example I was using in my previous article:
- You run a fitness app.
- You just discovered that users that upload a workout within 7d of downloading the app are far more likely to stay engaged with your app than users who don’t.
- You are now thinking about what would be the best way to move this metric.
Root Cause Analysis (RCA) is a structured approach to problem-solving that helps you identify the underlying cause or causes of a problem, rather than simply addressing its symptoms. It allows you to really get to the WHY — why is something happening the way it is happening. The output can take multiple forms — but usually can be represented as a ‘decision tree’ or ‘failure tree’, something easy to comprehend, very visual, that business stakeholders particularly appreciate.
Digressing a bit — this methodology can be applied to virtually any problem,
and in its simplest form, it doesn’t require any particular data knowledge.
If properly used, it can be a great tool to make you think about complex
systemic interactions and allow you to move away from a symptom to its cause.
It typically involves ~5 steps: defining the problem, gathering data, identifying and evaluating the different causes, identifying the root cause(s), and developing a plan of action to address them. Additionally to allowing you to understand the “WHY”, it can allow you to have rough estimates of how much you’ll be able to move the needle on your problem (and from these estimates you can build prioritization models, OKRs, etc.)
#1: Defining the Problem: The first step in RCA is to clearly define the problem you’re trying to solve. In our example (and at this stage), it is pretty straightforward: “How might we increase the number of new users who upload a workout within 7 days of signing up for our mobile fitness app in order to improve user retention rates?”
#2: Gathering the Data: The next step is to gather data about the problem. This involves gathering information about the issue, such as when it occurs / what happens / who is impacted / how they are impacted. It is important to get both a quant and qual vision of the issue and to talk with people who are either impacted and/or are subject matter experts in the topic.
#3: Identifying & Evaluating Possible Causes: That’s where the real fun begins! Once you have the data, the next step is to identify the possible causes of the problem. This involves brainstorming with some subject matter experts and talking to your users to be able to have a good understanding of all the factors that could be contributing to the issue — the very visible ones, but also the underlying causes that may be less obvious.
During this step you can use Regression analysis to get some inspiration. Regression analysis can be useful when you have 0 idea about where to look at. It can help you find features highly correlated with your issue, and from there, manually deep dive to get a better understanding of the problem.
In our example, let’s imagine we find that certain countries or certain devices are highly negatively correlated with the likelihood of becoming activated. In that case, there might be something worth looking into:
- maybe there is an issue with the translation?
- maybe there is a display bug at some point on certain devices?
- etc.
Note that as always, correlation is not causation — this method can help you
find subsets of users worth looking into, but the results shouldn’t be taken
directly (i.e. without further checking) for your root cause analysis.
Once you have identified all the different possible causes, and you started building your tree, it is important to evaluate those and understand how much they are contributing to your problem. For some of those causes, it will be easy as you will have the data, for some others, a bit more complicated, and you might have to gather some more data and/or use some imagination to produce some realistic estimates — but by having these weights, it will make things easier when you will start writing your recommendations and decide which levers to move.
#4: Iterate until you have identified the root causes: Now that you have identified the first nodes of your decision tree, you can re-iterate, all the way until you get to the actual causes. As a rule of thumb, you should try to iterate this as much as 5 times (the so-called “5 Whys” technique) — this is a good forcing function to challenge you to really think about the second/third order effect, and not just the obvious causes.
If we take our example back, this would look like something along those lines:
#5: Develop an Action plan: Now that you have a clear picture of the potential root causes, the final step is to develop a plan of action to address them. I wrote an article about “how to prioritize which data science project to work on”: the idea is similar here: you want to prioritize which lever you are going to be pulling based on different parameters, which can be (but are not limited to): the size of the opportunity, ability to action it, confidence in the outcome, time to market, etc.
Not only RCA allows you to get a better sense of what is driving the issue, but it also allows you to have a better understanding of how to fix it, and how your fix might affect the final metric.
The work you did here, can actually also be useful in setting targets / OKRs for your company: in the example above, you can use the different weights (maybe discounted using the parameters you used for prioritization, like confidence in outcome) to inform the different targets the company can set for itself.
In the end, RCA is an easy-to-use, beautiful tool, that can help your team takes better decisions. It doesn’t require a high level of technical skills but requires to be organized and to mix of quantitative and qualitative data. It can have an incredible impact — if properly done.
Hope you enjoyed reading this piece! Do you have any tips you’d want to share? Let everyone know in the comment section!
And If you want to read more of me, here are a few other articles you might like:
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.