Why shouldn’t the focus of a project be on using complex techniques? In my opinion there are three main reasons, which I will explain here.
Reason 1. The business doesn’t care
The first and most important reason is that the business doesn’t care! Your stakeholders are not interested in the technical details of your model. Whether you used boosted trees or a neural network, to them, it’s all the same. What they want to know is how your model is helping them achieve their business goals. If the model needs to be retrained often, you can justify your decision to use a simple model like logistic regression over a neural network because it’s super fast to train.
Often, the main goal of a machine learning model is not to reach 100% accuracy. Instead, a machine learning model helps with business processes. Spending too much time optimizing the model will delay the time it takes to deliver a working product to the market. It’s better to create an MVP, ensure it meets the business requirements, and get it into production. It’s essential to take not only performance but also interpretability, computation speed, development costs, robustness, and training time into account. These factors are important too and can be as relevant to business people as performance.
Besides yourself, there are other people who care about a complex model and state-of-the-art methods. Those people are often researchers or data science colleagues. If you work too closely with them instead of with the business, you can get to the point where you believe modeling is the main goal. To overcome this, try to work closer with business people. Demo your product after every new feature implementation and ask the business if your assumptions are correct. Decisions that seem small can be really important for business people.
Reason 2. A complex model adds less value than a working MVP
The more time you spend on the model, the less time you have for good engineering principles, such as writing modular code, testing, architecture, logging, and monitoring. Setting these things up in a good way at the beginning saves a lot of time later. You can easily add new features to a solid codebase. This is more valuable than having a complex model in a Jupyter Notebook that performs slightly better but doesn’t run in production. Another benefit of a simple model is interpretability, which can help convince stakeholders because they can see the predictions make sense.
Especially in the beginning, focus on creating a product that works and has robust code and a well-crafted CI/CD pipeline. This makes it easier to improve the solution later on. If the business doesn’t feel the urge to improve the current solution, you can move on to another project. You didn’t waste your time creating a ‘perfect’ model.
What relates to this is the Pareto principle. It is a rule that states that 80% of results can be achieved through 20% of our efforts (aka the 80/20 rule). Often, creating a complex model that performs slightly better than a simple model doesn’t fall into the 80% of the results but is a task that is hard and takes a lot of time. The complex model is that last hard-to-reach 20% that takes 80% of the effort. Before you start, convince yourself it is worth it.
Reason 3. Complex projects require more maintenance
The more complex the project, the more resources and time are needed to maintain it. This means that you’ll spend more time fixing bugs, optimizing the model, keeping the data up to date, and less time adding new features or improving the product. A simple project, on the other hand, requires less maintenance, which means that you can spend more time iterating on the MVP and adding new features to improve the product.
An important thought to keep in mind is that the best solution is often the simplest solution that fits the requirements. This can help you determine if that deep learning state-of-the-art model is truly worth the extra work that comes with it! If there are two models that perform equally well, and one is simple and the other is complex, go with the simple one.
One example from my work at a company: I tried to solve a scheduling problem with reinforcement learning. It was quite complex, and we were progressing slowly. The business became a bit annoyed and disappointed because we couldn’t show good results. When we switched our solution method to (good old) mathematical optimization, it went much faster! It was less interesting, but we gained the trust of the business and could implement new features and constraints easily.
Why shouldn’t the focus of a project be on using complex techniques? In my opinion there are three main reasons, which I will explain here.
Reason 1. The business doesn’t care
The first and most important reason is that the business doesn’t care! Your stakeholders are not interested in the technical details of your model. Whether you used boosted trees or a neural network, to them, it’s all the same. What they want to know is how your model is helping them achieve their business goals. If the model needs to be retrained often, you can justify your decision to use a simple model like logistic regression over a neural network because it’s super fast to train.
Often, the main goal of a machine learning model is not to reach 100% accuracy. Instead, a machine learning model helps with business processes. Spending too much time optimizing the model will delay the time it takes to deliver a working product to the market. It’s better to create an MVP, ensure it meets the business requirements, and get it into production. It’s essential to take not only performance but also interpretability, computation speed, development costs, robustness, and training time into account. These factors are important too and can be as relevant to business people as performance.
Besides yourself, there are other people who care about a complex model and state-of-the-art methods. Those people are often researchers or data science colleagues. If you work too closely with them instead of with the business, you can get to the point where you believe modeling is the main goal. To overcome this, try to work closer with business people. Demo your product after every new feature implementation and ask the business if your assumptions are correct. Decisions that seem small can be really important for business people.
Reason 2. A complex model adds less value than a working MVP
The more time you spend on the model, the less time you have for good engineering principles, such as writing modular code, testing, architecture, logging, and monitoring. Setting these things up in a good way at the beginning saves a lot of time later. You can easily add new features to a solid codebase. This is more valuable than having a complex model in a Jupyter Notebook that performs slightly better but doesn’t run in production. Another benefit of a simple model is interpretability, which can help convince stakeholders because they can see the predictions make sense.
Especially in the beginning, focus on creating a product that works and has robust code and a well-crafted CI/CD pipeline. This makes it easier to improve the solution later on. If the business doesn’t feel the urge to improve the current solution, you can move on to another project. You didn’t waste your time creating a ‘perfect’ model.
What relates to this is the Pareto principle. It is a rule that states that 80% of results can be achieved through 20% of our efforts (aka the 80/20 rule). Often, creating a complex model that performs slightly better than a simple model doesn’t fall into the 80% of the results but is a task that is hard and takes a lot of time. The complex model is that last hard-to-reach 20% that takes 80% of the effort. Before you start, convince yourself it is worth it.
Reason 3. Complex projects require more maintenance
The more complex the project, the more resources and time are needed to maintain it. This means that you’ll spend more time fixing bugs, optimizing the model, keeping the data up to date, and less time adding new features or improving the product. A simple project, on the other hand, requires less maintenance, which means that you can spend more time iterating on the MVP and adding new features to improve the product.
An important thought to keep in mind is that the best solution is often the simplest solution that fits the requirements. This can help you determine if that deep learning state-of-the-art model is truly worth the extra work that comes with it! If there are two models that perform equally well, and one is simple and the other is complex, go with the simple one.
One example from my work at a company: I tried to solve a scheduling problem with reinforcement learning. It was quite complex, and we were progressing slowly. The business became a bit annoyed and disappointed because we couldn’t show good results. When we switched our solution method to (good old) mathematical optimization, it went much faster! It was less interesting, but we gained the trust of the business and could implement new features and constraints easily.