Introducing Reloading: Never Re-Run Your Python Code Again To Print More Details | by Avi Chawla | Nov, 2022
Modify your code during run-time and save hours of work time
While running Python scripts, I have often found myself in situations where I forgot to print all the necessary details to track the pipeline’s progress.
This is typically observed in training machine learning models. More often than not, folks (including me) often forget to:
- Add necessary logging details.
- Print essential training details/metrics such as accuracy, error, precision, etc.
- Save the model after every
kepochs, and many more.
I am sure you have been there too.
Of course, the problem is not just limited to machine learning. Many face the same issue in other domains as well, such as Web Scrapping, where people realize after running their codes that they should have scrapped some more details, etc.
Left with no choice, one has to unwillingly stop the code, add the necessary details and re-run the code again. This can be incredibly frustrating if your pipeline has been running for a few hours.
But what if I told you that there is a neat trick to this? In other words, it is actually possible to make changes to an already running code without losing the current progress.
This is what I will explore in this blog.
Let’s begin 🚀!
Reloading, as the name suggests, is a Python library that allows you to reload a loop (or a function) from the source before each iteration.
Therefore, you can modify an already running code and add more details to it without losing any current progress. Isn’t that cool?
To install it, use the following command:
pip install reloading
Reloading A Loop
Consider that you have a loop that takes an initial value and halves it after every iteration.
However, we mistakenly forgot to print the iteration number in this loop and now want to modify it.
Of course, without reloading, you have no choice but to re-run it.
However, if you want to reload the body of the for loop before each iteration, wrap the iterator with reloading, as shown below:
Now, you can modify the code during run-time. A demo is shown below:
As demonstrated above, we modified the body of the for loop. As a result, we get to see new details in the output panel, while retaining the current progress.
Reloading a Function
Similar to reloading a loop, you can also reload the body of a function after each iteration. Consider the function half_value below:
To reload the body of a function, decorate it with the reloading decorator. This is shown below:
Now, you can modify the function during run-time. A demo is shown below:
Perfect!
With this, we come to the end of this blog. I hope you learned something new.
I am confident that this trick will save you tons of time while running some high-run-time codes, such as training machine learning models.
Found this tip interesting?
If you want to learn more such elegant tips and tricks about Data Science and Python, I post an informative tip daily on LinkedIn.
You can find all the tips I have posted in My LinkedIn Post Archive. You can follow me on LinkedIn to see all future posts.
Alternatively, you can also receive them via email:
Get a weekly summary of the top 1% research papers, news, repos, and tweets in Machine Learning.
I like to explore, experiment, and write about data science concepts and tools. You could connect with me on LinkedIn.
Thanks for reading!
Modify your code during run-time and save hours of work time
While running Python scripts, I have often found myself in situations where I forgot to print all the necessary details to track the pipeline’s progress.
This is typically observed in training machine learning models. More often than not, folks (including me) often forget to:
- Add necessary logging details.
- Print essential training details/metrics such as accuracy, error, precision, etc.
- Save the model after every
kepochs, and many more.
I am sure you have been there too.
Of course, the problem is not just limited to machine learning. Many face the same issue in other domains as well, such as Web Scrapping, where people realize after running their codes that they should have scrapped some more details, etc.
Left with no choice, one has to unwillingly stop the code, add the necessary details and re-run the code again. This can be incredibly frustrating if your pipeline has been running for a few hours.
But what if I told you that there is a neat trick to this? In other words, it is actually possible to make changes to an already running code without losing the current progress.
This is what I will explore in this blog.
Let’s begin 🚀!
Reloading, as the name suggests, is a Python library that allows you to reload a loop (or a function) from the source before each iteration.
Therefore, you can modify an already running code and add more details to it without losing any current progress. Isn’t that cool?
To install it, use the following command:
pip install reloading
Reloading A Loop
Consider that you have a loop that takes an initial value and halves it after every iteration.
However, we mistakenly forgot to print the iteration number in this loop and now want to modify it.
Of course, without reloading, you have no choice but to re-run it.
However, if you want to reload the body of the for loop before each iteration, wrap the iterator with reloading, as shown below:
Now, you can modify the code during run-time. A demo is shown below:
As demonstrated above, we modified the body of the for loop. As a result, we get to see new details in the output panel, while retaining the current progress.
Reloading a Function
Similar to reloading a loop, you can also reload the body of a function after each iteration. Consider the function half_value below:
To reload the body of a function, decorate it with the reloading decorator. This is shown below:
Now, you can modify the function during run-time. A demo is shown below:
Perfect!
With this, we come to the end of this blog. I hope you learned something new.
I am confident that this trick will save you tons of time while running some high-run-time codes, such as training machine learning models.
Found this tip interesting?
If you want to learn more such elegant tips and tricks about Data Science and Python, I post an informative tip daily on LinkedIn.
You can find all the tips I have posted in My LinkedIn Post Archive. You can follow me on LinkedIn to see all future posts.
Alternatively, you can also receive them via email:
Get a weekly summary of the top 1% research papers, news, repos, and tweets in Machine Learning.
I like to explore, experiment, and write about data science concepts and tools. You could connect with me on LinkedIn.
Thanks for reading!
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.