Stop using “pip freeze” for your Python projects | by Prakhar Rathi | Jun, 2022

A quick discussion into why `pip freeze` is not as cool when it comes to managing Python dependencies

I had come across pip freeze and virtual env a couple years ago and I was absolutely blown away. I was someone who was always scared of virtual environments (Don’t even ask) but once I learnt how easy it was to manage my dependencies, I couldn’t stop. I felt like a senior developer, making a virtualenv for all my projects. Ever since then my developer skills have increased multifold and I found the perfect way to manage my Python projects, right? Wrong!

A few months into this fairy tale, I started facing issues when I would go back to my old projects. They would stop running or the available dependencies would throw some compatibility errors. I was perplexed because I thought I had done everything right. I had separated the projects and their dependencies by creating a virtual environments so why weren’t my old projects running. Turns out the very thing that made me a better Python developer was becoming my hurdle — pip freeze. This was because of the way sub-dependencies were handled using pip freeze.

Earlier, when I was starting a new project and as soon as I would install the libraries, I would run my favorite command.

pip freeze > requirements.txt

Here’s why it caused issues. Let’s say you install package A in your project, which might have a sub-dependency B, C and D. Now, your requirements.txt file with the above command would look like

A==1.0
B==2.0
C==1.4
D==1.2

Now say, the owner of library A releases a new version which uses a different version of library B and removes library C. Since B and C are already installed pip freeze would pick it up automatically and dump them in the exact versions as they were originally installed. Now, in a project with 100s of dependencies, your requirements file will become very problematic when you change libraries. You would have to identify all the sub-dependencies and delete them accordingly. In this example if A was now removed from the project, you would still be stuck with B, C and D even though they were only installed because of A. Deleting each of them is a mammoth task and can be very annoying on large projects.

There are other many issues that stem out of this problem which can break your project any day in the future.

However, I am not here with just problems but I also have a solution. I found a library called pipreqs, which fixes all the above issues and is very user-friendly.

Why is it better?

Here are reasons why switching to pipreqs is a better idea than using pip freeze for the requirements file.

1. pip freeze only saves the packages that were installed with pip install in the virtual environment [1]

pip freeze only installs those packages which were installed using the pip install command. However, pip is not the only python package manager. We can also use chocolatey, conda, setuptools etc. and they are not supported by pip freeze so we would have to write them manually in the requirements.txt file. pipreqs, on the other hand, has no such restriction.

2. pip freeze saves all packages and dependencies in the environment including those that you don’t use in your current project [1]

This is the biggest drawback of pip freeze. In a project, the dependencies constantly change and have to be added, updated and deleted. However, it is a monumental task to achieve this using pip freeze because it dumps whatever is already installed in the environment. pipreqs, on the other hand, only puts those libraries in the requirements file which have been used in the project through imports. This is extremely powerful when you are trying to change the requirements file later.

3. pipreqs is extremely easy to use

To generate a requirements.txt file, all you have to do is run the following command.

$ pipreqs

If the requirements.txt file already exists then run the following command.

$ pipreqs --force

This generates a requirements.txt file in the home directory of your project. You can also provide a path if you want to save the file in a specific location.

$ pipreqs /home/project/location

To read more about the library, click here. There are other alternatives like we pip-tools and poetry as well which you could check out.

A quick discussion into why `pip freeze` is not as cool when it comes to managing Python dependencies

I had come across pip freeze and virtual env a couple years ago and I was absolutely blown away. I was someone who was always scared of virtual environments (Don’t even ask) but once I learnt how easy it was to manage my dependencies, I couldn’t stop. I felt like a senior developer, making a virtualenv for all my projects. Ever since then my developer skills have increased multifold and I found the perfect way to manage my Python projects, right? Wrong!

A few months into this fairy tale, I started facing issues when I would go back to my old projects. They would stop running or the available dependencies would throw some compatibility errors. I was perplexed because I thought I had done everything right. I had separated the projects and their dependencies by creating a virtual environments so why weren’t my old projects running. Turns out the very thing that made me a better Python developer was becoming my hurdle — pip freeze. This was because of the way sub-dependencies were handled using pip freeze.

Earlier, when I was starting a new project and as soon as I would install the libraries, I would run my favorite command.

pip freeze > requirements.txt

Here’s why it caused issues. Let’s say you install package A in your project, which might have a sub-dependency B, C and D. Now, your requirements.txt file with the above command would look like

A==1.0
B==2.0
C==1.4
D==1.2

Now say, the owner of library A releases a new version which uses a different version of library B and removes library C. Since B and C are already installed pip freeze would pick it up automatically and dump them in the exact versions as they were originally installed. Now, in a project with 100s of dependencies, your requirements file will become very problematic when you change libraries. You would have to identify all the sub-dependencies and delete them accordingly. In this example if A was now removed from the project, you would still be stuck with B, C and D even though they were only installed because of A. Deleting each of them is a mammoth task and can be very annoying on large projects.

There are other many issues that stem out of this problem which can break your project any day in the future.

However, I am not here with just problems but I also have a solution. I found a library called pipreqs, which fixes all the above issues and is very user-friendly.

Why is it better?

Here are reasons why switching to pipreqs is a better idea than using pip freeze for the requirements file.

1. pip freeze only saves the packages that were installed with pip install in the virtual environment [1]

pip freeze only installs those packages which were installed using the pip install command. However, pip is not the only python package manager. We can also use chocolatey, conda, setuptools etc. and they are not supported by pip freeze so we would have to write them manually in the requirements.txt file. pipreqs, on the other hand, has no such restriction.

2. pip freeze saves all packages and dependencies in the environment including those that you don’t use in your current project [1]

This is the biggest drawback of pip freeze. In a project, the dependencies constantly change and have to be added, updated and deleted. However, it is a monumental task to achieve this using pip freeze because it dumps whatever is already installed in the environment. pipreqs, on the other hand, only puts those libraries in the requirements file which have been used in the project through imports. This is extremely powerful when you are trying to change the requirements file later.

3. pipreqs is extremely easy to use

To generate a requirements.txt file, all you have to do is run the following command.

$ pipreqs

If the requirements.txt file already exists then run the following command.

$ pipreqs --force

This generates a requirements.txt file in the home directory of your project. You can also provide a path if you want to save the file in a specific location.

$ pipreqs /home/project/location

To read more about the library, click here. There are other alternatives like we pip-tools and poetry as well which you could check out.