Solving All Python Dependency Issues on Apple Silicon | by David Farrugia | Feb, 2023
A guide to managing multiple Python architectures on the Mac M1 using pyenv
I love MacOS.
Perhaps its the certified Unix trait, its simplicity and elegance, or its seamless integration. But I love MacOS.
From my experience, things just work.
At the start of my professional career, I was a predominantly Windows user (and playing with Linux distributions on the side). Once I switched to Apple, as cliche as it might be, I never looked back.
Long gone where the days of following step-by-step tutorials only to face weird errors.
Long gone where the days of having to find boilerplate code for some Python packages to make them work specifically on a Windows system.
And this happened often. As a data scientist, I am constantly experimenting and trying out new Python packages.
I’ve been crunching numbers on my trusted 2019 Macbook Pro since its release. But lately, I’ve got the opportunity to spend some time working on a 2020 Macbook Pro M1.
I was excited. I’ve heard awesome news on how much more powerful the M1 chip was in comparison to the Intel chip.
Now, first things first. What a chip! The M1 is stupidly fast, power efficient, and miles ahead of the competition. The difference in performance from my older system is immediately noticeable.
However, as I started to set up my machine for data science and machine learning, I quickly realised that the one thing I loved most about Macs has quickly disappeared. I was unable to complete the most rudimentary of things — installing Python packages through pip. The M1 was throwing me all sorts of dependancy errors and weird conflicts.
This is primarily due to the switch from x86 to ARM64. Although the situation is drastically improved nowadays, with a lot of developers updating their packages to support ARM64 systems, a significant number of Python packages are still not updated.
During my research, I cam across Pyenv — a Python version manager.
Similar to how we use condaor venvto manage different packages and environments, Pyenv allows us to have multiple Python versions on the same machine. Besides the actual version (for example, Python 3.8.1 and Python 3.9.7), using Pyenv we can also decide whether we want the x86 version of Python or not.
Therefore, on a single machine, we can have an x86 version of Python 3.9.7 and an ARM64 version of Python 3.9.7. We can switch between the different Python versions as if they were native to our system.
We can still use conda or venv with them as well.
So, how can we get started with Pyenv.
Step 1: Install xcode on your system
xcode-select --install
Step 2: Install Homebrew
Make sure that you have brew installed. You can run brew help in your terminal to see if it is installed.
If you need to install brew, you can follow the instructions on their page.
Step 3: Install Pyenv
We can install Pyenv through Homebrew as follows:
brew install pyenv
Step 4: Update Shell Profiles
We need to update our shell profiles PATH (e.g., ~/.bash_profile, ~/.zshrc, etc.) to include Pyenv.
Also, to enable Pyenv to load up by default, we need to initialise it.
##### ~/.zprofile #####
eval "$(pyenv init --path)"##### ~/.zshrc #####
if command -v pyenv 1>/dev/null 2>&1; then
eval "$(pyenv init -)"
fi
Step 5: Install a Python Version
To list all available versions, we can use:
pyenv install --list
To install a specific Python version, we can use:
pyenv install 3.9.7
After installation, we can also set the new version as our global Python version.
pyenv global 3.9.7
Step 6: Install Packages
Once everything is setup, we can now start using our new Python version as if its native to our machine. We can create different virtual environments, and install different packages through pipor setup.py as we normally would.
We can use Rosetta and Pyenv to setup x86 Python versions on Apple silicon.
Rosetta is a tool bult by Apple to translate x86 architecture to ARM64.
We can install Rosetta using:
softwareupdate --install-rosetta
Once installed, we can set our terminal to open using Rosetta. This will convert our terminal to be x86.
To do this, right click on your Terminal application, click Get Info and tick Open using Rosetta.
TIP: duplicate the Terminal application so that you will have one that opens up using Rosetta (i.e., x86), and the other for ARM64.
You can now follow the same process as mentioned above on the x86 terminal to get access to x86 Python environments.
In this post we go through a simple yet effective solution to working with x86 Python versions on Apple silicon using Pyenv. Besides this benefit, Pyenv also offers a number of advantages, including the management of multiple Python versions on a single machine.
I would love to hear your thoughts on the topic, or anything AI and Data.
Drop me an email at [email protected] should you wish to get in touch.
A guide to managing multiple Python architectures on the Mac M1 using pyenv
I love MacOS.
Perhaps its the certified Unix trait, its simplicity and elegance, or its seamless integration. But I love MacOS.
From my experience, things just work.
At the start of my professional career, I was a predominantly Windows user (and playing with Linux distributions on the side). Once I switched to Apple, as cliche as it might be, I never looked back.
Long gone where the days of following step-by-step tutorials only to face weird errors.
Long gone where the days of having to find boilerplate code for some Python packages to make them work specifically on a Windows system.
And this happened often. As a data scientist, I am constantly experimenting and trying out new Python packages.
I’ve been crunching numbers on my trusted 2019 Macbook Pro since its release. But lately, I’ve got the opportunity to spend some time working on a 2020 Macbook Pro M1.
I was excited. I’ve heard awesome news on how much more powerful the M1 chip was in comparison to the Intel chip.
Now, first things first. What a chip! The M1 is stupidly fast, power efficient, and miles ahead of the competition. The difference in performance from my older system is immediately noticeable.
However, as I started to set up my machine for data science and machine learning, I quickly realised that the one thing I loved most about Macs has quickly disappeared. I was unable to complete the most rudimentary of things — installing Python packages through pip. The M1 was throwing me all sorts of dependancy errors and weird conflicts.
This is primarily due to the switch from x86 to ARM64. Although the situation is drastically improved nowadays, with a lot of developers updating their packages to support ARM64 systems, a significant number of Python packages are still not updated.
During my research, I cam across Pyenv — a Python version manager.
Similar to how we use condaor venvto manage different packages and environments, Pyenv allows us to have multiple Python versions on the same machine. Besides the actual version (for example, Python 3.8.1 and Python 3.9.7), using Pyenv we can also decide whether we want the x86 version of Python or not.
Therefore, on a single machine, we can have an x86 version of Python 3.9.7 and an ARM64 version of Python 3.9.7. We can switch between the different Python versions as if they were native to our system.
We can still use conda or venv with them as well.
So, how can we get started with Pyenv.
Step 1: Install xcode on your system
xcode-select --install
Step 2: Install Homebrew
Make sure that you have brew installed. You can run brew help in your terminal to see if it is installed.
If you need to install brew, you can follow the instructions on their page.
Step 3: Install Pyenv
We can install Pyenv through Homebrew as follows:
brew install pyenv
Step 4: Update Shell Profiles
We need to update our shell profiles PATH (e.g., ~/.bash_profile, ~/.zshrc, etc.) to include Pyenv.
Also, to enable Pyenv to load up by default, we need to initialise it.
##### ~/.zprofile #####
eval "$(pyenv init --path)"##### ~/.zshrc #####
if command -v pyenv 1>/dev/null 2>&1; then
eval "$(pyenv init -)"
fi
Step 5: Install a Python Version
To list all available versions, we can use:
pyenv install --list
To install a specific Python version, we can use:
pyenv install 3.9.7
After installation, we can also set the new version as our global Python version.
pyenv global 3.9.7
Step 6: Install Packages
Once everything is setup, we can now start using our new Python version as if its native to our machine. We can create different virtual environments, and install different packages through pipor setup.py as we normally would.
We can use Rosetta and Pyenv to setup x86 Python versions on Apple silicon.
Rosetta is a tool bult by Apple to translate x86 architecture to ARM64.
We can install Rosetta using:
softwareupdate --install-rosetta
Once installed, we can set our terminal to open using Rosetta. This will convert our terminal to be x86.
To do this, right click on your Terminal application, click Get Info and tick Open using Rosetta.
TIP: duplicate the Terminal application so that you will have one that opens up using Rosetta (i.e., x86), and the other for ARM64.
You can now follow the same process as mentioned above on the x86 terminal to get access to x86 Python environments.
In this post we go through a simple yet effective solution to working with x86 Python versions on Apple silicon using Pyenv. Besides this benefit, Pyenv also offers a number of advantages, including the management of multiple Python versions on a single machine.
I would love to hear your thoughts on the topic, or anything AI and Data.
Drop me an email at [email protected] should you wish to get in touch.
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.