Recommended way of installing python packages on Arch

Question

What's the recommended way of installing python packages on Arch? Searching for them on the AUR and installing them from there (or create a PKGBUILD file to make a package yourself) or using pip?

I started off by installing stuff from pacman and the AUR and don't know if it would be wise to mix with pip packages.

Answer 1

If you don't need the python packages for all users then you can install them in your home like this:

pip install --user packagename

Installing in your home will not conflict with the package manager.

By default pip install --user will install in your "user site" directory. Usually that is something like: /home/lesmana/.local/lib/python3.6/site-packages.

The following command will print, among others, your "user site" location:

python -m site

To customize the install location:

PYTHONUSERBASE=$HOME/some/dir pip install --user packagename

this will install everything under $HOME/some/dir

to run:

PYTHONUSERBASE=$HOME/some/dir $HOME/some/dir/bin/progname

See the pip manual for more information.

---

if you do want the python package for all users then the best place to install it is /opt. for example like this:

PYTHONUSERBASE=/opt/packagedir pip install packagename

(note the missing --user)

and to run, as above:

PYTHONUSERBASE=/opt/packagedir /opt/packagedir/bin/progname

Background explanation: /opt is commonly acknowledged by gnu/linux distributions as the directory where the local user or system administrator can install his own stuff. in other words: the package manager of distributions usually do not touch /opt. this is more or less standardized in the Filesystem Hierarchy Standard

For comfort for the users you will still want to write a wrapper script and place it in /bin or /usr/bin. This still bears risk of colliding with the distribution package manager but at least it is just one wrapper script file. So the damage that might be done is minimal. You can name the wrapper script something like local-foo or custom-foo to further minimize the risk of collision with the distribution package manager.

Alternatively you can modify PATH to include /opt/bin and place your wrapper script there. But this again requires you to modify a (or some) system files where PATH is defined which again may be overwritten by the distribution package manager.

In short: if you want to install for all users then do so in /opt. Where you place the wrapper script for comfort is a judgement call.

More Information about /opt and Filesystem Hierarchy Standard:

• What is the difference between /opt and /usr/local?
• http://www.pathname.com/fhs/2.2/fhs-3.12.html

Answer 2

The right way for ArchLinux

The right way to install PYTHON packages in ArchLinux is using PACMAN! To install packages to Python3 you have to use

sudo pacman -S python-'package'

If you want to install packages from Python2, you have to use

sudo pacman -S python2-'package'

Most python packages are in the ArchLinux repositories and the packages that are not in AUR (ArchLinux User Repositories) - for these packages you have to download the PKGBUILD file and compile. After that, you have to use PACMAN to finish the installation

makepkg -s
sudo pacman -U 'compiled-package'

The second right way for ArchLinux

When the package isn't in the AUR or the PKGBUILD isn't working, you can use PIP to install it to Python3

sudo pip install 'python-package'

or Python2

sudo pip2 install 'python-package'

BE AWARE: when you are using pip the same installation folder is shared with pacman and most of time, especially when you are updating all system packages (sudo pacman -Suy), will raise a conflict error. You always have to prefer the first option above.To solve conflict problems, just uninstall pip package and install they equivalent package on pacman (pip uninstall 'python-package').

You could give a chance to virtualenv or even conda

If you are planning to develop some python application or python package your better option is use virtual environments.

For python packaging applications you should try poetry it is current better option to manage application from start to finish. It is a much better option than requirements.txt + setup.py.

Another more simple option is use python-virtualenv. This can bring portability to your code and maintain old packages as well. Install it with

sudo pacman -S python-virtualenv

and try this

virtualenv -p /usr/bin/python3 yourenv
source yourenv/bin/activate
pip install package-name

When you create this environment yourenv, you will setup pip to install packages only into this environment, not to the entire system.

These other links can be useful with you want to learn more about managing packages on Linux with conda or virtualenv:

Installing Python Packages from a Jupyter Notebook

If you follow these rules, your ArchLinux will not break and won't have dependency problems between PACMAN and PIP.

Hope it's useful!

Answer 3

Update in 2021 for current practice:

If you are trying to ship software as part of the distribution, use the distribution's package manager. If you're doing anything else, the answer is most likely to use a venv (see https://docs.python.org/3/tutorial/venv.html).

This will help you with cases where your distribution ships old packages, or has some packages that depend on an old version of a package, and helps you avoid the impact of system upgrades and incompatibilities between versions of python.

Avoid using pip outside a venv, and especially avoid using pip as root. That will just clobber system packages and break things. The awkwardness between the language having a package manager that can be system-wide and the distribution's need to encapsulate package dependency management interact poorly in practice, but a venv allows you to escape that.

---

Answer from 2013:

Typically, in a distribution, it's recommended that you use the distribution's package manager. You can of course install things using pip (or, in the perl world, cpan), or compile and install things yourself. However, when you do this, the distribution's package manager doesn't know about them and can't manage dependencies or updates for them.

Using pip is pretty much equivalent to compiling and installing your own package. Do it if you need to, but prefer the distribution's package manager.

Answer 4

For certain packages (ones that I most probably don't want to hack), I make my own package using this:

https://github.com/bluepeppers/pip2arch

then build and install the PKGBUILD produced.

I leave virtualenvs for packages I might want to modify or hack.

Answer 5

In addition to the other answers here, check out the python-virtualenv package. It might be very useful if you are doing development on several projects with different dependencies with mismatching version numbers.

https://wiki.archlinux.org/index.php/Python_VirtualEnv

Also beware that there are two variants of pip and virtualenv. One for Python 2 and one for Python 3. If installation fails with a syntax error, you might be trying with the wrong version.

Answer 6

If you get This environment is externally managed... when you attempt a pip install in archlinux/manjaro/ubuntu/debian/other distros, then the supposedly UNRECOMMENDED way (assuming version 3.11) is:

sudo rm /usr/lib/python3.11/EXTERNALLY-MANAGED

Feel free to downvote the answer... but you are in charge of your system and IT IS YOU who decides whether your pip packages should be 'externally managed', what belongs in a virtualenv and what can be global.

Answer 7

• In 2023 the Arch recommended way to install non-Arch-packaged python modules is to use pipx.

• Trying to install a system wide module with pip now displays a warning showing a --break-system-packages flag being required.

• pipx creates self contained installs in their own venv: ~/.local/pipx/venvs/package_name