How does /usr/bin/env know which program to use?

Question

When I use the shebang #!/usr/bin/env python to run a script, how does the system know which python to use? if I look for a python bin path in the environment variables I find nothing.

env | grep -i python

Answer 1

The shebang line (from “sharp bang”, i.e. #!) is processed by the kernel. The kernel doesn't want to know about environment variables such as PATH. So the name on the shebang line must be an absolute path to an executable. You can also specify an additional argument to pass to that executable before the script name (with system-dependent restrictions I won't go into here). For example, for a Python script, you can specify

#!/usr/bin/python

on the first line, and when you execute the script, the kernel will in fact execute /usr/bin/python /path/to/script. But that's not convenient: you need to specify the full path of the command. What if you have python in /usr/bin on some machines and /usr/local/bin on others? Or you want to set your PATH to /home/joe/opt/python-2.5/bin so as to use a specific version of Python? Since the kernel won't do the PATH lookup for you, the idea is to make the kernel run a command that in turns looks up the desired interpreter in the PATH:

#!/fixed/path/to/path-lookup-command python

That path-lookup-command must take the name of an executable as an argument and look it up in PATH and execute it: the kernel will run /fixed/path/to/path-lookup-command python /path/to/script. As it happens, the env command does just that. Its main purpose is to run a command with a different environment, but since it looks up the command name in $PATH, it's perfect for our purpose here.

Although this is not officially guaranteed, historic Unix systems provided env in /usr/bin, and modern systems have kept that location precisely because of the widespread use of #!/usr/bin/env. So, in practice, the way to specify that a script must be executed by the user's favorite Python interpreter is

#!/usr/bin/env python

Answer 2

The shebang expects a full path to the interpreter to use so the following syntax would be incorrect:

#!python

Setting a full path like this might work:

#!/usr/local/bin/python

but would be non portable as python might be installed in /bin, /opt/python/bin, or wherever other location.

Using env

#!/usr/bin/env python

is a method allowing a portable way to specify to the OS a full path equivalent to the one where python is first located in the PATH.

Answer 3

Right, so run:

env | grep PATH

Your $PATH is a list of directories. Unix will go through that list of directories, in order, until it finds "python".

You can see which directory it finds with the 'which' command:

which python