13

When I run this script, intended to run until killed...

# foo.sh

while true; do sleep 1; done

...I'm not able to find it using ps ax:

>./foo.sh

// In a separate shell:
>ps ax | grep foo.sh
21110 pts/3    S+     0:00 grep --color=auto foo.sh

...but if I just add the common "#!" header to the script...

#! /usr/bin/bash
# foo.sh

while true; do sleep 1; done

...then the script becomes findable by the same ps command...

>./foo.sh

// In a separate shell:
>ps ax | grep foo.sh
21319 pts/43   S+     0:00 /usr/bin/bash ./foo.sh
21324 pts/3    S+     0:00 grep --color=auto foo.sh

Why is this so?
This may be a related question: I thought "#" was just a comment prefix, and if so "#! /usr/bin/bash" is itself nothing more than a comment. But does "#!" carry some significance greater than as just a comment?

Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
StoneThrow
  • 1,717

3 Answers3

14

When the current interactive shell is bash, and you run a script with no #!-line, then bash will run the script. The process will show up in the ps ax output as just bash.

$ cat foo.sh
# foo.sh

echo "$BASHPID"
while true; do sleep 1; done

$ ./foo.sh
55411

In another terminal:

$ ps -p 55411
  PID TT  STAT       TIME COMMAND
55411 p2  SN+     0:00.07 bash

Related:

The relevant sections form the bash manual:

If this execution fails because the file is not in executable format, and the file is not a directory, it is assumed to be a shell script, a file containing shell commands. A subshell is spawned to execute it. This subshell reinitializes itself, so that the effect is as if a new shell had been invoked to handle the script, with the exception that the locations of commands remembered by the parent (see hash below under SHELL BUILTIN COMMANDS) are retained by the child.

If the program is a file beginning with #!, the remainder of the first line specifies an interpreter for the program. The shell executes the specified interpreter on operating systems that do not handle this executable format themselves. [...]

This means that running ./foo.sh on the command line, when foo.sh does not have a #!-line, is the same as running the commands in the file in a subshell, i.e. as

$ ( echo "$BASHPID"; while true; do sleep 1; done )

With a proper #!-line pointing to e.g. /bin/bash, it is as doing

$ /bin/bash foo.sh
Kusalananda
  • 333,661
  • I think I follow, but what you say is also true in the second case: bash also runs the script in the second case, as can be observed when ps shows the script running as "/usr/bin/bash ./foo.sh". So in the first case, as you say, bash will run the script, but wouldn't that script need to be "passed" to the forked bash executable, as with the second case? (and if so, I imagine it'd be findable with the pipe to grep...?) – StoneThrow Jun 08 '18 at 18:34
  • @StoneThrow See updated answer. – Kusalananda Jun 08 '18 at 18:41
  • "... except that you get a new process" - well, you get a new process either way, except that $$ still points at the old one in the subshell case (echo $BASHPID / bash -c 'echo $PPID'). – Michael Homer Jun 09 '18 at 09:43
  • @MichaelHomer Ah, thanks for that! Will update. – Kusalananda Jun 09 '18 at 09:46
12

When a shell script starts with #!, that first line is a comment as far as the shell is concerned. However the first two characters are meaningful to another part of the system: the kernel. The two characters #! are called a shebang. To understand the role of the shebang, you need to understand how a program is executed.

Executing a program from a file requires action from the kernel. This is done as part of the execve system call. The kernel needs to verify the file permissions, free the resources (memory, etc.) associated to the executable file currently running in the calling process, allocate resources for the new executable file, and transfer control to the new program (and more things that I won't mention). The execve system call replaces the code of the currently running process; there's a separate system call fork to create a new process.

In order to do this, the kernel has to support the format of the executable file. This file has to contain machine code, organized in a way that the kernel understands. A shell script doesn't contain machine code, so it can't be executed this way.

The shebang mechanism allows the kernel to defer the task of interpreting the code to another program. When the kernel sees that the executable file begins with #!, it reads the next few characters and interprets the first line of the file (minus the leading #! and optional space) as a path to another file (plus arguments, which I won't discuss here). When the kernel is told to execute the file /my/script, and it sees that the file begins with the line #!/some/interpreter, the kernel executes /some/interpreter with the argument /my/script. It's then up to /some/interpreter to decide that /my/script is a script file that it should execute.

What if a file neither contains native code in a format that the kernel understands, and does not start with a shebang? Well, then the file isn't executable, and the execve system call fails with the error code ENOEXEC (Executable format error).

This could be the end of the story, but most shells implement a fallback feature. If the kernel returns ENOEXEC, the shell looks at the content of the file and checks whether it looks like a shell script. If the shell thinks the file looks like a shell script, it executes it by itself. The details of how it does this depends on the shell. You can see some of what's happening by adding ps $$ in your script, and more by watching the process with strace -p1234 -f -eprocess where 1234 is the PID of the shell.

In bash, this fallback mechanism is implemented by calling fork but not execve. The child bash process clears its internal state by itself and opens the new script file to run it. Therefore the process that runs the script is still using the original bash code image and the original command line arguments passed when you invoked bash originally. ATT ksh behaves in the same way.

% bash --norc
bash-4.3$ ./foo.sh 
  PID TTY      STAT   TIME COMMAND
21913 pts/2    S+     0:00 bash --norc

Dash, in contrast, reacts to ENOEXEC by calling /bin/sh with the path to the script passed as an argument. In other words, when you execute a shebangless script from dash, it behaves as if the script had a shebang line with #!/bin/sh. Mksh and zsh behave in the same way.

% dash
$ ./foo.sh
  PID TTY      STAT   TIME COMMAND
21427 pts/2    S+     0:00 /bin/sh ./foo.sh
Gilles 'SO- stop being evil'
  • 829,060
  • Great, comprehensize answer. One question RE: the fallback implementation you explained: I suppose since a child bash is forked, it has access to the same argv[] array as its parent, which is how it knows "the original command line arguments passed when you invoked bash originally", and if so, this is why the child is not passed the original script as an explicit argument (hence why it is not findable by the grep) - is that accurate? – StoneThrow Jun 08 '18 at 18:58
  • 1
    You can actually turn off the kernel shebang behaviour (the BINFMT_SCRIPT module controls this and can be removed/modularized, although it's usually statically linked into the kernel), but I don't see why you would ever want to, except perhaps in an embedded system. As a workaround for this possibility, bash has a configuration flag (HAVE_HASH_BANG_EXEC) to compensate! – ErikF Jun 08 '18 at 18:59
  • 2
    @StoneThrow It's not so much that the child bash “knows the original command line arguments”, as that it doesn't modify them. A program can modify what ps reports as the command line arguments, but only up to a point: it has to modify an existing memory buffer, it can't enlarge this buffer. So if bash tried to modify its argv to add the name of the script, it wouldn't always work. The child is not “passed an argument” because there's never an execve system call in the child. It's just the same bash process image that keeps running. – Gilles 'SO- stop being evil' Jun 08 '18 at 19:12
-1

In the first case, the script is run by a forked off child from your current shell.

You should first run echo $$ and then have a look at a shell that has the process id of your shell as parent process id.

schily
  • 19,173