6

I understand that when I run exit it terminates my current shell because exit command run in the same shell. I also understand that when I run exit & then original shell will not terminate because & ensures that the command is run in sub-shell resulting that exit will terminate this sub-shell and return back to original shell. But what I do not understand is why commands with and without & looks exactly the same under pstree, in this case sleep 10 and sleep 10 &. 4669 is the PID of bash under which first sleep 10 and then sleep 10 & were issued and following output was obtained from another shell instance during this time:

# version without &
$ pstree 4669
bash(4669)───sleep(6345)

# version with &
$ pstree 4669
bash(4669)───sleep(6364)

Should't the version with & contain one more spawned sub-shell (e.g. in this case with PID 5555), like this one?

bash(4669)───bash(5555)───sleep(6364)

PS: Following code was omitted from output of pstree beginning for better readability:

systemd(1)───slim(1009)───ck-launch-sessi(1370)───openbox(1551)───/usr/bin/termin(4510)───bash(4518)───screen(4667)───screen(4668)───
Wakan Tanka
  • 785
  • 2
    I usually check the Related questions for possible duplicates (then flag them) before answering a question. In this case, I strongly disagree that this is a duplicate since it focuses on commands being run in the background – not enclosed in parentheses. This question is informative, thought-provoking and offers a greater insight into an aspect of subshells not covered in the “duplicate”; IMO, this deserves more attention (and upvotes). BTW, I’ve extended and further clarified my answer to incorporate suggestions from the comments (and deleted my own comments to minimise the clutter). – Anthony Geoghegan Jan 21 '16 at 13:44

1 Answers1

8

Until I started answering this question, I hadn’t realised that using the & control operator to run a job in the background starts a subshell. Subshells are created when commands are wrapped in parentheses or form part of a pipeline (each command in a pipeline is executed in its own subshell).

The Lists of Commands section of the Bash manual (thanks jimmij) states:

If a command is terminated by the control operator ‘&’, the shell executes the command asynchronously in a subshell. This is known as executing the command in the background. The shell does not wait for the command to finish, and the return status is 0 (true).

As I understand it, when you run sleep 10 & the shell forks to create a new child process (a copy of itself) and then immediately execs to replace this child process with code from the external command (sleep). This is similar to what happens when a command is run as normal (in the foreground). See the Fork–exec Wikipedia article for a short overview of this mechanism.

I couldn’t understand why Bash would run backgrounded commands in a subshell but it makes sense if you also want to be able to run shell builtins such as exit or echo to be run in the background (not just external commands).

When it’s a shell builtin that’s being run in the background, the fork happens (resulting in a subshell) without an exec call to replace itself with an external command. Running the following commands shows that when the echo command is wrapped in curly braces and run in the background (with the &), a subshell is indeed created:

$ { echo $BASH_SUBSHELL $BASHPID; }
0 21516
$ { echo $BASH_SUBSHELL $BASHPID; } &
[1] 22064
$ 1 22064

In the above example, I wrapped the echo command in curly braces to avoid BASH_SUBSHELL being expanded by the current shell; curly braces are used to group commands together without using a subshell. The second version of the command (ending with the & control operator) clearly demonstrates that terminating the command with the ampersand has resulted in a subshell (with a new PID) being created to execute the echo builtin. (I’m probably simplifying the shell’s behaviour here. See mikeserv’s comment.)

I would never have thought of running exit & and had I not read your question, I would have expected the current shell to quit. Knowing now that such commands are run in a subshell, your explanation that it’s the subshell which exits makes sense.

“Why is subshell created by background control operator (&) not displayed under pstree”

As mentioned above, when you run sleep 10 &, Bash forks itself to create the subshell but since sleep is an external command, it calls the exec() system call which immediately replaces the Bash code and data in the child process with a running copy of the sleep program. By the time you run pstree, the exec call will already have completed and the child process will now have the name “sleep”.

While away from my computer, I tried to think of a way of keeping the subshell running long enough for the subshell to be displayed by pstree. I figured we could run the command through the time builtin:

$ time sleep 11 &
[2] 4502
$ pstree -p 26793
bash(26793)─┬─bash(4502)───sleep(4503)
            └─pstree(4504)

Here, the Bash shell (26793) forks to create a subshell (4502) in order to execute the command in the background. This subshell runs its own time builtin command which, in turn, forks (to create a new process with PID 4503) and execs to run the external sleep command.

Using named pipes, jimmij came up with a clever way to keep the subshell created to run exit alive long enough for it to be displayed by pstree:

$ mkfifo file
$ exit <file &
[2] 6413
$ pstree -p 26793
bash(26793)─┬─bash(6413)
            └─pstree(6414)
$ echo > file
$ jobs
[2]-  Done    exit < file

Redirecting stdin from a named pipe is clever as it causes the subshell to block until it receives input from the named pipe. Later, redirecting the output of echo (without any arguments) writes a newline character to the named pipe which unblocks the subshell process which, in turn, runs the exit builtin command.

Similarly, for the sleep command:

$ mkfifo named_pipe
$ sleep 11 < named_pipe &
[1] 6600
$ pstree -p 26793
bash(26793)─┬─bash(6600)
            └─pstree(6603)

Here we see that the subshell created to run the command in the background has a PID of 6600. Next, we unblock the process by writing a newline character to the pipe:

$ echo > named_pipe

The subshell then execs to run the sleep command.

$ pstree -p 26793
bash(26793)─┬─pstree(6607)
            └─sleep(6600)

After the exec() call, we can see that the child process (6600) is now running the sleep program.

Anthony Geoghegan
  • 12,950
  • My premise is based on this article: "This trailing ampersand directs the shell to run the command in the background, that is, it is forked and run in a separate sub-shell, as a job, asynchronously." http://bashitout.com/2013/05/18/Ampersands-on-the-command-line.html – Wakan Tanka Jan 20 '16 at 22:43
  • & always causes the shell to fork. The forked shell process either handles the command itself (which is the case with exit), or makes a system call to (some form of) exec to replace itself with the given command. – chepner Jan 20 '16 at 22:49
  • I assume the reason that exit starts a subshell is that it is a shell builtin. The shell has to create a new process. But what process should that be if there is no external command to run? Technically this may the same like a normal subshell but "semantically" it seems slightly different. You don't call a subshell but the technical circumstances enforce a subshell. – Hauke Laging Jan 20 '16 at 22:52
  • Bash manual states it clearly: If a command is terminated by the control operator &, the shell executes the command in the background in a subshell. The shell does not wait for the command to finish, and the return status is 0. Thus subshell is always created with & and one can see this with echo "$$ $BASHPID" &. – jimmij Jan 20 '16 at 22:56
  • There are a lot of factors besides which affect the shell's behavior here. For example active traps will preclude an implicit exec , often trailing newlines do also, or i/o attachments associated w/ complex commands (like your { echo; }& thing). Basically if there is any chance of the shell having to do something else it probably shouldn't exec. use `PS4=$$\ + "$0" -cx 'test cases' to more simply test. – mikeserv Jan 20 '16 at 23:26
  • @AnthonyGeoghegan "when you run sleep 10 & the shell forks to create a new child process (a copy of itself) and then immediately execs to replace this child process with code from the sleep command." So in theory if you are not fast enough you cannot see created sub-shell? "Having read Chepner’s comment, it seems that when it’s a shell builtin that’s being run in the background, the fork happens without the exec." Sorry I did not still catch it, even if exit would be executed with exec the situation will be exactly the same (sub-shell will terminate and original persists) am I wrong? – Wakan Tanka Jan 20 '16 at 23:34
  • @Wakan Tanka cmd & is grosso modo equivalent (from the subshells point of view) of bash -c 'cmd'. If there is only one command bash replace itself with it (if thats external command), otherwise stays. Obviously you are not fast enough to see subshell in the pstree if cmd is builtin, but e.g. run mkfifo file, and then after exit<file & watch pstree. – jimmij Jan 20 '16 at 23:42
  • @Anthony Geoghegan You can see how sleep replaces bash in the same manner as I previously wrote. mkdir file and then sleep 10 <file & and observe pstree in the other terminal. See the bash with some pid? Now run echo foo >file (still in some other terminal) and again pstree. See the sleep with the same pid as previously bash had? – jimmij Jan 21 '16 at 00:22
  • @WakanTanka - you can catch the subshell before its replaced by explicitly tracking its PID. bash -c cmd is not equivalent to cmd & in that bash -c gets its own $$ while & doesn't. alias 'tracepid=PS4='\''$0 -- $* : $PPID.$$ + '\'' sh -cx '\''"$0" "$@"'\'' '. You can call up any simple command with that alias and it will always print the commands PID and its args to stderr before the command is run. stick that inside your & subshells to the subshell's PIDs in realtime and to get the & subshell's PID's as you go. tracepid sleep 10 & for example. – mikeserv Jan 21 '16 at 00:42
  • @HaukeLaging "But what process should that be if there is no external command to run?" A fork always creates a copy of the current process; there's no such thing as a process without a command. The shell starts any external command by forking, and the new copy of the shell then executes exec to replace itself with the new command. – chepner Jan 21 '16 at 01:05
  • @Anthony Geoghegan: Interesting answer.  A quick note: reading from a pipe (named or otherwise) waits for input to be available.  (Or for the pipe to be closed; EOF acts like input in this context.)  But sleep 11 < named_pipe doesn’t read from the pipe, it only *opens it for reading, because sleep doesn’t read from stdin.  An open for reading* on a named pipe blocks only until somebody *opens the fifo for writing.  So, you could replace echo > named_pipe with : > named_pipe (using the no-op command, :*) or even, in some shells, > named_pipe (using a null command). – G-Man Says 'Reinstate Monica' Oct 25 '16 at 20:06
  • Thanks @G-Man for the very clear explanation of pipes and blocking (great use of Markdown for adding appropriate emphasis). I didn't realise that a FIFO only has to be opened (for writing) to unblock a process that has it opened for reading. This answer could do with a rewrite and when I find the time, I'll incorporate that information into it. – Anthony Geoghegan Oct 26 '16 at 09:21