6

When I execute a program in Bash, for example, [ls][2], it sends its output to standard output (fd &1). And the ouput of the executed program is displayed in the terminal. How does Bash/terminal get the output of the ls command?

Peter Mortensen
  • 1,025
Max Koretskyi
  • 1,319
  • Are you talking about how just the output of ls shows up on the terminal, or how $(ls) allows the shell to get the output of ls as a string? – muru Dec 10 '14 at 11:27
  • I'd think that those are the same processes, and since you asked they are not :), so probably the first - how the output of ls shows up on the terminal – Max Koretskyi Dec 10 '14 at 11:30
  • @muru, so which answer is closer to how the output of ls shows up on the terminal? – Max Koretskyi Dec 10 '14 at 11:57
  • I would say mine. Check the update to see if it makes things clearer. – muru Dec 10 '14 at 12:44
  • @muru, ok, thanks! I'll read your updated answer and the stuff about dev/pts a little and get back to you with questions later :) – Max Koretskyi Dec 10 '14 at 12:47

2 Answers2

9

As a child process of the shell, ls inherits the open file descriptors of the shell. And the standard file descriptors (stdin, stdout, stderr (or 0, 1, 2)) are connected to a pseudo-terminal, which is handled by the terminal emulator.

For example (on a Linux system):

$ ls /proc/$$/fd -l
total 0
lrwx------ 1 muru muru 64 Dec 10 16:15 0 -> /dev/pts/3
lrwx------ 1 muru muru 64 Dec 10 16:15 1 -> /dev/pts/3
lrwx------ 1 muru muru 64 Dec 10 16:15 2 -> /dev/pts/3
lrwx------ 1 muru muru 64 Dec 10 16:15 255 -> /dev/pts/3
$ ls /proc/$(pgrep terminator -f)/fd -l | grep pts/3
lrwx------ 1 muru muru 64 Dec 10 16:15 26 -> /dev/pts/3

That is, the output of ls, or for that matter the shell itself, is not handled by the shell, but by the terminal emulator (GNOME Terminal, terminator, xterm, etc.).

You can test this out:

On Linux, find a pseudoterminal (pts) used by your terminal emulator (say GNOME Terminal):

$ ls -l /proc/$(pgrep -n gnome-terminal)/fd | grep pts
lrwx------ 1 muru muru 64 Dec 10 18:00 1 -> /dev/pts/1
lrwx------ 1 muru muru 64 Dec 10 18:00 15 -> /dev/pts/20
lrwx------ 1 muru muru 64 Dec 10 18:00 2 -> /dev/pts/1

Now, the non-standard fds (those other than 0,1,2) of gnome-terminal would be used by it to provide input and output for a shell. The terminal emulator reads in data send to that PTS and (after some processing, for colours and such) presents it on the screen. In this case, that would be 15, connected to pts/20. If I write something to that pts, I can expect it to appear in that terminal:

enter image description here

Further reading:

The other case, where I do things like:

echo $(ls)
a=$(date)
vim `command -v some_script`

is called Command Substitution. In command substitution, the output of the command is captured by the shell itself, and never reaches the terminal, unless you do print it out (for example, echo $(ls)). This case is handled in Hauke Laging's answer.

Community
  • 1
muru
  • 72,889
  • @HaukeLaging how much depth would be proper? An explanation of how an emulator handles the pseudoterminal? – muru Dec 10 '14 at 11:15
  • Sorry, I misunderstood the title (and didn't really read the question because it seemed so clear...). I thought the question was about command substitution. – Hauke Laging Dec 10 '14 at 11:23
  • @HaukeLaging It could be about command substitution, in which case you're right and this answer doesn't make sense. I'll ask OP for a clarification. – muru Dec 10 '14 at 11:26
  • That 1 -> /dev/pts/1 in the fds of gnome-terminal point to the terminal gnome-terminal was launched from. The terminal emulators don't need access to the slave side of the pseudo terminal as they interact with it on the master side (/dev/ptmx). xterm for instance doesn't have any fd open to the slave side after it has started the slave application (usually the shell). – Stéphane Chazelas Dec 10 '14 at 13:02
  • @StéphaneChazelas I launched gnome-terminal from GNOME Shell's run prompt, so shouldn't it go to /dev/null? I tested out xterm and as you said it opens /dev/ptmx. Time for me to go and read what a master side is. – muru Dec 10 '14 at 13:06
  • @StéphaneChazelas Ok, reading pts(4), the pts that xterm got by opening ptmx is only opened by the shell (and its children), right? – muru Dec 10 '14 at 13:08
  • gnome-terminal is special because it implements a kind of client/server mechanism. See http://unix.stackexchange.com/q/117981 for more info on ptys. – Stéphane Chazelas Dec 10 '14 at 13:08
  • @StéphaneChazelas yet I observe the same behaviour in terminator, xfce4-terminal, lxterminal and konsole. So which one is special? – muru Dec 10 '14 at 13:16
  • I don't know about your GNOME Shell software, possibly it also uses a pseudo-terminal to connect to the stdin/out/err of the applications it launches or it inherits it from the program that started it. You could try and see if it also has a fd open to /dev/ptmx or what its stdin/out/err are. See How can we know who's at the other end of a pseudo-terminal device? – Stéphane Chazelas Dec 10 '14 at 13:19
  • @StéphaneChazelas the GNOME Shell (and pts/1) is secondary. If I understand your comment correctly, xterm doesn't open the slave side it was allocated (say, pts/20), gnome-terminal is special because it does open the slave side. I tested out 4 other emulators, and all four open the slave side as well. So, is xterm special or are these five special? – muru Dec 10 '14 at 13:23
  • I meant "special" in that when you run a gnome-terminal command, it just instructs a server to spawn a new window, so the extra gnome-terminal processes don't inherit the fds of the gnome-terminal command you run, but those of that server. All those terminals you mention are probably all using vte underneath which probably forgets to close that fd. I'd bet they don't do anything with that fd. xterm and rxvt (separate implementations) don't have a fd open on the slave. It doesn't make sense for the emulator to have a fd there unless they want to run a new shell when the first one's done f.i. – Stéphane Chazelas Dec 10 '14 at 13:43
3

It turns out that I misunderstood the question in the sense of command substitution. Only in that case the shell is involved in output handling.

Hope this is of interest, too...

I attach to a shell with strace -p 2140 before I run echo $(/bin/echo foo) in this thell. This is part of the result:

pipe([3, 4])
pipe([5, 6])
...
read(3, "foo\n", 128)

This is what happens in the child process:

dup2(4, 1)
close(4)
close(3)
...
execve("/bin/echo", ...

The shell connects the file descriptors 3 and 4 and then forks. The child process makes fd 4 its stdout before it runs the new program. Thus everything the child writes to stdout can be read by the parent shell on its fd 3.

Hauke Laging
  • 90,279
  • Are you talking about reading the output of command substitution here? – muru Dec 10 '14 at 11:22
  • thanks, so you answer doesn't apply to this how the output of ls shows up on the terminal? – Max Koretskyi Dec 10 '14 at 12:17
  • @Maximus No. My answer covers the more complicated case of $(/bin/echo foo) instead of echo foo. In your simple case the shell is not involved in the output at all. – Hauke Laging Dec 10 '14 at 12:34
  • @HaukeLaging, I see, thanks. I don't understand muru' answer though. And if it's not involved, how come the output appears on the screen? – Max Koretskyi Dec 10 '14 at 12:40
  • 3
    @Maximus The problem is probably that you don't know what a terminal / pseudo-terminal is. Have a look at Wikipedia: http://en.wikipedia.org/wiki/Pseudo_terminal Both the shell and the programs it runs write their output to a file descriptor – like they would write to a real file. The terminal they happen to write to makes you see their (printable) output. – Hauke Laging Dec 10 '14 at 12:46
  • Yeah, most likely this is what I need to read before trying to understand the matter. I'll do that. Thanks! Best! – Max Koretskyi Dec 10 '14 at 12:48