0

I'm playing around with file descriptors to better grok them and I'm failing to understand the following.

$ grep "..." 3>&1 1>/dev/null
1
12
13
123
321
3

The above does not show in the shell any of the matches, this is obviously happening because I'm redirecting to /dev/null. What I don't understand is why 3>&1 doesn't make it so that I still see the output, since I made a copy of it in fd 3.

What am I missing?

exit_status
  • 327
  • Why would grep output anything to fd 3? – muru May 20 '19 at 00:21
  • Why wouldn't it? After all 1 is stdout and I'm duplicating it. Obviously I'm missing something, but I don't know what. – exit_status May 20 '19 at 00:22
  • Because nothing tells it to? I don't see any reason why grep should magically start printing output on some random fd. – muru May 20 '19 at 00:34
  • As for your duplicating fd 1, see https://askubuntu.com/a/860245 – muru May 20 '19 at 00:37
  • @muru Is the case that only fds 1 and 2 are supposed to print to the screen? My reasoning is as follows: I make a copy of fd 1 in 3. Since 1 prints to the screen and 3 is a copy of 1, so will fd 3 regardless of what I do to fd 1. I can't explain it better. I know I'm missing something, but my knowledge is insufficient to phrase it better than this. – exit_status May 20 '19 at 00:43
  • 1
    A fd does not "print" anything, but it's printed to. And grep is not printing anything to fd 3, so it doesn't matter if 3 points to the "screen" or elsewhere. –  May 20 '19 at 00:52
  • Ok, take your reasoning to its conclusion: why should grep automagically start using this fd 3? No reason, so it doesn't – muru May 20 '19 at 01:15
  • @muru I think I get it now. What I needed to understand this was to read something like "Commands will use fds 1 and 2 to print to the screen regardless of redirections". I think it's perfectly reasonable to think that a copy will maintain every property from the original object, here, it doesn't maintain the property "this will be used to print to screen" and that was my confusion. – exit_status May 20 '19 at 07:47

2 Answers2

5

Your redirections do make FD 3 a copy of FD 1, so FD 3 will point to standard output and anything written to it will go to (by default) the TTY. You then redirect FD 1 to /dev/null, so anything written to FD 1 will be discarded. Because grep doesn't ever write to FD 3, nothing visible happens with it.

These happen in sequence: firstly a copy (dup2) of FD 1 is made to FD 3, so FD 3 henceforth points at what FD 1 currently points at, and secondly FD 1 is replaced by a pointer to /dev/null.

The end result is shown in the following diagram:

Diagram of redirections 3>&1 1>/dev/null

Standard error (in pink, FD 2) and FD 3 are to the TTY, and standard output is to /dev/null. FD 3 still points at the TTY because that's where FD 1 pointed when the copy was made. However, the grep command will not try to write anything to FD 3, so the copy never gets any use.

The "copying" is directional: after 3>&1 is processed, anything written to FD 3 will go to where FD 1 pointed at the time of processing. It doesn't "preserve" anything beyond that: if you subsequently redirect FD 1, anything written to it goes to the new place. What you've done is held on to the original destination of FD 1 in case you wanted to use it later. The only redirection that affects where grep's standard output ends up is the one 1>... that explicitly talks about where it goes.

If grep were to write to FD 3, it would appear in the terminal as expected. Because it only outputs to FD 1 normally, all of its actual output is being discarded.

We could make a grep outputting to FD 3 if we wanted:

( grep "..." >&3 )

That will take the regular output on FD 1 and point it at (the newly-created) FD 3 instead. This won't work if you run it directly, because FD 3 doesn't go anywhere, but we can incorporate it into something like this that makes use of it:

( grep "..." >&3 ) 3>&1 1>/dev/null

The command in parentheses outputs to FD 3. The redirections afterwards 1) point FD 3 (which now actually has content on it) at FD 1, and 2) then direct FD 1 away again (which has no effect). The end result is that you'll get the grep "..." output to standard output again, exactly where it would have been without all the fuss.

A practical use of this sort of redirection is something like

cmd 3>&1 1>&2 2>&3 3>&- | foo

which swaps FDs 1 and 2, using FD 3 as a temporary storage spot. It's also sometimes used for shell-script tricks, like faking process substitution in POSIX sh with ( cmd1 | ( cmd2 | ( main_command /dev/fd/3 /dev/fd/4 ) 4<&0 ) 3<&0 ). Otherwise, it's fairly rare that commands innately use any non-standard file descriptor by number (of course many open a file themselves and get one).

Michael Homer
  • 76,565
  • I think I'm close to understanding it. Your answer seems to pass very close to the confusion I stated here. So grep by default does not write to fd 3, but it writes to fd 1 and fd 2, is that it? And I'm assuming every program is like this? How do I make it write to fd 3? – exit_status May 20 '19 at 00:55
  • Get the source code, change it, and recompile. Very few programs use arbitrary file descriptors floating around, and if they did you'd hardly ever see it. – Michael Homer May 20 '19 at 00:59
  • 1
    This will run a grep outputting to FD 3, and then re-send output from there to the TTY so you can see what it said, again redirecting FD 1 away (for no reason this time): ( grep x >&3 ) 3>&1 1>/dev/null – Michael Homer May 20 '19 at 01:00
1

You're redirecting file-descriptor 3 to file-descriptor 1, and you're redirecting 1 to /dev/null. You're not making a copy of anything.

The normal way to copy a file-descriptor is with tee.

Also, I don't know what your context is, but 3 isn't a built-in file descriptor. Presumably you'd already set it up to point to something?

ShapeOfMatter
  • 181
  • I haven't pointed it to anything, the whole thing is in the example. I see here it being described as being duplicated (which I, for better or for worse, understood as making a copy). So I don't see how what you're saying does not contradict this. – exit_status May 20 '19 at 00:23
  • In this source, while talking about 2>&1, the author says that "This one duplicates file descriptor 2 to be a copy of file descriptor 1". – exit_status May 20 '19 at 00:29
  • I understand what you're saying now. You're correct: Fd 3 is now pointing to a copy of the thing fd 1 was pointing to before you redirected fd 1 to /dev/null. So if anything were written to fd 3, yes, it would get printed to the terminal. But you haven't sent anything to fd 3. – ShapeOfMatter May 20 '19 at 00:49
  • But if I were to use 2 instead of 3, it would print to the terminal and I wouldn't have sent anything either (not that I know of, I mean). There's a difference here that escapes me. – exit_status May 20 '19 at 00:52