22

Apparently I don't know all the output destinations that are available for use. I know about stdout(&1) and stderr (&2). However, after redirecting both descriptors, I sometimes still get some output in my console!

The easiest example I can think of is GNU Parallel; Each time I use it, I see a citation notice. Even when I do &2>1 > file, I still see the notice.

And the same applies to emerge: When I run emerge and there are some problems, some informations aren't printed to stdout nor stdin, since I redirect them and they still get through.

I mostly solve these problems by using script, but I am still wondering what's causing this issue.

MatthewRock
  • 6,986
  • 1
    Please provide a full example. – Kusalananda Jul 15 '16 at 11:13
  • which shell? have a look at http://mywiki.wooledge.org/BashFAQ/055 and https://stackoverflow.com/questions/876239/how-can-i-redirect-and-append-both-stdout-and-stderr-to-a-file-with-bash – Sundeep Jul 15 '16 at 11:22
  • I don't really understand the downvote, but it looks like my case is exactly the FAQ thing. @spasic would you care to turn your comment into an answer? – MatthewRock Jul 15 '16 at 11:24
  • Is &2>1 > file exactly what you typed? It should be >file 2>&1 to do what you want. What you typed will do something completely different (background the main command, to start with!). Order of operation is also important; redirect stdout before stderr. – Stephen Harris Jul 15 '16 at 11:25
  • @StephenHarris see my previous comment - yes, it is, and I suspect that this is excatly the problem. – MatthewRock Jul 15 '16 at 11:26
  • 8
    You won't get them all. A script can always write to /dev/tty. – Satō Katsura Jul 15 '16 at 11:26
  • @MatthewRock tbh, I don't have clear understanding as well.. if your problem has been solved using that FAQ, you can post an answer yourself explaining problem and solution.. with a link to that FAQ – Sundeep Jul 15 '16 at 11:29
  • 1
    As for GNU parallel: mkdir ~/.parallel; touch ~/.parallel/will-cite will disable the annoying message. Alternatively, look around for other implementations of parallel. – Satō Katsura Jul 15 '16 at 11:30
  • Why not do as GNU Parallel suggests? Run 'parallel --citation' once. – Ole Tange Jul 15 '16 at 13:01
  • 2
    @OleTange Because it isn't a problem - I'm asking why something is happening and I'm using parallel as an example. – MatthewRock Jul 15 '16 at 13:55
  • @MatthewRock this is an additional info. I feel like Ole answered to Sato maybe? – Rolf Jul 22 '16 at 05:39

2 Answers2

42

The syntax you used is wrong.

cmd &2>1 >file

will be split down as

cmd &
2>1 >file

This will:

  1. Run cmd as a background job with no redirections
  2. In a separate process (without a command!) will redirect stderr to a file literally called 1 and redirect stdout to file

The syntax you want is:

cmd >file 2>&1

The order of operations is important. This will:

  1. Redirect stdout to file
  2. Redirect stderr to &1 - ie the same filehandle as stdout

The result is that both stderr and stdout will be redirected to file.

In bash, a simpler non-standard (and so I don't recommend it, on portability grounds) syntax of cmd &> file does the same thing.

Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
Stephen Harris
  • 44,540
  • Nice, thanks. The other problem might be /dev/tty, but hopefully this doesn't happen too often(if at all). – MatthewRock Jul 15 '16 at 11:43
  • 5
    If you have the at command on your machine and have privileges to use it, then you can run the command via at now. See the manpage for details. This will run the command via a batch process mechanism and the process will never have a tty to write to. But, in general, I wouldn't worry about this edge case. Typically only processes requiring interaction and deliberately need to display things to users despite redirection will use /dev/tty. – Stephen Harris Jul 15 '16 at 11:48
  • been there, done that – davidbak Jul 15 '16 at 17:59
13

There are two problems.

The first one is that the order matters, the second one is /dev/tty.

Let's use this script as an example script that we want to capture output from:

test.sh:

#!/bin/bash

echo dada
echo edada 1>&2
echo ttdada >/dev/tty

Now let's see the outputs of the commands:

./testmyscript.sh 2>&1 >/dev/null:

edada
ttdada

Because the order of evaluation is from left to right, we first get "redirect stderr to wherever stdout is outputting(so, console output)". Then we get "redirect stdout to /dev/null. We end up with situation like this:

stdout -> /dev/null stderr -> console

So we get it right:

./testmyscript.sh >/dev/null 2>&1

And we get:

ttdada.

Now we do "Redirect stdout to /dev/null", and then "Redirect stderr to where stdout is pointing"(so, /dev/null). Hurray!

However, we still have a problem; program prints to /dev/tty. Now I don't know how to fix this kind of behaviour, so you're most likely going to need script, but hopefully this behaviour won't happen too often.

MatthewRock
  • 6,986
  • how do you, redirect output from /dev/tty using script? – Akash Karnatak Jul 07 '20 at 05:20
  • Yeah, you're right about /dev/tty. I'm having that problem with the partclone.* family of commands, which output timing information to /dev/tty. The --quiet option says it disables it, but that does not seem to work. – 4dummies Jan 11 '21 at 21:46