Why is the behavior of `command 1>file.txt 2>file.txt` different from `command 1>file.txt 2>&1`?

Question

When you want to redirect both stdout and stderr to the same file, you can do it by using command 1>file.txt 2>&1, orcommand &>file.txt. But why is the behavior of command 1>file.txt 2>file.txt different from the above two commands?

The following is a verification command.

$ cat redirect.sh
#!/bin/bash

{ echo -e "output\noutput" && echo -e "error" 1>&2; } 1>file.txt 2>&1
{ echo -e "output\noutput" && echo -e "error" 1>&2; } 1>file1.txt 2>file1.txt
{ echo -e "error" 1>&2 && echo -e "output\noutput"; } 1>file2.txt 2>file2.txt
{ echo -e "output" && echo -e "error\nerror" 1>&2; } 1>file3.txt 2>file3.txt
{ echo -e "error\nerror" 1>&2 && echo -e "output"; } 1>file4.txt 2>file4.txt

$ ./redirect.sh

$ echo "---file.txt---"; cat file.txt;\
echo "---file1.txt---"; cat file1.txt; \
echo "---file2.txt---"; cat file2.txt; \
echo "---file3.txt---"; cat file3.txt; \
echo "---file4.txt----"; cat file4.txt;
 ---file.txt---
output
output
error
---file1.txt---
error

output
---file2.txt---
output
output
---file3.txt---
error
error
---file4.txt----
output
rror

As far as the results are seen, it looks like that the second echo string overwrites the first echo string when you run command 1>file.txt 2>file.txt, but I do not know why it will. (Is there a reference somewhere?)

Answer 1

You need to know two things:

• An open file descriptor known to the application-mode side of a process references an internal kernel object known as a file description, which is an instance of an open file. There can be multiple file descriptions per file, and multiple file descriptors sharing a file description.
• The current file position is an attribute of a file description. So if multiple file descriptors map to a single file description, they all share the same current file position, and a change to the file position enacted using one such file descriptor affects all of the other such file descriptors.

  Such changes are enacted by processes calling the read()/readv(), write()/writev(), lseek(), and suchlike system calls. The echo command calls write()/writev() of course.

So what happens is this:

• command 1>file.txt 2>&1 only creates one file description, because the shell only opens a file once. The shell makes both the standard output and standard error file descriptors map to that single file description. It duplicates standard output onto standard error. So a write via either file descriptor will move the shared current file position: each write goes after the previous write the common file description. And as you can see the results of the echo commands do not overwrite one another.
• command 1>file.txt 2>file.txt creates two file descriptions, because the shell opens the same file twice, in response to the two explicit redirections. The standard output and standard error file descriptors map to two different file descriptions, which then in turn map to the same single file. The two file descriptions have entirely independent current file positions, and each write goes immediately the previous write on the same file description. And as you can see the result is that what is written via one can overwrite what is written via the other, in various different ways according to what order you execute the writes in.

Further reading

• What is an open file description?
• What exactly is a file offset in lsof output?

Answer 2

Using > tells it to overwrite the file. Since you have stdout and stderr writing to the file in two different operations, the last one to write will overwrite the first.

You can do:

command 1>>file.txt 2>>file.txt

or

command &>file.txt Only bash v4 and above.

>> tells it to append the file so it won't replace the previous operations' output.

&> is just an easier way to write 2>&1