12

For some commands, it is possible to specify certain input as either stdin or a command line argument.

Specifically, suppose command can take stdin input and a filename as command line argument, and command < myfile, cat myfile | command and command myfile can produce the same result.

For example,

When the command is sed:

sed s/day/night/ <myfile >new   
sed s/day/night/ myfile >new    
cat myfile | sed s/day/night/ >new

When the command is cat:

cat < myfile
cat myfile
  1. I was wondering if there are some general rules about their performances, i.e. which one of them is usually the most efficient, and which the least?
  2. Is redirection always better than pipe?
Caleb
  • 70,105
Tim
  • 101,790
  • 1
    I wish everyone asking these (duplicated) questions go and write their own shell from scratch as an exercise. – alex Jul 22 '11 at 04:53
  • 1
    please don't use "Thanks!" in your questions. Vote answers up to express your gratefulness. – alex Jul 22 '11 at 04:54
  • @Alex: If this is a dupe, please link to the duplicate and we'll work on closing it. Typically you would refrain from answering a question you know is a duplicate and flag it for moderator attention. – Caleb Jul 22 '11 at 07:39
  • 1
    @alex: Where can I learn how to write my own shell? – Tim Jul 22 '11 at 12:37
  • @Caleb: I'm certain this was asked like 2 or 3 times in the past month, just don't have the link handy :-p – alex Jul 22 '11 at 13:10
  • @Tim: ugh, read a book on C programming for nix? Also I've found info libc (glibc-doc package in Debian and derivatives) to be invaluable resource for understanding file and process model of a typical nix system. – alex Jul 22 '11 at 13:15
  • 1
    @alex: There have been 3 or 3 questions in the past month on similar issues but I don't see one that's an exact duplicate. Don't comment on OPs asking duplicate questions if you can't actually point to a duplicate. I think this one has a unique angle that makes it different from the others even if some of the answers overlap. – Caleb Jul 22 '11 at 14:23

2 Answers2

9

The cat file | command syntax is considered a Useless Use of Cat. Of all your options, it takes a performance hit because it has to spawn another process in the kernel. However insignificant this may turn out to be in the big picture, it's overhead the other forms don't have. This has been covered on questions such as: Should I care about unnecessary cats?

Between the other two forms there are virtually no performance differences. STDIN is a special file node that the process has to open and read just like any other. Passing a file name instead of STDIN just makes it open a different file.

The difference would be in what features / flexibility you are looking for.

  • Passing the file name to the program would mean the input file was seekable. This may or may not matter to the program but some operations can be sped up if the stream is seekable.
  • Knowing the actual input file allows your program to potentially write to it. For example sed -i for in-place editing. (Note: since this has to create a new file behind the scenes it's not a performance gain over other redirects but it is a convenience step.)
  • Using shell redirects gives you the ability to concatenate multiple files or even use process redirection. sed [exp] < file1 file2 or even sed [exp] < <(grep command). Details of this use case can be found on this question: Process substitution and pipe
Caleb
  • 70,105
  • Process substitution should work without requiring you to pipe the result; sed [exp] < <(grep command) will work just fine as sed [exp] <(grep command) (since <(grep command) creates a named temporary file for the length of the command which sed is perfectly capable of opening on its own without shell assistance). – ShadowRanger Jul 25 '18 at 17:03
2

  1. Given that command file just opens the file and from then on works like if it was stdin, there's little difference. With shell redirection you just open the file beforehand (shell does,) as opposed to command binary itself.

  2. If we're talking about cat file | command vs. command <file, then the latter is preferred. You aren't going to notice significant performance difference between the two, but the former is unnecessarily complicated (extra process and shared memory buffer for the pipe, with limited throughput.) Also, you cannot seek (change the file pointer position arbitrarily) in a pipe, while you can in an ordinary file. Some commands may use more efficient algorithm when seek-ing in the input file is possible.

alex
  • 7,223
  • I would say that command file is preferred over command < file, because the command might do some sort of non-sequential access. – user606723 Jul 22 '11 at 14:12
  • And what would stop it from doing so with <file? Your point is valid for using the input file name to derive output file name tough, e.g.: gzip file produces file.gz. – alex Jul 22 '11 at 15:07
  • maybe I don't understand how redirection works internally. Lets say we redirect a 12GB movie into mplayer/vlc, and then we skip to the end. What exactly would happen in this case? – user606723 Jul 22 '11 at 15:14
  • 1
    Shell opens the file and forks a sub-process, which inherits the file descriptor. The forked process closes stdin and calls dup on the opened file descriptor, so it replace the old stdin (which was some sort of tty in most cases.) From the movie player point of view there's no difference between that and opening the file by it's name in the player itself. The file descriptor is seekable in both scenarios, so when we skip to the end there's no user-detectable difference. – alex Jul 22 '11 at 16:20