3

When we run this with a POSIX shell,

$ cmd0 | cmd1

STDOUT of cmd0 is piped to STDIN of cmd1.

Q: On top of this, how can I also pipe STDOUT of cmd1 to STDIN of cmd0?

Is it mandatory to use redirect from/into a named pipe (FIFO) ? I don't like named pipes very much because they occupy some filesystem paths and I need to worry about name collisions. Or do I have to call pipe(2) via C, Python or some general purpose programming languages?

(Both cmd0 and cmd1 do some network I/O, so they won't block each other forever.)

nodakai
  • 419
  • Good to know some *nixes have "bidirectional pipes"... I wasn't imaginative enough to search across the archive with such a term. – nodakai Jul 18 '16 at 05:19

1 Answers1

18

On systems with bi-directional pipes (not Linux), you can do:

cmd0 <&1 | cmd1 >&0

On Linux, you can do:

: | { cmd1 | cmd2; } > /dev/fd/0

That works because on Linux (and Cygwin, but generally not other systems) /dev/fd/x where x is a fd to a pipe (named or not) acts like a named pipe, that is, opening it in read mode gets you the reading end and in write mode gets you the writing end.

With the yash shell and its x>>|y pipeline redirection operator:

{ cmd0 | cmd1; } >>|0

With shells with coproc support:

  • zsh:

       coproc cmd0
   cmd1 <&p >&p

  • ksh

       cmd0 |&
   cmd1 <&p >&p

  • bash4+

       coproc cmd0
   cmd1 <&"${COPROC[0]}" >&"${COPROC[1]}"

Dedicated tool approaches (shamelessly copied from answers on this very similar question):

Using pipexec

pipexec [ A /path/to/cmd0 ] \
        [ B /path/to/cmd1 ] \
        '{A:1>B:0}' '{A:0>B:1}'

Or using dpipe

dpipe cmd0 = cmd1

Or using socat:

socat EXEC:cmd0 EXEC:cmd1,nofork                 # using socketpairs
socat EXEC:cmd0,commtype=pipes EXEC:cmd1,nofork  # using pipes

I don't know about python, but that's also relatively easily done in perl:

perl -e 'pipe STDOUT,STDIN; exec "cmd0 | cmd1"'

You can always also resort to named pipes, but finding unique names for them and some safe directory to create them in, restricting access to them (so other processes can't open them and interfere) and cleanup afterwards become additional problems which are hard to overcome reliably and portably.

In any case, beware of deadlocks!

Stéphane Chazelas
  • 544,893
  • Doesn't the perl -e… example buffers piped data by default? – Kentzo Jan 08 '21 at 21:18
  • 1
    @Kentzo, not sure what you mean, but in any case, perl is not involved at all in the flow of data there, it just creates the pipe and execs the shell which runs cmd0 | cmd1 in the same process, so perl is gone by the time that shell starts, and the output of cmd1 will be connected to the input of cmd0 via that pipe created by perl earlier. – Stéphane Chazelas Jan 08 '21 at 21:23
  • Perl buffers writes to STDOUT by defaults as far as I know. Thus data output by cmd1 may be retained by perl's internals until some threshold is reached before it's piped to STDIN. – Kentzo Jan 08 '21 at 21:31
  • 2
    @Kentzo, no again, perl doesn't read nor write anything here. It is gone (has been replaced by sh) by the time cmd0 and cmd1 are started. You can check with perl -e 'pipe STDOUT,STDIN; exec "ps -f >&2"' or perl -e 'pipe STDOUT,STDIN; exec q((lsof -p "$$" +E -ad0,1 >&2); exit)' to convince yourself. – Stéphane Chazelas Jan 09 '21 at 07:58
  • You can simplify the Linux example to : | { cmd1 | cmd2; } >/dev/fd/0 –  Feb 11 '21 at 23:45
  • @user414777. Thanks. That's much better indeed (also because in ksh/zsh, cmd2 is still not run in a subshell). I've now edited in. – Stéphane Chazelas Feb 12 '21 at 07:32