How to make a redirection loop

Question

This one is mostly for academic reasons. ;-)

Consider the following Bash statement:

foo | bar

This executes two processes, foo and bar, such that the standard-out of foo is connected to the standard-in of bar. So far, so good.

Is there a way to invoke foo and bar so that we also have a connection in the opposite direction?

foo.stdout -> bar.stdin
foo.stdin  <- bar.stdout

As I understand it, Bash calls pipe(2) to create a pair of FDs connected to each other, which can then be used to replace stdin / stdout in the respective child processes. At the kernel level, I see no reason why you couldn't call pipe(2) twice and arrange the circular arrangement depicted above.

But I can't figure out the shell syntax to actually do that.

So far, the best idea I can come up with is to use named pipes, or maybe to launch a subshell and do some crazy FD-to-FD redirecting somehow...

Answer 1

In bash, you can do it with either a coproc (bash has lousy support for multiple coprocs, but you only need one here):

#!/bin/bash
set -e
coproc { while read -r line; do echo "$BASHPID read: $line";  done; }
i=0; while :; do
    echo "$BASHPID writing>> $i"
    echo $i >&"${COPROC[1]}"
    read -r line <&"${COPROC[0]}"
    echo "$BASHPID coproc produced>> $line"
    i=$((i+1))
done

or named pipes (those work in simple POSIX shells too):

#!/bin/bash
set -e
trap 'rm -rf "$tmpd"' EXIT
tmpd=$(mktemp -d)

mkfifo "$tmpd/p0" "$tmpd/p1"
exec 3<>"$tmpd/p0"
exec 4<>"$tmpd/p1"
rm -rf "$tmpd"

( while read -r line; do echo "$BASHPID read: $line";  done; ) <&3 >&4  &
i=0; while :; do
    echo "$BASHPID writing>> $i" 
    echo $i >&3
    read -r line <&4
    echo "$BASHPID coproc produced>> $line"
    i=$((i+1))
done 

They both might seem ugly if you're not used to fd-handling in shells.

Also, due to the effect pipes have on scheduling (writing to pipes with a full pipe buffer blocks you as does reading from a pipe with an empty one), you might get deadlocked with certain read/write patterns.

Outputs of the above two examples might look like this:

32435 writing>> 0
32435 coproc produced>> 32441 read: 0
32435 writing>> 1
32435 coproc produced>> 32441 read: 1
32435 writing>> 2
32435 coproc produced>> 32441 read: 2
32435 writing>> 3
32435 coproc produced>> 32441 read: 3
32435 writing>> 4
32435 coproc produced>> 32441 read: 4

Answer 2

I don't think you can do it with a pipe (|), but it's easy to do with asynchronous processes writing and reading each other.

This ksh93 script (bash version further down) starts up two while-loops which toss a number in-between them, adding one to the number in each transaction:

while read data; do
    print $(( data + 1 ))
done |&

print -p 1

while read -p data; do
    print $(( data + 1 ))
done >&p

1. The first loop is started as a co-process. It waits for input on its read.
2. The number 1 is given to the read of the co-process to start the whole thing off.
3. The second loop is started doing the same thing as the first loop, but not as a co-process.

Running it with xtrace enabled (and with a modified script that sets PS4, the trace prompt, to "& " for the first loop, and to "> " for the second loop, just to show what's what):

$ ksh -x script.sh
& PS4='& '
> PS4='> '
> print -p 1
> 1>& p
> read -p data
& read data
& print 2
& read data
> print 3
> read -p data
& print 4
> print 5
> read -p data
& read data
& print 6
& read data
> print 7
> read -p data
(etc.)

bash also does co-processes (look for the coproc builtin), but I have less familiarity with those. You can probably do this in a shell without co-processes too.

EDIT: Something like this for bash:

coproc while read data; do
    echo $(( data + 1 ))
done

echo 1 >&${COPROC[1]}

while read -u ${COPROC[0]} data; do
    echo $(( data + 1 ))
done >&${COPROC[1]}

Running it (with modified trace prompt as above):

$ bash -x script.sh
+ PS4='& '
& PS4='> '
> echo 1
> read -u 63 data
& read data
& echo 2
> echo 3
> read -u 63 data
& read data
& echo 4
> echo 5
> read -u 63 data
& read data
& echo 6
> echo 7
> read -u 63 data
(etc.)

Answer 3

For more complicated file-descriptor redirection trees, there's also pipexec, which has made it into the Debian Repository.

Truly, it's a limitation of the syntax in BASH and in Bourne-like shells in general.