10

I know how to combine the result of different command 

paste -t',' <(commanda) <(commandb)

I know pipe same input to different command

cat myfile | tee >(commanda) >(commandb)

Now how to combine these command? So that I can do

cat myfile | tee >(commanda) >(commandb) | paste -t',' resulta resultb

Say I have a file

myfile:

1
2
3
4

I want to make a new file 

1 4 2
2 3 4
3 2 6
4 1 8

I used 

cat myfile | tee >(tac) >(awk '{print $1*2}') | paste

would gives me result vertically, where I really want paste them in horizontal order.

Gilles 'SO- stop being evil'
  • 829,060
user40129
  • 341

4 Answers4

8

When you tee to multiple process substitutions, you're not guaranteed to get the output in any particular order, so you'd better stick with

paste -t',' <(commanda < file) <(commandb < file)

Assuming cat myfile stands for some expensive pipeline, I think you'll have to store the output, either in a file or a variable:

output=$( some expensive pipeline )
paste -t',' <(commanda <<< "$output") <(commandb <<< "$output")

Using your example:

output=$( seq 4 )
paste -d' ' <(cat <<<"$output") <(tac <<<"$output") <(awk '$1*=2' <<<"$output")

1 4 2
2 3 4
3 2 6
4 1 8

Another thought: FIFOs, and a single pipeline

mkfifo resulta resultb
seq 4 | tee  >(tac > resulta) >(awk '$1*=2' > resultb) | paste -d ' ' - resulta resultb
rm resulta resultb

1 4 2
2 3 4
3 2 6
4 1 8
glenn jackman
  • 85,964
4

The yash shell has unique features (pipeline redirection and process redirection) that make that easier there:

cat myfile | (
  exec 3>>|4
  tee /dev/fd/5 5>(commanda >&3 3>&-) 3>&- |
    commandb 3>&- |
    paste -d , /dev/fd/4 - 3>&-
)

3>>|4 (pipeline redirection) creates a pipe where the writing end is on fd 3 and the reading end on fd 4.

3>(commanda>&3) is process redirection, a bit like ksh/zsh/bash process substitution but just does the redirection and doesn't substitute with the /dev/fd/n. ksh's >(cmd) is more or less the same as yash's n>(cmd) /dev/fd/n (there n is a file descriptor chosen by ksh on which you have no control).

Stéphane Chazelas
  • 544,893
3

With zsh:

pee() (
  n=0 close_in= close_out= inputs=() outputs=()
  merge_command=$1; shift
  for cmd do
    eval "coproc $cmd $close_in $close_out"

    exec {i}<&p {o}>&p
    inputs+=($i) outputs+=($o)
    eval i$n=$i o$n=$o
    close_in+=" {i$n}<&-" close_out+=" {o$n}>&-"
    ((n++))
  done
  coproc :
  read -p
  eval tee /dev/fd/$^outputs $close_in "> /dev/null &
    " exec $merge_command /dev/fd/$^inputs $close_out
)

Then use as:

$ echo abcd | pee 'paste -d,' 'tr a A' 'tr b B' 'tr c C'
Abcd,aBcd,abCd

That's adapted from this other question where you'll find some detailed explanations and hints at the limitations (beware of deadlocks!).

Community
  • 1
Stéphane Chazelas
  • 544,893
0

For your particular example there should be no need for paste and the rest. It is often true that when we encounter a limit with the standard toolset it is because what we want to do one way can be done another. Such as:

set 1 2 3 4
while [ "$#" -gt 0 ]
do    echo "$1" "$#" "$(($1*2))"
shift;done

...which prints...

1 4 2
2 3 4
3 2 6
4 1 8

You can get a file with contents like you mention into your shell "$@" array like...

set -f; IFS='
'; set -- $(cat)

And to validate the arg values in a loop like the one above you can change the initial test a bit...

while { [ "$1" -eq "${1-1}" ] ;} 2>&"$((2+!$#))"
do    echo "$1" "$#" "$(($1*2))"
shift;done  3>/dev/null >outfile

...which prints an error to stderr only if a line read in with set -- $(cat) contains a line which does not consist entirely of a single integer.

mikeserv
  • 58,310