8

I want to merge two unzipped files f1 and f2 in one command, like 

paste (zcat f1.gz) (zcat f2.gz).

What is the right syntax?

AdminBee
  • 22,803
user196711
  • 181
  • It would be helpful if you could post an example of the (uncompressed) input and output you expect; you can edit your question to add this information. As it stands, it's unclear to me how you want to "merge" the two files; is it with the actual command paste, or otherwise? – dhag Apr 14 '15 at 14:12
  • "I want to merge two unzipped files" --> zipped? – JJoao Apr 14 '15 at 14:39

2 Answers2

14

almost there...

paste <(zcat f1.gz) <(zcat f2.gz)
JJoao
  • 12,170
  • 1
  • 23
  • 45
-1

Great answer: Thanks a lot.

I've used it for pasting three columns, each one in a compressed file.

Just for sharing : source 3200 redhat machines, 250 millions of points aprox per day...

This is not exactly the way we get all the files locally but catch the idea ( we use ansible in the real task)

for i in $(cat list_of_hostnames.txt)
do
 sadf -U -- -A <file from yesterday> | pigz -9 > host_date_file.tsv.gz
done

Asume we get all such files in the working directory:

pigz -cd *.tsv.gz| sed -E 's/\t/\n/g' | split --numeric-suffixes=1 -nr/6 - kk.

After such command, you get six compressed files called kk.01 to kk.06 corresponding to hostmane, interval, timestamp in seconds from epoch, device, metric and value.

Just for saving space :

rm kk.02

( I don't need interval having timestamps) and then

pigz -9 kk.0[13456]

and now is when I use : 

paste <(zcat kk.05.gz ) <(zcat kk.01.gz) <(zcat kk.04.gz) <(zcat kk.06.gz) | grep '%idle' | pigz -9 > metric_host_device_value.tsv.gz
AdminBee
  • 22,803
user3220948
  • 11