6

So, I understand that by using 'tee' I can redirect the output of a command such as 'ping' to stdout as well as a file.

For example:

> ping google.com | tee somefile

This would display the ping statistics on terminal and write them to 'somefile'.

Now, if I wish to modify the output from 'ping', I can use 'cut' in such a way:

> ping google.com | cut -d' ' -f 1

But, if I wish to use all the three commands together, I get a no output on stdout and an empty file.

> ping google.com | cut -d' ' -f 1 | tee somefile

What am I doing wrong? Is there a better way to do this? Somehow I feel I am not using 'tee' properly. Any insight would be appreciated.

I'm using bash shell, if this is relevant.

Aniruddha Deshpande
  • 63
  • 2
    You're simply not waiting long enough; eventually the pipe buffer fills up and dipslays the output. – Jeff Schaller Feb 01 '16 at 19:22
  • 6
    see http://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe – Jeff Schaller Feb 01 '16 at 19:24
  • I thought I wasn't waiting long enough, but after a couple of minutes, I realized that that wasn't the problem. Thanks for the link though, it addresses the issue I am facing precisely. But, the solutions provided involve installing a new package. I was hoping for a more generic solution without installing external packages. – Aniruddha Deshpande Feb 01 '16 at 19:42
  • My bad, the inbuilt stdbuf works perfectly. – Aniruddha Deshpande Feb 02 '16 at 09:56

1 Answers1

12

Your pipe commands, as a non-terminal destination, are buffering your output. It will show up eventually, but only when quite a lot of output builds up or the ping command exits.

You can use ping -c 5 google.com to set a specific number of packets to be sent and then ping will exit. Your output comes back and the pipes should work as expected.

Edit: another workaround uses stdbuf to avoid pipe buffering and awk to avoid some internal cut buffering and lets the ping run continuously:

ping www.google.com | stdbuf --output=0 awk '{print $1}' | tee /tmp/file
Steve Bonds
  • 1,276
  • 1
    You are correct, I tried it with count specified, and it did display the result on stdout and in the file. However, if I don't specify the count and kill the process by using Ctrl-C, would I still see the output? Or would all of it be lost because of the interrupt? – Aniruddha Deshpande Feb 01 '16 at 19:48
  • 1
    If you hit ^C from the terminal, the interrupt signal would be sent to the last command you ran-- the tee so you would not get the output. Now you could be sneaky and find the PID of the ping command itself then use kill -INT <PID> and ONLY the ping command would stop and the output would be sent to cut and tee, letting you see it. – Steve Bonds Feb 01 '16 at 19:53
  • Ah! Killing the long_running_process (ping) did the job. Thanks! However it does seem sort of a workaround. The link provided by @Jeff above describes the need to unbuffer the output. But the only built-in command stdbuf doesn't work with tee :/. Are there better ways to format and (print the output and write to file) ? – Aniruddha Deshpande Feb 01 '16 at 20:22
  • 1
    On my linux test host, there seems to be something "special" about cut in particular. I tried the same test using awk '{print $1}' and the no-output problem went away. Perhaps cut is buffering things internally too? ping www.google.com | stdbuf --output=0 awk '{print $1}' | tee /tmp/file Note that the output to "file" will still be buffered. – Steve Bonds Feb 01 '16 at 20:33
  • 1
    Thanks Steve, I think I wasn't using stdbuf correctly. Your solution works as expected. Interestingly, when I tried awk, my output was still buffered, it was displayed/written in chunks. However when I used cut it wasn't the case. The output was displayed line by line. So to me it seems that awk is buffering whereas for you it is the opposite. Strange I must say. Or maybe I'm missing something crucial.. – Aniruddha Deshpande Feb 02 '16 at 09:53
  • The problem with command internal buffering is you never know what will work where. For a portable solution, whipping up a quick Python or Perl script to read, parse, and output using buffering of your own choice might be the best option. If you only need it to work on that one system, you're already done! – Steve Bonds Feb 02 '16 at 16:20