0

The following chain of commands are to

  1. ping with datestamp (UNIX),

  2. convert the UNIX datestamp to more human-readable format, and

  3. output to the terminal and a log file.

ping -D localhost 2>&1 | sed 's/^\[\([0-9]*\.[0-9]*\)\]\(.*$\)/echo "[`date -d @\1 +"%Y-%m-%d %H:%M:%S"`] \2"/e' | tee -a ping.log

Problem is that as written, the output seems to be buffered in chunks of almost a minute or ~50 lines, unlike the usual second-by-second and line-by-line output from ping.

What is causing the buffering and how can it be avoided?

adatum
  • 1,141

1 Answers1

2

If it's available in your system, use unbuffer. Should be as easy as:

unbuffer ping -D localhost 2>&1 | unbuffer sed 's/^\[\([0-9]*\.[0-9]*\)\]\(.*$\)/echo "[`date -d @\1 +"%Y-%m-%d %H:%M:%S"`] \2"/e' | tee -a ping.log
msb
  • 2,654
  • sed -u looks like a simpler and cleaner solution for now, but do you know when it would be preferable to use unbuffer? And which commands would need it? – adatum Sep 11 '18 at 01:10
  • unbuffer is generic and works with any command. I've used it extensively with various programs including awk, perl and others. It's needed whenever piping is buffered automatically. – msb Sep 11 '18 at 01:13
  • Good to know. When is piping buffered automatically? It confused me that the piped commands, depending on how many and which commands, sometimes get buffered and sometimes don't. I initially thought it was a problem with the code. – adatum Sep 11 '18 at 04:22
  • 1
    That's a good question, idk myself. I've read around about stdbuf which is another program analogous to unbuffer that controls stdin/stdout/stderr streams. I guess it's something that either the shell or the kernel controls, but I'd have to do some research on it to know who controls. All I know is that it's external to the program. I had my own scripts with autoflush and everything, and they'd still get buffered, which was really puzzling and how I found unbuffer. – msb Sep 11 '18 at 04:29