3

I'm try to pipe chat log files to some speech output, and want to remove timestamps with cut and remove special characters with tr. Unfortunately, cut seems to halt when using it with tail -f in combination with tr:

//works
$ tail /path/to/chatlogs | cut -b18- 
test
test

//works, too
$ tail /path/to/chatlogs | tr -C -d  '[:alnum:][:cntrl:] '
test
test

// does not work
$ tail -f /path/to/chatlogs | tr -C -d  '[:alnum:][:cntrl:] ' | cut -b18-
//no output

Why? How can I work around this?

It even hangs when piping two times into cut:

$ tail -f file | cut -b5- | cut -b5-
//no output
cweiske
  • 521
  • hmmm... your works example seems to be identical to your does not work example. if there is a problem with tail -f it might have to with its checking for new input only every 60 seconds or so. if it were to try to open() the file all the time it would eat a lot of resources. try while <file; do :; done here <file is some regular, readable file, in the terminal and have a look at top in another terminal while it runs. or : >file; until read v; do : ; done <file. – mikeserv Dec 16 '15 at 11:47
  • 3
    The second tail command waits for additional input because of the follow (-f) switch and never exits. – Lambert Dec 16 '15 at 11:52
  • What Lambert said; why are you using -f if you don't want it to wait for more output? – wurtel Dec 16 '15 at 11:55

2 Answers2

10

The syntax you use only processes new input on /path/to/chatlogs. Try to run the command and log a new entry in /path/to/chatlogs and see what the output is or try:

tail -1000f /path/to/chatlogs | tr -C -d  '[:alnum:][:cntrl:] ' | cut -b18-

to have the last 1000 lines also processed.

The 'hanging' part is actually the tail process which is waiting for further input to be passed through the pipe.

To disable buffering from the tr command use:

tail -f /path/to/chatlogs | stdbuf -i0 -oL tr -C -d  '[:alnum:][:cntrl:] ' | cut -b18-

The above is used from Turn off buffering in pipe

Community
  • 1
Lambert
  • 12,680
0

When using multiple pipes, I'm running into stdio buffering issues.

The problem is described in detail at http://www.pixelbeat.org/programming/stdio_buffering/ and there does not seem to be a solution.

cweiske
  • 521