1

I want to process all changed logfiles from apache and made a simple oneliner:

inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk '{ print }'

This is exactly my start. But if I pipe that output further, I get no output:

inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk '{ print }' | wc -l

Anyway, if I use 'ls', 'tail' (what I really want), there is never output (in another console, running the first statement, I know, there are changes). I tried with adding stdbuf, fflush(), nothing works. I dont understand the problem! I want to keep this onliner into a script, which I can use as a string point. Otherwise, I could (but dont wish to) use xargs. Any help or tips would be fine!

mabra
  • 161
  • So it still doesn't work with the correct quoting in place? – jesse_b Jul 18 '18 at 22:14
  • 1
    turn off buffering (or at least make it line-based) for all the things. https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe – thrig Jul 18 '18 at 22:25
  • Thanks for all hints! It just needs to much time to solve this problem - I am working for hours on that. After applying all recommendations to disable buffering (some of them I tried also bevore) and none worked, I am going back to use xargs. It is even an unsupported debian (squeeze - a new hardware is on my table sinde two years ...). Thanks anyone. – mabra Jul 18 '18 at 23:26
  • 1
    I answered the question, but it's really not clear what your desired result is. The awk command here does nothing at all. wc and tail don't print anything until all data is read. ls doesn't read stdin at all. – jordanm Jul 19 '18 at 01:35
  • you may have a different problem to me, however I found I can get non-buffered output from awk with fflush() -- As: gawk -e '{ prnt ; fflush(); }'. It must have the empty parentheses"()". Alternatively try ... | mawk as: mawk -Winteractive -e '{ print $0; }' | ... – will Sep 12 '19 at 06:18

1 Answers1

2

The reason you do not get output is because wc -l is consuming all of the output. You can use tee in combination with process substitution to get awk to output to the console as well as read by wc:

inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk '{ print }' | tee >(wc -l)

This is still slightly problematic as inotifywait is a command that never exits, but wc -l does not print output until it has read all the data, in this case when the pipe closes. You can do your counting directly in awk:

inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk 'BEGIN { total=0 } { print; total++; print total } END { print "final total: " total }'
jordanm
  • 42,678