6

The following produces nothing in bash:

while true ; do upower -d ; sleep 1 ; done | grep percentage | uniq

I've discovered that it doesn't matter what the last- or even the second to last- program are in this chain. The second-to-last program always produces expected output, and the last always fails to produce anything.

It also doesn't appear to matter if I wrap the while loop in a subshell (via ( while ... done ) ), or wrap everything but the last command in a subshell and pipe into that last command.

I am deeply confused by this behavior.

My gut tells me...

...that there's some sort of I/O deadlocking in the chain created by blocking reads and writes and the fact that the while loop isn't always producing output. But, on the other hand, I've done this kind of thing plenty of times before, with no issue. Besides, if it were that, wouldn't the problem exist between the while loop and the next command? So I'm flummoxed. Will provide bash version information if no one else can reproduce.

In case it isn't immediately obvious...

The point of this line of code is to print every change in battery percentage, but only print new battery levels. It uses polling. I suppose I could get the same behavior with simply running

upower --monitor-detail | grep percentage | uniq

But this is a one-off thing, and I didn't plan on expending more than 5 seconds of thought on this until the above started to fail. At which point it became an interesting problem. Plus, I don't know whether monitor-detail just does polling under the hood, anyway (and I'm not running an strace to check).

EDIT: Apparently, the above --monitor-detail version also doesn't produce anything (or, at least, it seems to. The polling / update frequency is pretty low, so I might just not have waited long enough. Although I know I waited long enough for the original issue). Now I'm very, very confused. I guess I should run that strace after all...

Gilles 'SO- stop being evil'
  • 829,060
Parthian Shot
  • 782
  • try: while true ; do upower -d ; sleep 1 ; done | stdbuf -i0 -o0 -e0 grep percentage | uniq – VaTo Jun 15 '15 at 03:04

3 Answers3

10

You should use grep --line-buffered percentage or else it will take a very long time for the grep stdout buffer to be filled by its output.

Marco d'Itri
  • 488
8

In addition to Marco d'Itri's answer, not every tool supports custom buffering. In case the tool you're running doesn't, or in case you want to run it using a custom buffer size, you can use stdbuf to override the tool's buffering behavior; in this case, for example, to force the output to be line-buffered:

while true ; do upower -d ; sleep 1 ; done | stdbuf -oL grep 'percentage' | uniq

To force the output to be unbuffered:

while true ; do upower -d ; sleep 1 ; done | stdbuf -o0 grep 'percentage' | uniq

To force the output to buffered in chunks of 512 B:

while true ; do upower -d ; sleep 1 ; done | stdbuf -o512 grep 'percentage' | uniq

You can add K, M, G, [...] to specify the output in KB, MB, GB, [...]:

while true ; do upower -d ; sleep 1 ; done | stdbuf -o512K grep 'percentage' | uniq
kos
  • 2,887
2

With any POSIX sed you can achieve line-buffered output w/ the write command.

while  sleep 1
do     upower -d
done | 
sed -n /percentage/w\ /dev/fd/1 |
uniq

...will work on systems supporting the /dev/fd/[num] links (such as practically any linux system).

But you could just do:

while  sleep 1
do     upower -d
done | 
sed '/percentage/!d;H;x
     /^\(.*\)\n\1$/d;g'

The above should do instead - it just packs the functions of uniq and grep together into a single little sed script.

mikeserv
  • 58,310
  • My sed-fu is not up to scratch: would that combined regex wait until it sees a non-duplicate line before outputing the first line of the now previous duplicates? – Mark Hurd Jun 15 '15 at 05:25
  • @MarkHurd - it doesn't wait. It prints the first line - and holds it. All identical lines which follow are held until one comes in which is not and it is printed, and rinse, repeat. So they get printed immediately, but all of the following repeats are removed from output. – mikeserv Jun 15 '15 at 14:01