6

Suppose I have a executable which for which I want to log STDOUT and STDERR to separate files, like so:

python write.py > out 2> err

When I use this command with GNU timeout, my out file is always empty. Why does this happen, and how can I fix this?

timeout 5s python write.py > out 2> err

Example write.py:

#!/bin/bash
import sys, time

i = 0
while True:    
    print i
    print >> sys.stderr, i
    time.sleep(1)
    i += 1
jaynp
  • 663
  • 3
  • 9
  • 18

1 Answers1

7
$ timeout 5s python write.py > out 2> err
$ ls -l out err
-rw-r--r-- 1 yeti yeti 10 Apr  6 08:12 err
-rw-r--r-- 1 yeti yeti  0 Apr  6 08:12 out
$ timeout 5s python -u write.py > out 2> err
$ ls -l out err
-rw-r--r-- 1 yeti yeti 10 Apr  6 08:13 err
-rw-r--r-- 1 yeti yeti 10 Apr  6 08:13 out
$ cmp err out && echo same
same
$ cat out 
0
1
2
3
4

python uses buffered writes on stdout but not on stderr. So everything written to sys.stderr is written immediately but stuff for sys.stdout is kept in a buffer until the buffer is full to minimise write operations. Within 5 seconds, the buffer does not fill enough to get written even once and timeout terminates the python interpreter.

Adding the -u option to python's invocation, writes to stdout will be unbuffered too.

You can force the same behaviour for a Python program by setting the environment variable PYTHONUNBUFFERED. For other programs you can use unbuffer from the expect package or stdbuf from GNU coreutils (see Turn off buffering in pipe).

Community
  • 1