Some errors are not logged, after redirecting both stderr and stdout to a file

Question

In a Linux server, I run a program and log its stdout and stderr outputs to a file by:

myprogram.py > logfile 2>&1

All the outputs to stdout and stderr are logged, except that some errors in the following are not but appear on the terminal. I wonder why not all the errors are logged?

*** glibc detected *** xgboost: munmap_chunk(): invalid pointer: 0x0000000000bc0a10 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75f4e)[0x7f089d19ef4e]
xxx/gcc/4.8.1/lib64/libgomp.so.1(+0x8d72)[0x7f089dc40d72]
xgboost[0x40b1de]
xgboost[0x41585d]
xgboost[0x403f81]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f089d147d5d]
xgboost[0x403fd9]
======= Memory map: ========
00400000-00437000 r-xp 00000000 00:13 2044979336                         xxx/xgboost/v0.32/bin/xgboost
00437000-00439000 rw-p 00037000 00:13 2044979336                         xxx/xgboost/v0.32/bin/xgboost
00bae000-00d3c000 rw-p 00000000 00:00 0                                  [heap]
7f0897791000-7f0897792000 ---p 00000000 00:00 0
7f0897792000-7f0898192000 rw-p 00000000 00:00 0
7f0898192000-7f0898193000 ---p 00000000 00:00 0
7f0898193000-7f0898b93000 rw-p 00000000 00:00 0
7f0898b93000-7f0898b94000 ---p 00000000 00:00 0
7f0898b94000-7f0899594000 rw-p 00000000 00:00 0
7f0899594000-7f0899595000 ---p 00000000 00:00 0
7f0899595000-7f0899f95000 rw-p 00000000 00:00 0
7f0899f95000-7f0899f96000 ---p 00000000 00:00 0
7f0899f96000-7f089a996000 rw-p 00000000 00:00 0
7f089a996000-7f089a997000 ---p 00000000 00:00 0
7f089a997000-7f089b397000 rw-p 00000000 00:00 0
7f089b397000-7f089b398000 ---p 00000000 00:00 0
7f089b398000-7f089c599000 rw-p 00000000 00:00 0
7f089cd1d000-7f089cd24000 r-xp 00000000 fc:01 414519                     /lib64/librt-2.12.so
7f089cd24000-7f089cf23000 ---p 00007000 fc:01 414519                     /lib64/librt-2.12.so
7f089cf23000-7f089cf24000 r--p 00006000 fc:01 414519                     /lib64/librt-2.12.so
7f089cf24000-7f089cf25000 rw-p 00007000 fc:01 414519                     /lib64/librt-2.12.so
7f089cf25000-7f089cf27000 r-xp 00000000 fc:01 414530                     /lib64/libdl-2.12.so
7f089cf27000-7f089d127000 ---p 00002000 fc:01 414530                     /lib64/libdl-2.12.so
7f089d127000-7f089d128000 r--p 00002000 fc:01 414530                     /lib64/libdl-2.12.so
7f089d128000-7f089d129000 rw-p 00003000 fc:01 414530                     /lib64/libdl-2.12.so
7f089d129000-7f089d2b3000 r-xp 00000000 fc:01 414538                     /lib64/libc-2.12.so
7f089d2b3000-7f089d4b3000 ---p 0018a000 fc:01 414538                     /lib64/libc-2.12.so
7f089d4b3000-7f089d4b7000 r--p 0018a000 fc:01 414538                     /lib64/libc-2.12.so
7f089d4b7000-7f089d4b8000 rw-p 0018e000 fc:01 414538                     /lib64/libc-2.12.so
7f089d4b8000-7f089d4bd000 rw-p 00000000 00:00 0
7f089d4bd000-7f089d4d4000 r-xp 00000000 fc:01 393354                     /lib64/libpthread-2.12.so
7f089d4d4000-7f089d6d4000 ---p 00017000 fc:01 393354                     /lib64/libpthread-2.12.so
7f089d6d4000-7f089d6d5000 r--p 00017000 fc:01 393354                     /lib64/libpthread-2.12.so
7f089d6d5000-7f089d6d6000 rw-p 00018000 fc:01 393354                     /lib64/libpthread-2.12.so
7f089d6d6000-7f089d6da000 rw-p 00000000 00:00 0
7f089d6da000-7f089d75d000 r-xp 00000000 fc:01 414534                     /lib64/libm-2.12.so
7f089d75d000-7f089d95c000 ---p 00083000 fc:01 414534                     /lib64/libm-2.12.so
7f089d95c000-7f089d95d000 r--p 00082000 fc:01 414534                     /lib64/libm-2.12.so
7f089d95d000-7f089d95e000 rw-p 00083000 fc:01 414534                     /lib64/libm-2.12.so
7f089d95e000-7f089d95f000 r-xp 00000000 fc:01 393364                     /lib64/snoopy.so
7f089d95f000-7f089db5e000 ---p 00001000 fc:01 393364                     /lib64/snoopy.so
7f089db5e000-7f089db5f000 rw-p 00000000 fc:01 393364                     /lib64/snoopy.so
7f089db5f000-7f089db7f000 r-xp 00000000 fc:01 414524                     /lib64/ld-2.12.so
7f089dc1c000-7f089dc22000 rw-p 00000000 00:00 0
7f089dc22000-7f089dc37000 r-xp 00000000 00:13 2044935698                 xxx/gcc/4.8.1/lib64/libgcc_s.so.1
7f089dc37000-7f089dc38000 rw-p 00015000 00:13 2044935698                 xxx/gcc/4.8.1/lib64/libgcc_s.so.1
7f089dc38000-7f089dc45000 r-xp 00000000 00:13 2044935716                 xxx/gcc/4.8.1/lib64/libgomp.so.1.0.0
7f089dc45000-7f089dc46000 rw-p 0000d000 00:13 2044935716                 xxx/gcc/4.8.1/lib64/libgomp.so.1.0.0
7f089dc74000-7f089dc76000 rw-p 00000000 00:00 0
7f089dc76000-7f089dd5a000 r-xp 00000000 00:13 2044935770                 xxx/gcc/4.8.1/lib64/libstdc++.so.6.0.18
7f089dd5a000-7f089dd5b000 ---p 000e4000 00:13 2044935770                 xxx/gcc/4.8.1/lib64/libstdc++.so.6.0.18
7f089dd5b000-7f089dd63000 r--p 000e4000 00:13 2044935770                 xxx/gcc/4.8.1/lib64/libstdc++.so.6.0.18
7f089dd63000-7f089dd65000 rw-p 000ec000 00:13 2044935770                 xxx/gcc/4.8.1/lib64/libstdc++.so.6.0.18
7f089dd65000-7f089dd7c000 rw-p 00000000 00:00 0
7f089dd7c000-7f089dd7e000 rw-p 00000000 00:00 0
7f089dd7e000-7f089dd7f000 r--p 0001f000 fc:01 414524                     /lib64/ld-2.12.so
7f089dd7f000-7f089dd80000 rw-p 00020000 fc:01 414524                     /lib64/ld-2.12.so
7f089dd80000-7f089dd81000 rw-p 00000000 00:00 0
7ffcf1520000-7ffcf156f000 rw-p 00000000 00:00 0                          [stack]
7ffcf15ec000-7ffcf15ed000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

Which line of those you posted does not get logged? Random or? — Torxed, Jan 12 '16 at 18:28
All the lines posted are not logged. Aren't they supposed to be on stderr and then redirected to the file? — Tim, Jan 12 '16 at 18:32
See if this helps in the mean while: http://unix.stackexchange.com/q/80629/26852 — Torxed, Jan 12 '16 at 18:38
I am not sure. I am running other's program. The output shows libgomp.so.1, the openmp library. — Tim, Jan 12 '16 at 18:41
Do you have write permission on the file? whoami; ls -l logfile? — Torxed, Jan 12 '16 at 18:44
Sorry for the stupid questions, they are quite trivial but occasionally needed since the info is lacking in the questions. — Torxed, Jan 12 '16 at 18:46

score 2 · Answer 1 · answered Jan 12 '16 at 19:05

2

Your program just writes on descriptor other then 1 or 2.

Consider the following script:

#!/bin/bash

echo test >$(tty)

Now lets run it redirecting both stdin and stderr to /dev/null:

$ ./script >/dev/null 2>&1
test

As you see we internally redirected output to the terminal, so outside redirection to /dev/null got nothing to do. In your case it seems like some debug module was doing similar thing.

answered Jan 12 '16 at 19:05

jimmij

47,140

thanks. i think it is possible but unlikely the python program writes to descriptor other than 1 or 2, and I don't how a python program does that. Any other possibilities? – Tim Jan 12 '16 at 19:14
@Tim I doubt it is python itself - from the output one can conclude that something went wrong during compilation by gcc. Python probably only started this compilation process, but the error comes from other place, but hard to guess the details. – jimmij Jan 12 '16 at 20:00
http://stackoverflow.com/a/27797579/4957508 appears to imply that the Python script calls something xgboost which then has a glibc error, and glibc defaults to printing ot the tty. – Jeff Schaller Jan 12 '16 at 20:01
see also http://stackoverflow.com/a/7086177/4957508 for another possible work-around – Jeff Schaller Jan 12 '16 at 20:02

Some errors are not logged, after redirecting both stderr and stdout to a file

1 Answers1