5

I want to run a task (which takes quite a long time) remotely in Ubuntu Linux Bash via nohup and redirect stdout to a file, e.g.:

nohup task > out.txt &

From time to time I would like to check the progress of the task by looking into the output file:

cat out.txt

I haven't yet figured out how to do that because the file's content is shown as empty as long as the file is opened for writing by Bash.

So my question is: Is there any linux command that allows a 'shared' stdout redirection?

Gilles 'SO- stop being evil'
  • 829,060
Wildkeiler
  • 51
  • 1
    A side note - the file will have some content before it is closed for writing. The amount of text appearing in this file is dependent on the filesystem used and configuration parameters - e.g. for ext4 with default options it should be updated after every 4096 bytes written. – Paweł Rumian Jul 16 '14 at 14:46

2 Answers2

3

the file's content is shown as empty as long as the file is opened for writing by Bash.

That's not exactly what's happening. What's happening is that the task buffers its output — it is accumulated in memory for a while, then written to the file in chunks. The in-memory buffer has a fixed size of a few kilobytes. This is done for performance, as each file write has a relatively high overhead. So you won't see your program's output at the beginning, but you will see it come piece by piece. This has nothing to do with bash.

If you want to see the output immediately, you can start your program with unbuffer (from the expect software package). You trade performance for immediacy.

A second issue is that cat will show the output that has already been emitted, and exit as soon as it reaches the end of the file. If you want to keep reading output when it's appended, use tail's follow mode.

nohup unbuffer task >out.txt &
tail -n +1 -f out.txt

Press Ctrl+C to exit tail. Instead of tail, you can use less and its follow mode (press F; press Ctrl+C to stop reading).

Community
  • 1
Gilles 'SO- stop being evil'
  • 829,060
  • Very nice elaborate answer. I like the follow mode of tail and unbuffer works perfectly - also on OS X. So thanks a lot for that nice advice! – Wildkeiler Jul 18 '14 at 20:54
2

Take a look at tee command, here are some examples with it.

The following command (with the help of tee command) writes the output both to the screen (stdout) and to the file.

$ ls | tee file
fduff
  • 5,035
  • Thanks a lot for the replies. I've realised that there is some error in the program that I wanted to run. Using stdout as usual, it will be shown immediately on the screen. But when redirecting it to a file it didn't. What I did is changing the source code of the program and as soon as the cout is used instead of printf the output was put into the file immediately. This is a behaviour I didn't anticipate. Maybe anyone knows the reason for that?

    So $ nohup task > out.txt & works perfectly for my purpose. Nevertheless it's good to refresh the usage of good old tee

     – Wildkeiler Jul 16 '14 at 15:31
  • So the answer can be found here: http://stackoverflow.com/questions/499879/problem-redirecting-a-c-program-output-in-bash – Wildkeiler Jul 16 '14 at 15:39
  • @Wildkeiler I'm betting that in your cout version you used endl as well. endl requests a buffer flush in addition to writing a newline. – godlygeek Jul 16 '14 at 20:42
  • Indeed endl not cout solved it. – Wildkeiler Jul 18 '14 at 20:55