4

I am running the following git clone command through sudo and bash and I want to redirect STDOUT to a log file:

% sudo -u test_user bash -c "git clone https://github.com/scrooloose/nerdtree.git
/home/test_user/.vim/bundle/nerdtree >> /var/log/build_scripts.log"

What is happening is the STDOUT is continuing to be sent to the terminal. i.e.

Cloning into 'nerdtree'...
remote: Counting objects: 3689, done.
[...]
Checking connectivity... done.

I'm guessing the problem has something to do with the fact that sudo is forking a new process then bash is forking another, as demonstrated here:

% sudo -u test_user bash -c "{ git clone https://github.com/scrooloose/nerdtree.git
/home/test_user/.vim/bundle/nerdtree >> /var/log/build_scripts.log; ps f -g$$; }"  

  PID TTY      STAT   TIME COMMAND
 6556 pts/25   Ss     0:02 /usr/bin/zsh
 3005 pts/25   S+     0:00  \_ sudo -u test_user bash -c { git clone https://github.com/scrooloo
 3006 pts/25   S+     0:00      \_ bash -c { git clone https://github.com/scrooloose/nerdtree.
 3009 pts/25   R+     0:00          \_ ps f -g6556

I've tried

  • running this in a script and using exec >> /var/log/build_script.log before the command
  • wrapping the command in a function, then calling and redirecting the functions output

But I think these redirections are only applying to the parent and the child processes are defaulting to sending STDOUT to the /dev/tty/25 of their parent causing output to continue to the terminal.
How can I redirect the STDOUT of this command?

terdon
  • 242,166
the_velour_fog
  • 12,270

2 Answers2

11

The messages you mention are not printed to standard output but to standard error. So, to capture them, you either need to redirect standard error instead of standard output:

sudo -u user bash -c "git clone https://github.com/foo.git ~/foo 2>> log"

Or both STDERR and STDOUT:

sudo -u user bash -c "git clone https://github.com/foo.git ~/foo >> log 2>&1"

With bash, you can also use &>> for this:

sudo -u user bash -c "git clone https://github.com/foo.git ~/foo &>> log"

The csh, tcsh, zsh equivalent being >>& ((t)csh don't support 2>&1 so it's the only way there):

sudo -u user csh -c "git clone https://github.com/foo.git ~/foo >>& log"

In fish

sudo -u user fish -c "git clone https://github.com/foo.git ~/foo >> log ^&1"

For more on the different types of redirection operators, see What are the shell's control and redirection operators?

Now, in the specific case of git, there's another issue. Like a few other programs, git can detect that its output is being redirected and stops printing progress reports if so. This is probably because the reports are intended to be seen live and include \r which can be a problem when saved in a file. To get around this, use:

       --progress
       Progress status is reported on the standard error stream by default
       when it is attached to a terminal, unless -q is specified. This
       flag forces progress status even if the standard error stream is
       not directed to a terminal.

And:

sudo -u user bash -c "git clone --progress https://github.com/foo.git ~/foo >> log 2>&1"

If you want to both see the output as it comes and save to a file, use tee:

sudo -u user bash -c "git clone --progress https://github.com/foo.git ~/foo 2>&1 | 
    tee -a log
Community
  • 1
terdon
  • 242,166
  • thanks, I'm testing now, I need to append so I'm using 2>>&1 but bash is returning bash: -c: line 0: syntax error near unexpected token &' – the_velour_fog Apr 27 '16 at 08:47
  • When I run the command from @skwllsp answer, i.e. using &>> most of the output is lost. If git clone was sending to STDERR I would have thought these messages would have continued to the terminal when using &>> but they seem to disappear? – the_velour_fog Apr 27 '16 at 08:51
  • @the_velour_fog you can't use 2>>&1. Just use sudo .. git ... >> log 2>&1 that appends. However, it looks like git detects that its output is being redirected and stops printing the progress report. You can see the same behavior if you just run git clone https://github.com/scrooloose/nerdtree.git 2>err directly. The progress reports are no longer printed. I haven't been able to figure out a way to trick git into printing even when redirecting. – terdon Apr 27 '16 at 09:59
  • yes im discovering this weird behaviour of git clone just now. to restore the output use the --progress option as defined in the man page. apparently git clone will quite itself when piped etc. – the_velour_fog Apr 27 '16 at 10:03
  • @the_velour_fog yes, many programs behave differently when their output is being piped. For example, grep has the --color=auto option won't print color if it is being redirected. This is usually a good thing. – terdon Apr 27 '16 at 10:05
  • yes grep --auto turns off the color character sequences intended for a terminal but git clones "put everything through stderr" loses the ability to selectively redirect output . the only solution I can think of is to send everything to the terminal and just live with the noisy output so I don't miss error messages – the_velour_fog Apr 27 '16 at 10:14
  • @the_velour_fog no, you can save it to a file with the --progress option, See updated answer. – terdon Apr 27 '16 at 10:16
1
git clone https://github.com/scrooloose/nerdtree.git
/home/test_user/.vim/bundle/nerdtree &>> /var/log/build_scripts.log
  • thanks, this is partially working, its sending the first line to log_file but then remaining output it lost - I think @terdon may have explained whats happening, testing now... – the_velour_fog Apr 27 '16 at 08:42