4

For some reason, I will not have live output when piping the command output of e.g. SSH to sed:

ssh someuser@somehost 2>&1 | sed -e "s/\[32//g" | tee logging

I suspected that the output dit not have newlines, but when i remove the sed command and run:

wc -l logging

It returns 6, which is the correct number of returns. Does anybody have an idea what i am missing here?

[EDIT] I completely forgot to mention the following: running

ssh someuser@somehost 2>&1 | sed -e "s/\[32//g"

returns all values no problem, however, as soon as i add tee into the mix, i won't get outputs until I hit ^C... Not even standard output redirection works (ssh someuser@somehost 2>&1 | sed -e "s/[32//g" > file). Just ssh with tee also works fine.

Matthias Cotting
  • 55
  • Possibly related: Turn off buffering in pipe – steeldriver Nov 17 '18 at 03:03
  • It's the same old buffering problem -- on linux, you can work it around by replacing the sed ... with stdbuf -oL sed .... Though notice that you won't have echoing and prompting in the interactive ssh if you use sed at all (with or without tee). If your case, I wonder why you aren't simply using script(1). –  Nov 17 '18 at 16:12
  • or if sed is GNU sed, use sed -u. –  Nov 17 '18 at 16:19

3 Answers3

7

The stream editor, sed, buffers, disable with '-u' or '--unbuffered'.

tail -f some/log_file | sed -u 's/foo//g' | grep bar
tbh
  • 86
2

When you run ssh user@somehost without an explicit command, you're requesting ssh to start an interactive shell on the remote machine.

An interactive shell (such as bash) typically wants to have a terminal available, since it will use terminal commands to implement improved experience when editing the command line and browsing the history. (Terminal commands allow full-screen control.)

But ssh will only allocate a terminal by default (also called a pseudo-terminal) if its standard output is connected to a terminal.

If you simply run ssh user@somehost from a terminal program (such as gnome-terminal, rxvt, xterm, etc.), then its standard output will be a terminal, so ssh will create a pseudo-terminal and the remote interactive shell will behave nicely.

If you pipe ssh through something (anything), like ssh user@somehost | cat, then its standard output will be a pipe (and not the terminal), so ssh will not create a terminal and that might make the remote interactive shell misbehave.

A possible solution is to force ssh to create a pseudo-terminal, by passing it the -t option, like ssh -t user@somehost | cat, which might help. (Also, you might need a double option -tt to force allocation of the pseudo-terminal.)

Another possibility, if you're running ssh mainly because you're interested on a specific command, is to run the specific command from the ssh command line, such as ssh user@somehost mycommand | cat. If you run a specific command, an interactive shell does not get involved, in which case having a terminal available is likely not going to cause any issues.

filbranden
  • 21,751
  • 4
  • 63
  • 86
0

I would try the same command with the following addition:

ssh someuser@somehost 2>&1 | sed -e "s/\[32//g" | tee --output-error=warn logging

With that, you should be able to see if there is an error in writing to stdout. Let me know if that helps.

You could install expect, then try:

unbuffer ssh someuser@somehost 2>&1 | sed -e "s/\[32//g" | tee logging
Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
Michael Prokopec
  • 2,220