136

I have ServerAliveInterval and in case of few machines also ClientAliveInterval set to 540 in SSH client/server configuration files (I suppose setting it to more than that would not be a good idea). I work with many SSH sessions which currently freeze after a few minutes.

How can I fix it? What I want is to have a session to not freeze at all, so that if I open a session at 8 and don't use it for 4 hours, for example, to still use it again at 12 without having to log-in again.

syntagma
  • 12,311

10 Answers10

167

The changes you've made in /etc/ssh/ssh_config and /etc/ssh/sshd_config are correct but will still not have any effect.

To get your configuration working, make these configuration changes on the client:

/etc/ssh/ssh_config

Host *
ServerAliveInterval 100

ServerAliveInterval The client will send a null packet to the server every 100 seconds to keep the connection alive

NULL packet Is sent by the server to the client. The same packet is sent by the client to the server. A TCP NULL packet does not contain any controlling flag like SYN, ACK, FIN etc. because the server does not require a reply from the client. The NULL packet is described here: https://www.rfc-editor.org/rfc/rfc6592

Then configuring the sshd part on the server.

/etc/ssh/sshd_config

ClientAliveInterval 60
TCPKeepAlive yes
ClientAliveCountMax 10000

ClientAliveInterval The server will wait 60 seconds before sending a null packet to the client to keep the connection alive

TCPKeepAlive Is there to ensure that certain firewalls don't drop idle connections.

ClientAliveCountMax Server will send alive messages to the client even though it has not received any message back from the client.

Finally restart the ssh server

service ssh restart or service sshd restart depending on what system you are on.

  • I think a high ServerAliveCountMax is also needed in order for this to be as reliable as possible. And if both ServerAliveInterval and ClientAliveInterval are set low enough, I don't think there will be any need for TCPKeepAlive. Additionally, if there are any middleboxes, they can still lose state even with all of the previously mentioned settings configured exactly correct. It may help to use MPTCP (if both client and server support it). – kasperd May 04 '15 at 20:28
  • It's not clear at the start whether you mean config on the client side or server side (eventually I conclude, client side). Then at ServerAliveInterval description, you say "client will send... to the server", but in the next paragraph "NULL packet is sent by the server to the client". I find these a bit confusing. – Craig McQueen Jun 11 '17 at 07:23
  • @CraigMcQueen I've updated my answer and hope this clears things up. The null packet is sent by the client to the server using this directive ServerAliveInterval . The server sends the same packet to the client using ClientAliveInterval. – Valentin Bajrami Jun 12 '17 at 09:38
  • 1
    After adding to the server I get: /etc/ssh/ssh_config: line 57: Bad configuration option: clientaliveinterval /etc/ssh/ssh_config: line 59: Bad configuration option: clientalivecountmax – ajthinking Jul 29 '17 at 15:57
  • 6
    @Anders You get the error because ClientAliveInterval and ClientAliveCountMax are ssh server options, thus meant for sshd_config and not ssh_config – Valentin Bajrami Jul 30 '17 at 08:11
  • I would love to write a single script that would set these settings for client and server on Debian, Centos, Amazon Linux, and macOS. – Bruno Bronosky Apr 02 '19 at 03:44
  • @BrunoBronosky You actually can create your own .rpm package that does this. In the .rpm package you can include these 2 files and deploy them by installing the package. For debian you could do the same although I'm not so into debian packaging. – Valentin Bajrami Apr 02 '19 at 07:01
  • Link to an April fool's joke incorporated into a seriously looking answer :))) – pabouk - Ukraine stay strong Apr 10 '19 at 08:12
  • @ValentinBajrami, I made chnages to /et/ssh/ssh_config and after executing source /etc/ssh/ssh_config, I got error No command 'Host' found, did you mean: Command 'most' from package 'most' (universe) Command 'host' from package 'knot-host' (universe) Command 'host' from package 'bind9-host' (main) Host: command not found SendEnv: command not found HashKnownHosts: command not found GSSAPIAuthentication: command not found GSSAPIDelegateCredentials: command not found ServerAliveInterval: command not found. What could be wrong. I'm on ubuntu 16.04. – CKM Aug 13 '19 at 11:37
  • @chandresh You cannot source /etc/ssh/ssh_config because source is a shell built in and does not work like that. The shell thinks you are trying to execute the lines found in the file. Why are you trying to source the file in question? – Valentin Bajrami Aug 13 '19 at 13:12
  • I was sourcing to bring the updates in the file to activate. Does not it need to be sourced like we do for /etc/environment? – CKM Aug 14 '19 at 09:07
  • 1
    @chandresh you don't have to. After modifying ssh_config the new sessions will read that file. The /etc/environment file is a different thing which has the format of VAR="value" without spaces thus VAR = "value" would be invalid. Sourcing /etc/ssh/ssh_config that way will i.e read: Port 22 or Host * which will treat these as commands – Valentin Bajrami Aug 14 '19 at 12:38
  • I see. Thanks.. – CKM Aug 14 '19 at 12:40
  • I had a error with ClientAliveInterval 60 on the client side. – DPalharini Sep 27 '22 at 17:45
16

Personal suggestion: use screen on the remote host; it will manage to keep your connection alive for as long as it stays active in a terminal.

Here's what I typically add to /etc/screenrc for quick identification of my screen sessions:

hardstatus alwayslastline
hardstatus string "%{= kG}[ %{G}%H %{g}][ %{=kw}%?%-Lw%?%{r}(%{W}%n*%f%t%?(%u)%?%{r})%{w}%?%+Lw%?%?%= %{g}][%{B}%Y-%m-%d %{W}%c %{g}]"
defscrollback 8192

EDIT : Hints.

The hardstatus string will show a bottom status line such as this one: Screen session example with three open tabs

The scrollback buffer is also extended to 8192 lines instead of the usual 1000-1500 (depending on the distribution).

  • Thanks, two questions: 1. Can you explain the options in you screenrc file? How do I configure the system the launch screen/tmux right after I login? – syntagma May 04 '15 at 09:12
  • 14
    Just to clarify, screen does not keep your ssh connection alive. It runs processes in a virtual terminal independent of the user terminal you connect to via ssh so the process doesn't rely on your connection. You will still lose your ssh connection until it's resolved. – iyrin May 04 '15 at 09:52
  • 1
    By "keep the connection alive" I essentially mean "prevents the connection from idling", which has the same result. If there's no input for 4 hours, SSH will go idle and eventually close the connection, all the child processes gone, too. –  May 04 '15 at 11:46
  • 5
    As much as I loved screen, I advise new users to start with tmux. – dotancohen May 05 '15 at 07:10
7

If the problem is a hibernated laptop or a less-than-prefect network connection, I'd recommend using mosh which runs over ssh and allows automatic reconnecting.

From the website:

Mosh (mobile shell)

Remote terminal application that allows roaming, supports intermittent connectivity, and provides intelligent local echo and line editing of user keystrokes.

Mosh is a replacement for SSH. It's more robust and responsive, especially over Wi-Fi, cellular, and long-distance links.

Mosh is free software, available for GNU/Linux, BSD, macOS, Solaris, Android, Chrome, and iOS.

In combination with tmux (or the older screen), this allows me to connect via ssh to a server from my laptop and stay connected for days even when changing wifi connections and surviving mobile data drop-outs.

Tom Hale
  • 30,455
2

With OpenSSH:

You need to enable

TCPKeepAlive yes

both in your client ssh_config (e.g. /etc/ssh/ssh_config or in ~/.ssh/config) and your destination SSH server running OpenSSH (e.g. /etc/ssh/sshd_config).

So every time your connection idles, OpenSSH sends some dummy packet to your destination host...

  • This can also cause the connection to go down The default is ''yes'' (to send TCP keepalive messages), and the server will notice if the network goes down or the client host crashes. ...this means that connections will die if the route is down temporarily, and some people find it annoying. http://linux.die.net/man/5/sshd_config – iyrin May 04 '15 at 08:55
  • Honestly, but "[…] network goes down […] client host crashes […] route is down […]" are severe errors no SSH configuration can catch.

    The main issue here are idle SSH sessions, not network failures. Therefore to sum it up:

    • enable TCPKeepAlive yes in both server and client.
    • set ClientAliveInterval on the server
    • set IdleTimeout on the server
    • set ClientAliveCountMax

    That should do the trick...

    – Martin Allert May 04 '15 at 09:48
1

Check your sshd configuration on the host machine /etc/sshd_config for the IdleTimeout setting.

IdleTimeout time
Sets idle timeout limit to time in seconds (s or nothing afternumber), 
in minutes (m), in hours (h), in days (d), or in weeks (w).If the 
connection have been idle (all channels) for that long time thechild 
process is killed with SIGHUP, and connection is closed down.
iyrin
  • 1,895
1

A little hack I use if it's not possible to use screen, or edit the hosts ssh server settings, or any of the good and proper suggestions above...

Just run top whenever you are going to leave the session alone.

Generally that'll keep it alive indefinitely, although, as others have mentioned, that might not always be wise vis-a-vis security!

1

For me, the SSH connection used to hang up.

The below solution worked for me. Now the SSH connection always stays live.

From the server:


  1. Edit the file at /etc/ssh/sshd_config
$ sudo vi /etc/ssh/sshd_config
  1. Add or enable this line in the file.
ClientAliveInterval 60
  1. Save the file and restart the sshd service
$ sudo service sshd restart

From the local machine/client:


  1. Edit the file at /etc/ssh/ssh_config
$ sudo vi /etc/ssh/ssh_config
  1. Add this line to the file.
ServerAliveInterval 60
  1. Save the file

About these Parameters :

ServerAliveInterval: number of seconds that the client will wait before sending a null packet to the server (to keep the connection alive).

ClientAliveInterval: number of seconds that the server will wait before sending a null packet to the client (to keep the connection alive).

Important: Whenever any modification is performed on a Production instance, please ensure to create a backup.

1

I had a similar problem:

  • ssh connection remained ESTABLISHED
  • terminal freezes and un-freezes repeatedly
  • packets are visible in SEND-Q while terminal freezes
  • freeze time about 2-4 minutes
  • un-frozen for about 30 seconds

Turns out the firmware of my router was outdated!

  • fixed issue with firmware update
Leevi L
  • 131
0

In my case problem was in large MTU size. You can change MTU on router if you using NAT, but I change MTU on server:

sudo /sbin/ifconfig eth0 mtu 1036
sudo /etc/init.d/networking restart
Vasin Yuriy
  • 101
  • 2
0

My SSH started closing consistently after a minute of being open and I think it was because of a network adapter setting I changed in order to stop my PC from waking up from sleep. My PC had been waking up from sleep for no reason so I found this article on how to stop it and it worked. The article said to go into the Device Manager, find the Ethernet or Wi-Fi adapter in question under Network Adapters, right-click on it, choose Properties, and open the Power Management tab and uncheck the Allow This Device to Wake the Computer.

Alternatively, it said you can keep that box checked, and check Only Allow a Magic Packet to Wake the Computer. I did that and later my SSH would consistently close down after a minute so I believe that was the culprit. So make sure that you don't have that setting on your network adapter changed from the default. Might have to restart your computer for any changes you make to take effect.