3

I've got two VMs with arch linux and debian (running under virtual box). Not long ago, it started to take a lot of time connecting to arch VM over ssh. The first time after a while, successive attempts happen quickly. Here's what I see in the arch VM:

debug1: userauth-request for user yuri service ssh-connection method none
debug1: attempt 0 failures 0
debug3: Trying to reverse map address 10.0.2.2.
# ^^^ spends a lot of time after this line
debug2: parse_server_config: config reprocess config len 314
debug2: input_userauth_request: setting up authctxt for yuri
debug1: PAM: initializing for "yuri"
debug1: PAM: setting PAM_RHOST to "10.0.2.2"
debug1: PAM: setting PAM_TTY to "ssh"
debug2: input_userauth_request: try method none
Failed none for yuri from 10.0.2.2 port 35786 ssh2

10.0.2.2 is the default gateway for the VMs. What does this all mean?

P.S. Here's a good answer, thanks to which I was able to make some progress in researching the issue.

x-yuri
  • 3,373
  • If the link is an acceptable and working answer, you should post it by the "Provide Your Own Answer" button. This helps mark the question as answered and makes it easier for others who view your question Title to see there is a working answer. Editing the question and adding the answer is okay, but it makes it harder for people to see that an Answer has been given. – 0xSheepdog May 27 '14 at 17:55
  • 1
    @0xSheepdog, I believe the OP referred to the link mentioned and still not able to resolve the issue. That's why he has posted a question referring to that link in his question. – Ramesh May 27 '14 at 17:59
  • Thank you @Ramesh that point never occurred to me. Sorry x-yuri if I misunderstood you. – 0xSheepdog May 27 '14 at 18:00
  • It sounds to me like "UseDNS no" on the server should avoid the delay. (I often notice the delay when using ssh locally in the absence of working DNS). Is that the case or not? – sourcejedi May 27 '14 at 18:27
  • 1
    @0xSheepdog I added the link, because it helped me to find out more about the issue. But the issue is yet to be solved. – x-yuri May 28 '14 at 09:57
  • @sourcejedl I'm not sure if I understand you correctly. I don't have DNS server installed on the host machine. Also, both VMs had UseDNS = yes, but it took too long to connect to only arch linux VM. Could you explain what UseDNS really means? I didn't get it from the man page. Preferably on the example of my setup. – x-yuri May 28 '14 at 11:46
  • @sourcejedi It seems you were right about "UseDNS no" solving the issue. Would you care answering this question? Details on what "UseDNS" has to do with this are appreciated. – x-yuri May 29 '14 at 09:46

2 Answers2

2

At this stage, there are many possibilities that might be causing a slow connection. From this link, one issue that I see is related to encryption.

Finally I know what is the issue.

my /home/user dir is encrypted and therefore ssh wasn't able to access it unless I am logged in.

Also one more possibility from here is related to the SELinux service.

Yes, SELinux is likely the cause. The .ssh dir is probably mislabeled. Look at /var/log/audit/audit.log. It should be labeled ssh_home_t. Check with ls -laZ. Run restorecon -r -vv /root/.ssh if need be.

Also from this link, I also see the PAM authentication might be a reason for slow SSH.

Edit your "/etc/ssh/ssh_config" and comment out these lines:

GSSAPIAuthentication yes
GSSAPIDelegateCredentials no
Ramesh
  • 39,297
  • Sorry, I didn't mention I checked my GSSAPI* settings. As to the first case, my home dir is not encrypted and server didn't refuse my key. And I don't have /var/log/audit directory on both host and guest machine. I'm now thinking if it's caused by UseDNS = yes, though it's that way on both VMs. And the one connecting too slowly is with arch linux. Reasoning on how exactly UseDNS affects the issue is appreciated. – x-yuri May 28 '14 at 11:40
2

My guess is, that the problem are DNS requests timeouts. Try ... turning off the UseDNS option on the server you are connecting to.

The server log pretty much confirmed this (so well done for posting it :).

debug3: Trying to reverse map address 10.0.2.2.
# ^^^ spends a lot of time after this line

Looking at man sshd_config:

 UseDNS  Specifies whether sshd(8) should look up the remote host name and
         check that the resolved host name for the remote IP address maps
         back to the very same IP address.  The default is “yes”.

Note that this is pretty much pointless. The check does not kill the connection. If it doesn't pass, it just logs a finger-wagging message:

Jul 9 05:43:00 brick sshd[18971]: Address 200.41.233.234 maps to host234.advance.com.ar, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!

which only serves to generate scary false positives for users with incompetent ISPs.

OK, so how can this cause a delay?

If the DNS server doesn't respond immediately, the client will tend to wait & retry. (In case a DNS packet was delayed or lost due to network congestion). If the DNS server isn't responding at all, the client will eventually give up. E.g. dig on my system retries for 15 seconds. (It uses more specific dns library code, but the principle is the same).

$ time dig invalid @192.0.2.1

; <<>> DiG 9.9.4-P2-RedHat-9.9.4-12.P2.fc20 <<>> invalid @192.0.2.1
;; global options: +cmd
;; connection timed out; no servers could be reached

real    0m15.020s
user    0m0.010s
sys         0m0.011s

So the problem is the reverse lookup doesn't get a response. You can run the same lookup on the server yourself manually, and you should see the same delay as ssh. getent hosts 10.0.2.2.

sourcejedi
  • 50,249
  • Indeed, getent hosts 10.0.2.2 and dig -x 10.0.2.2 both take 5 second to execute and fail to make a reverse lookup. But I wanted to clarify about incompetent ISP or probably me. The difference between my VMs was that I specified a hostname for my host machine in /etc/hosts in debian VM, as opposed to arch linux VM. But I believe I did that just because I needed to access host machine once or more. So what is the proper /etc/hosts setup in the context of the issue? Do I possibly have to run DNS server locally? – x-yuri Jun 02 '14 at 11:13
  • 1
    Ok. The hosts file is used for reverse lookups as well as normal lookups, so that probably is what causes the difference. Are you saying the VMs are using an ISP DNS server, i.e. if you look /etc/resolv.conf it does not show an IP address in one of your own network ranges like 10.0.2.2 ? If so, then yes, it is ISP incompetence. (And by implication they're unlikely to be competent to fix it on advice of a single consumer. So you get to work around it yourself. In which case, e.g. installing unbound as a recursive resolver on the host machine would be a good idea in general). – sourcejedi Jun 03 '14 at 06:42
  • 1
    Ah, I missed two things. 1. the check SSH is running is not expected to cause a delay even if the reverse mapping is missing. The DNS should immediately return an empty answer, and the client would not retry the query. Your problem isn't like the particular ISP incompetence I referred to in the answer, if that's what your phrasing meant. – sourcejedi Jun 03 '14 at 07:05
  • 1
  • Note that dig does not refer to /etc/hosts. If /etc/resolv.conf lists the same nameserver on both VMs, I would expect dig -x 10.0.2.2 to timeout on the debian VM as well. And the same would apply to the host system i.e. the one VirtualBox is running on.
  • – sourcejedi Jun 03 '14 at 07:08
  • 1
    One suspicion is that VirtualBox almost certainly has an integrated DNS server, and it could be buggy somehow. Another would be an old buggy DNS relay software (I've seen some badness on outdated DNSMasq) is running on your home router. – sourcejedi Jun 03 '14 at 07:11
  • First, resolv.conf has nameserver 10.0.2.3 on both VMs. Then, dig -x 10.0.2.2 timeouts on both VMs, but not on host machine. Host machine has 'nameserver 127.0.1.1in/etc/resolv.conf. Host OS isubuntu`. As to incompetence, I didn't refer to anything in particular. I just would like to know who are to blame and which way things are supposed to be. – x-yuri Jun 03 '14 at 14:13
  • 1
    If you haven't already, I would also double-check one or two normal lookups on the VMs. Since it works correctly on the host, I don't think it's a problem with your ISP (or your local router). I'd say both your VMs are fine, it's VirtualBox that's broken (especially as they provide specific packages for each revision of Ubuntu). Whether that's a misconfiguration on your specific system, or a bug that needs fixing in VirtualBox? I don't know, sorry. Presumably there's a VirtualBox community forum you can sign up and get flamed at :-). – sourcejedi Jun 03 '14 at 17:08
  • Surprisingly, dig google.com occasionally takes about 5 seconds to execute too. And now I understand why ping didn't show bad results (judging by time value), but it took quite a while for each line to appear. It must also has to do with DNS. So it must be either router or ISP... – x-yuri Jun 03 '14 at 17:31
  • 1
    Yep, that's right about ping. – sourcejedi Jun 04 '14 at 18:24