6

I transfered /etc/* files from one Linux server to another and now I have a strange error. Both are Debian Squeeze.

When I run a command, e.g. ssh, I get an error:

bash: /usr/bin/ssh: No such file or directory

It looks like the binary exists:

-rwxr-xr-x 1 root root 358756 Sep 21 14:30 /usr/bin/ssh

Bash also autocompletes the command when I type ss and press Tab. I tried reinstalling, purging etc but it didn't solve anything. There is probably something cached under /etc/ but I don't know what excatly.

jasonwryan
  • 73,126
Lauris
  • 163
  • 1
  • 6
  • 1
    What's the output of ldd /usr/bin/ssh? It might depend on a library (.so file) that doesn't exist. – mrb Sep 28 '12 at 14:32
  • @mrb the output is "not a dynamic executable". – Lauris Sep 28 '12 at 14:33
  • How did you copy the files? What does file /usr/bin/ssh tell you? – Mat Sep 28 '12 at 14:38
  • @Mat Copied like this: scp -r /etc/* user@remote:/etc/ and the output of the "file" command is /usr/bin/ssh: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped – Lauris Sep 28 '12 at 14:42
  • If that helps I also installed the same packages using dpkg --set-selections & apt-get dselect-upgrade. – Lauris Sep 28 '12 at 14:45
  • 3
    So file says it uses shared libs, and ldd doesn't... something's screwed up. Try readelf -l /usr/bin/ssh and look for a "Requesting interpreter" line (or error). – Mat Sep 28 '12 at 14:48
  • @Mat ldd actually runs the program. So of course it doesn't work. (BTW: ldd is a shell script; less \which ldd`` and you can see how it works) – derobert Sep 28 '12 at 14:51
  • 1
    @mrb Same comment about ldd actually running the program to you, too. Its important that people know this, less they try ldd when analyzing an untrusted binary. – derobert Sep 28 '12 at 14:53
  • @derobert: it doesn't actually run the program, it invokes the runtime linker and does pretty much all the runtime linking with debug options, but it doesn't invoke the program's main. – Mat Sep 28 '12 at 14:54
  • @Mat: strace -f ldd \which cat` |& grep execve:… [pid 8490] execve("/bin/cat", ["cat"], [/* 46 vars */] <unfinished ...>` – derobert Sep 28 '12 at 14:55
  • @derobert: try ldd /usr/bin/vi or ldd /sbin/halt - this won't run vi or halt. The runtime linker gets debug options from the environment, set up from the ldd script, and stops short from actually running the code. – Mat Sep 28 '12 at 14:57
  • @Mat yes, the glibc start code (still part of the executable) doesn't invoke the program's main. But the point is that we know (from the error posted) that execve is failing. And so its not surprising that ldd does as well (as that strace shows, it actually does an execve of the program) – derobert Sep 28 '12 at 15:00
  • @derobert: I was reacting to your statement that ldd runs the executable, which would make ldd bloody dangerous. We both agree, I think, that ldd not managing to list dependencies and file saying it is dynamically linked, is a problem, right? – Mat Sep 28 '12 at 15:02
  • Is the binary the same architecture as the rest of the userspace (i386)? – jordanm Sep 28 '12 at 15:08
  • @Mat It is bloody dangerous to run on an executable you don't trust (see, e.g., http://man7.org/linux/man-pages/man1/ldd.1.html), though some versions of ldd are patched against this. The disagreement is a problem, but (for example) is exactly what is expected (I just tested this) if the executable is the wrong architecture. – derobert Sep 28 '12 at 15:09
  • You only transferred /etc? Weird that it broke ssh, but ls still works. Were the two machines the same architecture (or at least running the same, i.e., both running i386)? – derobert Sep 28 '12 at 15:15
  • 1
    ldd doesn't call _start/main as normal when called on a normal dynamically-linked executable, but an executable can be crafted to run arbitrary code on being examined with some versions of ldd. – Random832 Sep 28 '12 at 16:12
  • 1
    @strangefishinthetown And what happens when you run /usr/bin/ssh (i.e. full path)? Is the file actually executable? – jw013 Sep 28 '12 at 16:44
  • 4
    @strangefishinthetown You might try removing /etc/ld.so.cache and checking the contents of /etc/ld.so.conf and /etc/ld.so.conf.d. Run ldconfig to rebuild the cache afterwards. – Jim Paris Sep 28 '12 at 17:36

1 Answers1

4

I think Jim Paris's comment is right on the mark. You can get “no such file or directory” even if the specified file exists, if the dynamic loader for that file does not exist. A common case is trying to run a binary from a different architecture that the kernel supports but not the user-land system — typically a 32-bit binary on a 64-bit system that doesn't have 32-bit libraries installed. See Getting "Not found" message when running a 32-bit binary on a 64-bit system for a more detailed explanation of that case.

Here, it's possible that /usr/bin/ssh uses a library that is installed in a different place on the new system. Since you copied the dynamic linker's cache file /etc/ld.so.cache, the dynamic linker is looking for that library in a place where it doesn't exist. Run ldconfig to update the cache.

It's also possible that you'd enabled prelinking. If so, remove /etc/prelink.cache.

Community
  • 1
Gilles 'SO- stop being evil'
  • 829,060