What is the order that Linux's dynamic linker searches paths in?

Question

This is not a duplicate because this is dealing with a peculiarity I noticed when I use /etc/ld.so.conf.

To get the paths that the dynamic linker searches in for libraries, I run the command ldconfig -v | grep -v "^"$'\t' | sed "s/:$//g". When /etc/ld.so.conf has no paths listed in it. The output from the previous command is

/lib
/usr/lib

I figured that it searches /lib first and then /usr/lib. When I add a new path, such as /usr/local/lib, to /etc/ld.so.conf and then remake /etc/ld.so.cache, the output from ldconfig -v | grep -v "^"$'\t' | sed "s/:$//g" becomes

/usr/local/lib
/lib
/usr/lib

I find this strange because if I am correct that the order that the listed directories are searched in is from top to bottom, then additional directories are searched before /lib and /usr/lib. That the additional directories are searched before the trusted directories is not strange on its own, but when /lib is searched before /usr/lib, that is strange because /bin & /sbin are searched after /usr/bin & /usr/sbin in PATH.

Even if the paths listed by ldconfig -v | grep -Ev "^"$'\t' | sed "s/:$//g" were searched from bottom to top, it would still be a skewed ordering because additional directories would be searched after the trusted ones while /lib would be searched after /usr/lib.

So, what is the order that ld.so searches paths for libraries in? Why is /lib searched before /usr/lib? If it's not, then why are additional directories searched after /lib?

Answer 1

The order is documented in the manual of the dynamic linker, which is ld.so. It is:

1. directories from LD_LIBRARY_PATH;
2. directories from /etc/ld.so.conf;
3. /lib;
4. /usr/lib.

(I'm simplifying a little, see the manual for the full details.)

The order makes sense when you consider that it's the only way to override a library in a default location with a custom library. LD_LIBRARY_PATH is a user setting, it has to come before the others. /etc/ld.so.conf is a local setting, it comes before the operating system default. So as a user, if I want to run a program with a different version of a library, I can run the program with LD_LIBRARY_PATH containing the location of that different library version. And as an administrator, I can put a different version of the library in /usr/local/lib and list /usr/local/lib in /etc/ld.so.conf.

Trust doesn't enter into this. Any directory listed on this search path has to be trusted, because any library could end up being loaded from there. In theory, you could list the library names used by all the programs “requiring more trust” on your system and make sure that all these libraries are present in the “most trusted” directories, and then the “less trusted” directories wouldn't be used if they came after the more trusted directories on the search path, except for the programs “requiring less trust”. But that would be extremely fragile. It would also be pretty pointless: if an attacker can inject a value of LD_LIBRARY_PATH or an element of /etc/ld.so.conf, they surely have a more direct route to executing arbitrary code, such as injecting a value of PATH, of LD_PRELOAD, etc. Trust in the library load path does matter when execution crosses a trust boundary, i.e. when running a program with additional privileges (e.g. setuid/setgid program, or via sudo). What happens in this case is that LD_LIBRARY_PATH is blanked out.

As for /lib vs /usr/lib, it doesn't matter much: they're provided by the same entity (the operating system) and there shouldn't be a library that's present in both. It makes sense to list /lib first because it provides a (very tiny) performance advantage: the most-often-used libraries, especially the libraries used by small basic programs (for which load time is a higher fraction of total running time than large, long-running program), are located in /lib.