3

I am compiling a model using make. The model has a Makefile that connects the source code with dependent libraries via flags that look like -L/lib1 -L/lib2. But when I try to run that model, it fails unless I also ensure the environmental variable

export LD_LIBRARY_PATH=/lib1:/lib2

and points to the exact same libraries. This seems redundant to me.

What could be going on under the hood here? Why do I effectively have to specify the location of the libraries before compilation and before execution?

This might be a silly question; I'm not very experienced compiling to machine code, usually just use scripting languages.

Luke Davis
  • 334

1 Answers1

6

Although everyone uses compilation in the colloquial sense of turning source code into an executable it's technically a single step in a rather long pipeline:

  1. The input file is run through the preprocessor resulting in a single translation unit.
  2. The output of the preprocessor is compiled to assembly.
  3. The assembler takes that as input an outputs an object file.
  4. The linker stitches together the object files to produce an executable.

[ To be pedantic there's no requirement the steps be separate and modern compilers typically combine them for efficiency. ]

Our concern is the linking step, which combines your code with standard system libraries. The linker copies objects from static libraries directly into the executable. For shared libraries, however, it only provides a reference to the library.

Shared libraries have a lot of advantages. You can update them without recompiling programs and they use less memory because programs can share common code. They also have the obvious drawback that the code isn't in the executable.

The solution to that is the dynamic loader, which is responsible for resolving all shared library references at runtime. The loader is run automatically; instructions for doing so are one thing the linker includes in the executable. Of course this presupposes that the loader can find the libraries.

System libraries are in standard directories, which is straightforward. When that isn't the case the loader will search LD_LIBRARY_PATH. Why doesn't the linker just put the path in the executable? Because then you'd be unable to move or change the library.

In practice you couldn't really move the executable either, since the library is outside the system search path. If it only ran when the library was located in ~luke/lib then you can't give it to joe unless he can read your files. Sucks for joe if you move onto a new job.

Just FYI it would suck in a myriad of other ways as well. It'd make debugging an everlasting nightmare if you could only specify the library location at compile time, amongst other things.

Matthew Gauthier
  • 471