14

Time and time again I see questions such as these:

And these are the types of solutions we're typically pushing:

Is this really the best we can do? Aren't there binary builds of GLIBC that we can simply unzip into a directory such as /opt/myglibc, and set the $LD_LIBRARY_PATH or whatever and run whatever application we want, without issues?

An application, such as the newer builds of Chrome (28+) that appear to require GLIBC 2.14?

NOTE: This thread titled: Google Chrome 29 Released – Install on RHEL/CentOS 6 and Fedora 19/15 on tecmint.com is what ultimately got me thinking about this.

References

Community
  • 1
slm
  • 369,824

1 Answers1

2

If it were for any other library, but glibc ... I suppose there can't be quick ways, because the glibc is the place where stuff is "hard coded". The glibc fits to your kernel version and its loader is the instance that actually does the right thing (TM) with LD_LIBRARY_PATH.

Maybe the correct way is to: 

LD_LIBRARY_PATH="/opt/myglibc/;..." /opt/myglibc/ld-linux.so.2 the_program`

Not sure if this works, though.

At any rate, I think using an alternative glibc requires a yet to implement framework, because search paths are wired sometimes and the glibc always has to fit your OS/kernel, so there can't be generic binaries, IMO. Debian's multiarch shows that it's not trivial, but still can be done. If one was to have some other means of discerning libraries besides the target architecture.

The website just gave me this other related thread:

There, the accepted answer includes a link to a program called rtldi, which seems to solve the glibc issue. It's from 2004 so it may not work right out of the linker anymore, but maybe its worth looking into. Its source is GPLv2.

Jehova, Jehova

A friend of mine once came up with the idea, that the actual use of shared libraries is overrated. And he does have a point: shared libraries are good to not fill up your computer's memory with duplicates, but considering the individual application instance this is only a few MBs.

There is only a few applications where we would resort to actions such as providing them their own glibc. Saving us a lengthy analysis let's call them "immediate applications", that are of use by themselves, in the sense of getting work done. For example web browsers, mail user agents, office suits and music players allow the user to get what they want and there are only a few instances per user. To portrait the other side, system services, window managers, even entire desktop environments are all very important, but merely supporting and often not sufficiently uncommon or critical, so that people would be willing to give them their very own glibc.

The number of "immediate applications" is rather small, absolutely per user and relatively compared to what "basic" OSs and DEs spawn these days. If immediate applications, like Chrome, Firefox were compiled statically, the additional memory requirement for the average system would be a few 100MB. An argument that doesn't carry very far on today's many GB systems so statical linking for immediate applications can be an option.

There are also the concepts of swap space and SSDs which allow for rediculously fast swapin/-out, which also helps handling the increaed memory requirement.

The glibc issue discussed here is not really resolved though static linking, but for applications like web browser a sort of self contained distribution format is conceivable, where the X protocol, some sound daemon and some kernel meethods as the only interface. The advantage would be fewer library version incompatibilities.

Community
  • 1
Bananguin
  • 7,984
  • 2
    "we're talking about a few 100MB here" Uh, no. Although the bulk of the libraries themselves may not be that much (but probably a order of magnitude or two more than 100 MB -- try du -h /lib), keep in mind that if those were compiled in statically, that amount of RAM would be required for each and every application compiled with them. So if, eg. you have two apps using the same library stack, now you'll need twice as much memory. Three apps? Three times as much. Not to mention it would largely negate the benefits of caching... – goldilocks Nov 14 '13 at 20:26
  • 2
    ...since, of course, you couldn't just cache glibc -- you'd have to cache copies of every applications that's run (== ridiculous). In short, modern operating systems would plain and simple be totally impossible on modern hardware if it were not for modern techniques such as shared objects. You wouldn't need just a bit more memory -- you'd need 10 or 100 times as much memory. – goldilocks Nov 14 '13 at 20:28
  • My Debian has 235MB in /lib, of which 202MB are kernel modules. Yes, /usr/lib is 4GB, but that allows precicely no conclusion on how much the individual program requires. The caches of processors are only a few MB. With the memory consumption of something like a recent web browser, the impact of statically linked binaries on caching is also not that big and diminishing with the amount concurrently running programs; also for the reason of relatively small caches. My estimations seem more accurate than yours. Uh, yes. – Bananguin Nov 14 '13 at 21:48
  • Not to mention the other huge problem with static linking—updates are a PITA. If there is a security issue in glibc, no big deal: upgrade glibc, restart your programs. OTOH, were your programs statically linked, you'd have to download a new version of every program. And your distro would have had to recompile (or at least re-link, if they kept all the .o files, unlikely due to huge size) the entire distro. – derobert Nov 14 '13 at 22:44
  • @derobert: yes, that is a problem. but lets think about the application domain: if a distro has a current software package in its current distribution, there is no need for static linking. if static linking is an integral concept of the distro, dependencies can be formulated and all the recompiling can be automated (ports, portage, paludis). costs more energy, but not a real PITA. static linking is mainly for binary packages that are outside the "normal" distribution. I don't contest that statically linked programs have other issues, but they all seem quite manageable. – Bananguin Nov 14 '13 at 23:15
  • @Bananguin it may not be a problem for small distros, but e.g., if you want to do Debian, you need 50–100 ECC instances for half a day. That's for one architecture (of 14), the fastest one.¹ You're also talking about a three-quarter TB mirror hit.² These problems are manageable, I suppose, if you have a government budget. – derobert Nov 14 '13 at 23:21
  • "The caches of processors are only a few MB." -- by caching I meant disk caching. You can cache the same shared object and use it repeatedly with different applications, that would be pointless if everything were static. I'll stick by my point that a modern OS would be impossible with all static parts, beyond, e.g., booting to a single user console prompt. A statically linked version of a modular GUI desktop as they are designed now would require more memory than most desktops have to run. Filling memory with the same thing over and over when you don't have to is NOT a sane plan. – goldilocks Nov 15 '13 at 02:43
  • @derobert: look at the topic we are discussing here. we are talking about software that is outside of distributions and therefore requires its own libraries. Especially we are talking about a few applications, not entire distributions. It has never been suggested to have an entire distribution linked statically. Therefore the problem is by far not as big as you describe. – Bananguin Nov 15 '13 at 09:44
  • @goldilocks: you should reread my post which talks about applications that are used on top of DEs, like the example Chrome browser, which is not part of any Linux DE. I was never suggesting an entirely static linux distribution. The example, Chome, actually points to software which is often distributed as binary package by the maintainer. Those progams often ship their own versions of libraries anyways, negating any positive effect shared objects may have. Look at the topic discussed herein. We want to give programs their own glibc. For this purpose it hardly matters when we do the linking. – Bananguin Nov 15 '13 at 09:57
  • @derobert,goldilocks: I edited my answer to point out that I am talking about the problem posed by slm, not building software in general. – Bananguin Nov 15 '13 at 10:40
  • There is also the slight problem, as noted in that question you linked to, that glibc doesn't actually support static linking: NSS modules are always dynamically linked (and have to be, which one to link in is configured in /etc/nsswitch.conf). And I'm pretty sure the ABI isn't guaranteed stable, has changed in the past. Its probably similar for GTK input methods, etc... – derobert Nov 15 '13 at 10:55
  • @goldilocks I think the space savings aren't that large. I ran (as root): grep -h Shared_ /proc/[0-9]*/smaps | cut -d: -f2 | perl -ne 'BEGIN { $tot = 0 }; /^\s*(\d+) kB$/ or die "Bad line: $_"; $tot += $1; END { print "Total: $tot kB\n" }' and got numbers between 1G and 2G, depending on which desktop machine I ran it on. Not sure if that script is right, though. If it is, that's not too much, especially if you use same-page merging (which would recover a lot of it). [yes, using grep, cut, and perl is silly... but that line was built up in parts...] – derobert Nov 15 '13 at 10:58
  • 1
    @derobert : Seems fair. Clearly my claims were hyperbolic -- here with 1.8 GB committed that spat out 521 MB. So that would be a 30% increase. Of course, that is still not a selling point for a strategy that has no advantages (but "only requires 30% more RAM"). – goldilocks Nov 15 '13 at 14:57