what is the purpose of memory overcommitment on Linux?

Question

I know about memory overcommitment and I profoundly dislike it and usually disable it. I am not thinking of setuid-based system processes (like those running sudo or postfix) but of an ordinary Linux process started on some command line by some user not having admin privileges.

A well written program could malloc (or mmap which is often used by malloc) more memory than available and crash when using it. Without memory overcommitment, that malloc or mmap would fail and the well written program would catch that failure. The poorly written program (using malloc without checks against failure) would crash when using the result of a failed malloc.

Of course virtual address space (which gets extended by mmap so by malloc) is not the same as RAM (RAM is a resource managed by the kernel, see this; processes have their virtual address space initialized by execve(2) and extended by mmap & sbrk so don't consume directly RAM, only virtual memory).

Notice that optimizing RAM usage could be done with madvise(2) (which could give a hint, using MADV_DONTNEED to the kernel to swap some pages onto the disk), when really needed. Programs wanting some overcommitment could use mmap(2) with MAP_NORESERVE. My understanding of memory overcommitment is as if every memory mapping (by execve or mmap) is using implicitly MAP_NORESERVE

My perception of it is that it is simply useful for very buggy programs. But IMHO a real developer should always check failure of malloc, mmap and related virtual address space changing functions (e.g. like here). And most free software programs whose source code I have studied have such check, perhaps as some xmalloc function....

Are there real life programs, e.g. packaged in a typical Linux distributions, which actually need and are using memory overcommitment in a sane and useful way? I know none of them!

What are the disadvantages of disabling memory overcommitment? Many older Unixes (e.g. SunOS4, SunOS5 from the previous century) did not have it, and IMHO their malloc (and perhaps even the general full-system performance, malloc-wise) was not much worse (and improvements since then are unrelated to memory overcommitment).

I believe that memory overcommitment is a misfeature for lazy programmers.

The user of that program could setup some resource limit for setrlimit(2) called with RLIMIT_AS by the parent process (e.g. ulimit builtin of /bin/bash; or limit builtin of zsh, or any modern equivalent for e.g. at, crontab, batch, ...), or a grand-parent process (up to eventually /sbin/init of pid 1 or its modern systemd variant).

Certain packages for R have trouble on OpenBSD because they want large amounts of virtual memory and OpenBSD says nope. Those same packages are fine on Linux and do not cause the crazy drunk oom killer to start blasting away at the process table. — thrig, May 02 '18 at 19:15
@thrig that could be an answer, notably if you where more specific. What R packages? Can they still work in practice? Why do they need memory overcommit? — Basile Starynkevitch, May 02 '18 at 19:17
this was 5+ years ago and I could not find the offending package (or something has been fixed, meanwhile...) in a brief search — thrig, May 02 '18 at 22:48
A couple of comments on the edits you have made to the question: without memory overcommitment, a call to malloc would fail, even if there is plenty of memory still available, I think it would be unrealistic to require all programs to use madvise. The kernel usually gets the hint automatically anyway, by keeping a count of which pages have been recenlty used and which have not. The MAP_NORESERVE flag to the mmap system call only means no swap space is reserved for the mapping, it does not disable demand paging. — Johan Myréen, May 03 '18 at 07:33
what kind of "plenty of memory" do you refer to? virtual memory or RAM? IMHO RAM is managed by the kernel, and application code don't care about it. It uses virtual address space. — Basile Starynkevitch, May 03 '18 at 07:34
@JohanMyréen: Could you explain in your answer how malloc (actually mmap) could fail when there is plenty of virtual memory still available (and of course when there is no limit thru setrlimit) — Basile Starynkevitch, May 03 '18 at 08:04
I was trying to put a spin on your "Without memory overcommitment [..] malloc would fail". But I see I was not clear. What I meant was that a lot of physical RAM page frames are mapped to virtual addresses that are not used, i.e. they would be available for real use without this rigid mapping that follows from strict non-overcommitment. I was not talking about virtual memory. — Johan Myréen, May 03 '18 at 08:45

score 31 · Answer 1 · edited Mar 04 '20 at 10:16

The reason for overcommitting is to avoid underutilization of physical RAM. There is a difference between how much virtual memory a process has allocated and how much of this virtual memory has been actually mapped to physical page frames. In fact, right after a process is started, it reserves very little RAM. This is due to demand paging: the process has a virtual memory layout, but the mapping from the virtual memory address to a physical page frame isn't established until the memory is read or written.

A program typically never uses its whole virtual memory space, and the memory areas touched varies during the run of the program. For example, mappings to page frames containing initialization code that is executed only at the start of the run can be discarded and the page frames can be used for other mappings.

The same applies to data: when a program calls malloc, it reserves a sufficiently large contiguous virtual address space for storing data. However, mappings to physical page frames are not established until the pages are actually used, if ever. Or consider the program stack: every process gets a fairly big contiguous virtual memory area set aside for the stack (typically 8 MB). A process typically uses only a fraction of this stack space; small and well-behaving programs use even less.

A Linux computer typically has a lot of heterogeneous processes running in different stages of their lifetimes. Statistically, at any point in time, they do not collectively need a mapping for every virtual page they have been assigned (or will be assigned later in the program run).

A strictly non-overcommitting scheme would create a static mapping from virtual address pages to physical RAM page frames at the moment the virtual pages are allocated. This would result in a system that can run far fewer programs concurrently, because a lot of RAM page frames would be reserved for nothing.

I don't deny that overcommitting memory has its dangers, and can lead to out-of-memory situations that are messy to deal with. It's all about finding the right compromise.

Avoiding underutilization of RAM might be done with madvise(2) using MADV_DONTNEED (and perhaps should be done so by applications), so I am still not very convinced. But your explanation is interesting, so thanks! — Basile Starynkevitch, May 03 '18 at 05:10
And the usual scheme would be to reserve virtual memory (by allocating it on swap area), not RAM — Basile Starynkevitch, May 03 '18 at 05:23
Processes have a virtual address space (in virtual memory). RAM is managed by the kernel, whole system wise. So be careful in expliciting virtual address space, virtual memory, RAM — Basile Starynkevitch, May 03 '18 at 08:13
Even without overcommitment it may be possible to use some of that reserved RAM for cache. — gmatht, May 31 '20 at 05:05
Relying on madvise to tell the kernel what portions of virtual memory aren't needed would require modifying all programs that will run on the system. Too much work. — Oskar Skog, May 31 '20 at 08:07
A strictly non-overcommitting scheme would create a static mapping from virtual address pages to physical RAM page frames at the moment the virtual pages are allocated. This is only valid when no on disk swap area is set. Solaris is a strictly non over-committing OS but doesn't waste RAM at all if the swap area is large enough to back all reservations. — jlliagre, Jun 01 '20 at 16:37

score 8 · Answer 2 · answered May 03 '18 at 09:09

You say this as if laziness is not considered a virtue in programming :).

Large quantities of software are optimized for simplicity and maintainability, with surviving low-memory conditions as a very low priority. It is common to treat allocation failure as fatal. Exiting the process which exhausts memory avoids a situation where there is no free memory, and the system cannot make progress without either allocating more memory, or complexity in the form of comprehensive pre-allocation.

Notice how fine the difference is between checking allocations and dying, or not checking and crashing. It would not be fair to blame overcommit on programmers simply not bothering to check whether malloc() succeeded or failed.

There is a only a small amount of software which you can trust to continue "correctly" in the face of failed allocations. The kernel should generally be expected to survive. sqlite has a notoriously robust test which includes out of memory testing, specifically because it is intended to support various small embedded systems.

As a failure path not used in normal operation, handling low memory conditions correctly imposes a significant extra burden in maintenance and testing. If that effort does not yield a commensurate benefit, it can more profitably be spent elsewhere.

Where this breaks down, it is also common to have special cases for large allocations, to handle the most common causes of failure.

Allowing a certain amount of overcommit is probably best viewed in this context. It is part of the current default compromise on Linux.

Note the idea that one should disable kernel-level overcommit and instead provide more swap than you ever want to use, also has its haters. The gap in speed between RAM and rotating hard drives has grown over time, such that when the system actually uses the swap space you have allowed it, it can more often be described as "grinding to a halt".

Laziness could justify to not bother free-ing most heap mallocated memory and leave the operating system kernel to properly clean the process after its termination — Basile Starynkevitch, Aug 28 '19 at 10:32
@BasileStarynkevitch absolutely true, and for programs that only run for a brief period and then exit, doing that is acceptable (although not preferable). For programs that are expected to run indefinitely, OTOH, that won't work for obvious reasons. — Jeremy Friesner, May 29 '20 at 00:01

jlliagre · Answer 3 · 2018-05-03T12:16:35.423

I agree with and upvoted Johan Myréen answer but here are more explanations that might help you understanding the issue.

You seem to confuse the swap area, i.e. on disk space intended to store less used RAM and virtual memory. The latter is made of a combination of RAM areas and on disk areas.

Processes are reserving and using virtual memory. They have no idea about where it is stored. When they need to access some data which isn't in RAM, processes (or threads) are suspended until the kernel do the job for the data page to be available.

When there is RAM demand, the kernel free some RAM by storing less used process pages on the disk swap area.

When a process reserves memory (i.e. malloc and the likes), non overcommiting OSes mark unused portions of the virtual memory as unavailable. That means that when the process having made the allocation will actually need to access the reserved pages, they will be guaranteed to be present.

The drawback is given the fact that memory is not usable by any other process is preventing these processes to have their pages paginated out, so RAM is wasted if these processes are inactive. Worst, if the sum of reservations is larger than the size of the swap area, pages of RAM will also be reserved to match the reservation despite not containing any data. This is quite a bad scenario because you'll have RAM which is both unusable and unused. Finally, the worst case scenario is for a huge reservation not to be able to be accepted because there is no more virtual memory (swap + ram) available for it. The process doing the reservation will usually crash.

On the other hand, overcommiting OSes like Linux bet there will be no virtual memory shortage at any given time. They accept most memory reservations (but not unrealistic ones, this can be more or less tuned) and in general, this allows a better utilization of the RAM and swap resources.

This is similar to airline companies overbooking seats. This improve occupancy rate but some passengers might be unhappy. Hopefully, airlines just book them to another flight and possibly compensate them while Linux just throws the heavier passengers out of the flying plane...

To summarize, Linux reserves memory a lazy, "best effort" way while several other OSes do guarantee reservations.

Real cases scenario where overcommitting makes a lot of sense is when a program which uses a lot of virtual memory does a fork followed by an exec.

Let's say you have 4 GB of RAM from which 3 GB are available for virtual memory and 4 GB of swap. There is a process reserving 4 GB but only using 1 GB out of it. There is no pagination so the system performs well. On a non overcommiting OS, that process cannot fork because just after the fork, 4 GB more of virtual memory need to be reserved, and there is only 3 GB left.

On Linux, this fork (or clone) system call will complete successfully (but cheating under the cover) and after the following exec (if any), these reserved but unused 4 GB will be freed without any harm.

Would it be practical to design a Unix system so that the only allocations without pre-committed storage would be blocks tentatively duplicated by fork, and add a system call that would either commit all such blocks or report that it is unable to do so (allowing a program a chance to attempt a graceful shut down if possible). — supercat, Jun 01 '20 at 15:43
@supercat That wouldn't worth the effort. Adding a system call for such a corner case has zero chance to be accepted by Unix/Linux kernel maintainers. — jlliagre, Jun 01 '20 at 16:32
The main goal would be ensuring that the only program that could fail due to overcommit would be a spawned process that makes use of the parent process' address space. I would think that a call to implicitly touch all the copy-on-write pages wouldn't be particularly difficult to implement, but I don't know how things work internally. — supercat, Jun 01 '20 at 16:34
Implementing this would have an impact on performance. The fact pages belong to the parent space doesn't imply they are mapped in RAM. Touching unmapped pages would lead to useless I/Os. — jlliagre, Jun 01 '20 at 17:08
There may be some better way of ensuring that pages have storage committed to them, but the main point to provide a means by which applications that need some guarantees as to when failures could or could not occur would be able to achieve the required semantics without having to block all forms of over-commitment. — supercat, Jun 01 '20 at 17:16
If guaranteed semantics aren't needed often, having an inefficient way of demanding them when needed may not be as good as having an efficient way, but would often still be much better than having no targeted means of demanding them. — supercat, Jun 01 '20 at 17:19

score 2 · Answer 4 · answered Aug 28 '19 at 07:58

Sparse arrays

man proc for /proc/sys/vm/overcommit_memory actually cites an application:

In mode 1, the kernel pretends there is always enough memory, until memory actually runs out. One use case for this mode is scientific computing applications that employ large sparse arrays.

I'm not sure if this is actually used in practice though.

More info on /proc/sys/vm/overcommit_memory at: https://stackoverflow.com/questions/2798330/maximum-memory-which-malloc-can-allocate/57687432#57687432

score 1 · Answer 5 · answered May 31 '20 at 05:39

Killing processes can be desirable

Other answers discuss how over-commitment is more efficient. Additionally, sometimes the killing processes is the right thing to do.

First consider the code while(1){malloc(1)}. With overcommit this will eventually be killed by the OOM killer. Without overcommit it will gobble up all memory available bringing the system to its knees.

One solution without overcommit would be to limit amount of memory using cgroups. However, that leaves the challenge of picking sensible defaults. Sometimes there aren't any sensible defaults. Some scientific tasks are as hard as solving the halting problem, so you don't know how much memory and time they need until they finish. You maximise their chance of success by letting scientific users allocate every byte of available memory safe in the knowledge that the OOM killer will take them down if a more vital process needs their memory.

Somehow true, but the process (or its indirect parents, including /sbin/init of pid 1) could call setrlimit(2). This might be as simple as some ulimit or limit command of your unix shell. If you run a scientific code for months of elapsed time without any limits and without checkpoints, you deserve what you risk. — Basile Starynkevitch, May 31 '20 at 05:45

score -1 · Answer 6 · answered Mar 25 '24 at 14:54

A 6-year late partial answer to:

Are there real life programs, e.g. packaged in a typical Linux distributions, which actually need and are using memory overcommitment in a sane and useful way? I know none of them!

I have found that my system is unstable without memory overcommit. I believe the other answers describe why overcommitment is useful. This answer addresses the necessity of overcommit for common software (at least on my system ).

When experimenting with setting vm.overcommit_memory to 2 and vm.overcommit_ratio to 100, I was very surprised to find that firefox and chromium both die (firefox went 'poof' and the whole application crashed, chromium lost tab processes gracefully) as well as my entire gnome session. The sad dmesg logs are below.

Anecdotally, my system didn't look like it was out of memory. After the first crash I watched top report available memory around 10GB while the other crashes happened.

System

Asrock Z77 Extreme 4

16GB RAM, No Swap

i7 3770k, default clock speeds :/

NixOS, updated March 2024.

dmesg snippet

[70338.201808] __vm_enough_memory: pid: 588132, comm: Sandbox Forked, no enough memory for the allocation
[70338.204580] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.204595] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.204598] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.204600] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.204602] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.204604] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.204606] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.204608] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.204610] __vm_enough_memory: pid: 3874, comm: WRScene~ilder#1, no enough memory for the allocation
[70338.207114] Sandbox Forked[588132]: segfault at 0 ip 00007f0b6bb85322 sp 00007f0b489fc330 error 6 in libxul.so[7f0b69795000+567f000] likely on CPU 6 (core 2, socket 0)
[70338.207143] Code: 8b 0d e2 b3 72 03 48 89 01 c7 04 25 00 00 00 00 9b 02 00 00 e8 bf 2a 28 03 48 8d 05 8f 6f 4e fc 48 8b 0d c1 b3 72 03 48 89 01 <c7> 04 25 00 00 00 00 70 02 00 00 e8 9e 2a 28 03 48 8d 15 e9 24 50
[70343.245298] __vm_enough_memory: 2724 callbacks suppressed
[70343.245303] __vm_enough_memory: pid: 588185, comm: (sd-parse-elf), no enough memory for the allocation
[70343.249298] __vm_enough_memory: pid: 588185, comm: (sd-parse-elf), no enough memory for the allocation
[70343.297866] __vm_enough_memory: pid: 11282, comm: Isolated Web Co, no enough memory for the allocation
[70343.297876] __vm_enough_memory: pid: 11282, comm: Isolated Web Co, no enough memory for the allocation
[70343.297878] __vm_enough_memory: pid: 11282, comm: Isolated Web Co, no enough memory for the allocation
[70343.297881] __vm_enough_memory: pid: 11282, comm: Isolated Web Co, no enough memory for the allocation
[70343.297883] __vm_enough_memory: pid: 11282, comm: Isolated Web Co, no enough memory for the allocation
[70343.297886] __vm_enough_memory: pid: 11282, comm: Isolated Web Co, no enough memory for the allocation
[70343.297888] __vm_enough_memory: pid: 11282, comm: Isolated Web Co, no enough memory for the allocation
[70343.297890] __vm_enough_memory: pid: 11282, comm: Isolated Web Co, no enough memory for the allocation
[70591.470393] __vm_enough_memory: 4076 callbacks suppressed
[70591.470399] __vm_enough_memory: pid: 590218, comm: Sandbox Forked, no enough memory for the allocation
[70591.470842] __vm_enough_memory: pid: 590219, comm: .firefox-wrappe, no enough memory for the allocation
[70591.472835] Sandbox Forked[590218]: segfault at 0 ip 00007f0b6bb85322 sp 00007f0b489fc330 error 6 in libxul.so[7f0b69795000+567f000] likely on CPU 6 (core 2, socket 0)
[70591.472856] Code: 8b 0d e2 b3 72 03 48 89 01 c7 04 25 00 00 00 00 9b 02 00 00 e8 bf 2a 28 03 48 8d 05 8f 6f 4e fc 48 8b 0d c1 b3 72 03 48 89 01 <c7> 04 25 00 00 00 00 70 02 00 00 e8 9e 2a 28 03 48 8d 15 e9 24 50
[70591.476975] __vm_enough_memory: pid: 3839, comm: IPC Launch, no enough memory for the allocation
[70595.363188] __vm_enough_memory: pid: 590255, comm: (sd-parse-elf), no enough memory for the allocation
[70595.364372] __vm_enough_memory: pid: 590255, comm: (sd-parse-elf), no enough memory for the allocation
[70595.365027] __vm_enough_memory: pid: 590255, comm: (sd-parse-elf), no enough memory for the allocation
[70595.365059] __vm_enough_memory: pid: 590255, comm: (sd-parse-elf), no enough memory for the allocation
[70595.365065] __vm_enough_memory: pid: 590255, comm: (sd-parse-elf), no enough memory for the allocation
[70595.365069] __vm_enough_memory: pid: 590255, comm: (sd-parse-elf), no enough memory for the allocation
[70595.365075] __vm_enough_memory: pid: 590255, comm: (sd-parse-elf), no enough memory for the allocation
[70595.365080] (sd-parse-elf)[590255]: segfault at 8 ip 00007fae87764adc sp 00007fff32711db0 error 6 in libdw-0.190.so[7fae8771b000+52000] likely on CPU 4 (core 0, socket 0)
[70595.365089] Code: 00 00 00 4c 8d 15 71 8d 02 00 4c 89 54 24 58 e8 ba 6c fb ff 4c 8b 54 24 58 48 85 c0 49 89 c7 0f 84 d9 09 00 00 48 8b 44 24 28 <41> c7 47 08 ff ff ff ff 4c 89 d6 49 8d 7f 31 49 c7 47 10 00 00 00
[70595.513576] DOM Worker[590203]: segfault at 0 ip 00007fa8a0626762 sp 00007fa8940fdd80 error 4 in libxul.so[7fa8a0395000+567f000] likely on CPU 0 (core 0, socket 0)
[70595.513593] Code: 09 ff 15 21 9f 88 05 4c 8b 3b 48 89 6b 08 48 c7 43 10 00 00 00 00 4c 89 63 58 31 c0 48 8b 4c 24 08 e9 dc fe ff ff 49 8b 45 08 <4c> 8b 30 4d 85 f6 0f 85 b8 fe ff ff 48 8d 05 e2 1c 79 fe 48 8b 0d
[72714.610752] __vm_enough_memory: 4071 callbacks suppressed
[72714.610755] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610762] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610765] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610766] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610793] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610798] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610800] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610802] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610804] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[72714.610805] __vm_enough_memory: pid: 622703, comm: python3.11, no enough memory for the allocation
[152767.089470] __vm_enough_memory: 215447 callbacks suppressed
[152767.089474] __vm_enough_memory: pid: 1289576, comm: Sandbox Forked, no enough memory for the allocation
[152767.093725] __vm_enough_memory: pid: 1289577, comm: .firefox-wrappe, no enough memory for the allocation
[152767.096522] __vm_enough_memory: pid: 1288786, comm: Isolated Web Co, no enough memory for the allocation
[152767.096529] __vm_enough_memory: pid: 1288786, comm: Isolated Web Co, no enough memory for the allocation
[152767.096532] __vm_enough_memory: pid: 1288786, comm: Isolated Web Co, no enough memory for the allocation
[152767.096534] __vm_enough_memory: pid: 1288786, comm: Isolated Web Co, no enough memory for the allocation
[152767.096537] __vm_enough_memory: pid: 1288786, comm: Isolated Web Co, no enough memory for the allocation
[152767.096539] __vm_enough_memory: pid: 1288786, comm: Isolated Web Co, no enough memory for the allocation
[152767.096541] __vm_enough_memory: pid: 1288786, comm: Isolated Web Co, no enough memory for the allocation
[152767.096544] __vm_enough_memory: pid: 1288786, comm: Isolated Web Co, no enough memory for the allocation
[152767.099024] Sandbox Forked[1289576]: segfault at 0 ip 00007f64d4185322 sp 00007f64b0ebd330 error 6 in libxul.so[7f64d1d95000+567f000] likely on CPU 2 (core 2, socket 0)
[152767.099040] Code: 8b 0d e2 b3 72 03 48 89 01 c7 04 25 00 00 00 00 9b 02 00 00 e8 bf 2a 28 03 48 8d 05 8f 6f 4e fc 48 8b 0d c1 b3 72 03 48 89 01 <c7> 04 25 00 00 00 00 70 02 00 00 e8 9e 2a 28 03 48 8d 15 e9 24 50
[152779.207307] __vm_enough_memory: 2775 callbacks suppressed
[152779.207311] __vm_enough_memory: pid: 622934, comm: IPC Launch, no enough memory for the allocation
[152779.586601] __vm_enough_memory: pid: 1289754, comm: Sandbox Forked, no enough memory for the allocation
[152779.586854] __vm_enough_memory: pid: 622934, comm: IPC Launch, no enough memory for the allocation
[152779.596993] Sandbox Forked[1289754]: segfault at 0 ip 00007f64d4185322 sp 00007f64b0ebd330 error 6 in libxul.so[7f64d1d95000+567f000] likely on CPU 3 (core 3, socket 0)
[152779.597005] Code: 8b 0d e2 b3 72 03 48 89 01 c7 04 25 00 00 00 00 9b 02 00 00 e8 bf 2a 28 03 48 8d 05 8f 6f 4e fc 48 8b 0d c1 b3 72 03 48 89 01 <c7> 04 25 00 00 00 00 70 02 00 00 e8 9e 2a 28 03 48 8d 15 e9 24 50
[153355.717655] __vm_enough_memory: pid: 1294484, comm: .org.gnome.Naut, no enough memory for the allocation
[153355.724908] __vm_enough_memory: pid: 1294484, comm: .org.gnome.Naut, no enough memory for the allocation
[153355.724914] __vm_enough_memory: pid: 1294484, comm: .org.gnome.Naut, no enough memory for the allocation
[153355.724916] __vm_enough_memory: pid: 1294484, comm: .org.gnome.Naut, no enough memory for the allocation
[168502.866447] __vm_enough_memory: pid: 1419473, comm: Sandbox Forked, no enough memory for the allocation
[168502.879177] Sandbox Forked[1419473]: segfault at 0 ip 00007f64d4185322 sp 00007f64b0ebd330 error 6 in libxul.so[7f64d1d95000+567f000] likely on CPU 3 (core 3, socket 0)
[168502.879192] Code: 8b 0d e2 b3 72 03 48 89 01 c7 04 25 00 00 00 00 9b 02 00 00 e8 bf 2a 28 03 48 8d 05 8f 6f 4e fc 48 8b 0d c1 b3 72 03 48 89 01 <c7> 04 25 00 00 00 00 70 02 00 00 e8 9e 2a 28 03 48 8d 15 e9 24 50
[168504.721054] __vm_enough_memory: pid: 622934, comm: IPC Launch, no enough memory for the allocation
[168760.052354] __vm_enough_memory: pid: 1421616, comm: Sandbox Forked, no enough memory for the allocation
[168760.058708] __vm_enough_memory: pid: 622859, comm: .firefox-wrappe, no enough memory for the allocation
[168760.058715] __vm_enough_memory: pid: 622859, comm: .firefox-wrappe, no enough memory for the allocation
[168760.058738] __vm_enough_memory: pid: 622859, comm: .firefox-wrappe, no enough memory for the allocation
[168760.058745] __vm_enough_memory: pid: 622859, comm: .firefox-wrappe, no enough memory for the allocation
[168760.058746] __vm_enough_memory: pid: 622859, comm: .firefox-wrappe, no enough memory for the allocation
[168760.065988] Sandbox Forked[1421616]: segfault at 0 ip 00007f64d4185322 sp 00007f64b0ebd330 error 6 in libxul.so[7f64d1d95000+567f000] likely on CPU 0 (core 0, socket 0)
[168760.065998] Code: 8b 0d e2 b3 72 03 48 89 01 c7 04 25 00 00 00 00 9b 02 00 00 e8 bf 2a 28 03 48 8d 05 8f 6f 4e fc 48 8b 0d c1 b3 72 03 48 89 01 <c7> 04 25 00 00 00 00 70 02 00 00 e8 9e 2a 28 03 48 8d 15 e9 24 50
[168766.575425] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575443] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575446] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575448] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575451] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575453] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575455] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575457] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575459] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575461] __vm_enough_memory: pid: 1421685, comm: (sd-parse-elf), no enough memory for the allocation
[168766.575669] (sd-parse-elf)[1421685]: segfault at 8 ip 00007f1cc8a9dadc sp 00007fff7c11d3a0 error 6 in libdw-0.190.so[7f1cc8a54000+52000] likely on CPU 3 (core 3, socket 0)
[168766.575678] Code: 00 00 00 4c 8d 15 71 8d 02 00 4c 89 54 24 58 e8 ba 6c fb ff 4c 8b 54 24 58 48 85 c0 49 89 c7 0f 84 d9 09 00 00 48 8b 44 24 28 <41> c7 47 08 ff ff ff ff 4c 89 d6 49 8d 7f 31 49 c7 47 10 00 00 00
[168766.661472] .gnome-shell-wr[2013]: segfault at 38 ip 00007fbabf199468 sp 00007ffff9ac2200 error 6 in crocus_dri.so[7fbabe094000+11db000] likely on CPU 6 (core 2, socket 0)
[168766.661486] Code: 66 90 55 48 89 fd 53 48 89 f3 48 89 d6 89 ca 48 83 ec 08 48 8b 47 08 48 8b b8 e0 06 00 00 e8 af f8 fe ff 66 0f ef c0 48 89 03 <48> 81 48 38 80 00 00 00 0f 11 43 18 80 bd c8 00 00 00 00 c7 43 28
[170015.617311] __vm_enough_memory: 120 callbacks suppressed
[170015.617317] __vm_enough_memory: pid: 1423004, comm: IPC Launch, no enough memory for the allocation
[170015.617578] __vm_enough_memory: pid: 1441535, comm: TaskCon~ller #2, no enough memory for the allocation
[170015.617899] __vm_enough_memory: pid: 1441535, comm: TaskCon~ller #2, no enough memory for the allocation
[170015.620276] __vm_enough_memory: pid: 1441538, comm: TaskCon~ller #5, no enough memory for the allocation
[170015.620292] __vm_enough_memory: pid: 1441538, comm: TaskCon~ller #5, no enough memory for the allocation
[170015.620296] __vm_enough_memory: pid: 1441538, comm: TaskCon~ller #5, no enough memory for the allocation
[170015.620300] __vm_enough_memory: pid: 1441538, comm: TaskCon~ller #5, no enough memory for the allocation
[170015.620303] __vm_enough_memory: pid: 1441538, comm: TaskCon~ller #5, no enough memory for the allocation
[170015.620307] __vm_enough_memory: pid: 1441538, comm: TaskCon~ller #5, no enough memory for the allocation
[170015.620312] __vm_enough_memory: pid: 1441538, comm: TaskCon~ller #5, no enough memory for the allocation
[170024.500384] __vm_enough_memory: 10 callbacks suppressed
[170024.500395] __vm_enough_memory: pid: 1422979, comm: Renderer, no enough memory for the allocation
[170024.500399] __vm_enough_memory: pid: 1422979, comm: Renderer, no enough memory for the allocation
[170024.502299] __vm_enough_memory: pid: 1422979, comm: Renderer, no enough memory for the allocation
[170024.502303] __vm_enough_memory: pid: 1422979, comm: Renderer, no enough memory for the allocation
[170024.503204] __vm_enough_memory: pid: 1441645, comm: .firefox:gdrv0, no enough memory for the allocation
[170024.503211] __vm_enough_memory: pid: 1441645, comm: .firefox:gdrv0, no enough memory for the allocation
[170024.503256] __vm_enough_memory: pid: 1441645, comm: .firefox:gdrv0, no enough memory for the allocation
[170024.503258] __vm_enough_memory: pid: 1441645, comm: .firefox:gdrv0, no enough memory for the allocation
[170024.503652] __vm_enough_memory: pid: 1441645, comm: .firefox:gdrv0, no enough memory for the allocation
[170024.503655] __vm_enough_memory: pid: 1441645, comm: .firefox:gdrv0, no enough memory for the allocation
[170024.521595] .gnome-shell-wr[1422358]: segfault at 38 ip 00007f8027199468 sp 00007ffdb43f9aa0 error 6 in crocus_dri.so[7f8026094000+11db000] likely on CPU 5 (core 1, socket 0)
[170024.521606] Code: 66 90 55 48 89 fd 53 48 89 f3 48 89 d6 89 ca 48 83 ec 08 48 8b 47 08 48 8b b8 e0 06 00 00 e8 af f8 fe ff 66 0f ef c0 48 89 03 <48> 81 48 38 80 00 00 00 0f 11 43 18 80 bd c8 00 00 00 00 c7 43 28
[170387.862898] __vm_enough_memory: 52 callbacks suppressed
[170387.862901] __vm_enough_memory: pid: 1443246, comm: IPC Launch, no enough memory for the allocation
[170834.102309] __vm_enough_memory: pid: 1450011, comm: Sandbox Forked, no enough memory for the allocation
[170834.102746] Sandbox Forked[1450011]: segfault at 0 ip 00007f4ed3f85322 sp 00007f4eb0afe330 error 6 in libxul.so[7f4ed1b95000+567f000] likely on CPU 3 (core 3, socket 0)
[170834.102756] Code: 8b 0d e2 b3 72 03 48 89 01 c7 04 25 00 00 00 00 9b 02 00 00 e8 bf 2a 28 03 48 8d 05 8f 6f 4e fc 48 8b 0d c1 b3 72 03 48 89 01 <c7> 04 25 00 00 00 00 70 02 00 00 e8 9e 2a 28 03 48 8d 15 e9 24 50
[170834.104131] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.104138] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.104142] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.104146] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.104159] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.104162] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.104165] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.104169] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.104172] __vm_enough_memory: pid: 1449955, comm: TaskCon~ller #7, no enough memory for the allocation
[170834.109927] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.110328] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.133854] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.134262] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.134380] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.134394] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.134405] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.134416] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.134428] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.134439] Huh VM_FAULT_OOM leaked out to the #PF handler. Retrying PF
[170834.356940] Isolated Web Co[1446322]: segfault at 0 ip 00007fa752eac8d7 sp 00007ffd6b26c170 error 6 in libxul.so[7fa74e595000+567f000] likely on CPU 7 (core 3, socket 0)
[170834.356955] Code: 8b 38 48 8d 35 35 0d f5 f9 48 8d 0d a8 87 e7 f9 4c 89 f2 41 b8 02 05 00 00 31 c0 e8 83 bc d5 00 48 8b 05 0c 3e 20 01 4c 89 30 <c7> 04 25 00 00 00 00 04 05 00 00 e8 e9 b4 d5 00 cc cc cc cc cc cc
[170864.253528] __vm_enough_memory: 5028 callbacks suppressed
[170864.253532] __vm_enough_memory: pid: 1450340, comm: Sandbox Forked, no enough memory for the allocation
[170864.255492] __vm_enough_memory: pid: 1450317, comm: TaskCon~ller #7, no enough memory for the allocation
[170864.267303] __vm_enough_memory: pid: 1443177, comm: .firefox-wrappe, no enough memory for the allocation
[170864.267975] __vm_enough_memory: pid: 1443177, comm: .firefox-wrappe, no enough memory for the allocation
[170864.268096] __vm_enough_memory: pid: 1443177, comm: .firefox-wrappe, no enough memory for the allocation
[170864.268437] __vm_enough_memory: pid: 1443177, comm: .firefox-wrappe, no enough memory for the allocation
[170864.268440] __vm_enough_memory: pid: 1443177, comm: .firefox-wrappe, no enough memory for the allocation
[170864.268467] __vm_enough_memory: pid: 1443177, comm: .firefox-wrappe, no enough memory for the allocation
[170864.268477] __vm_enough_memory: pid: 1443177, comm: .firefox-wrappe, no enough memory for the allocation
[170864.268479] __vm_enough_memory: pid: 1443177, comm: .firefox-wrappe, no enough memory for the allocation
[170864.269791] Sandbox Forked[1450340]: segfault at 0 ip 00007f4ed3f85322 sp 00007f4eb0afe330 error 6 in libxul.so[7f4ed1b95000+567f000] likely on CPU 2 (core 2, socket 0)
[170864.269801] Code: 8b 0d e2 b3 72 03 48 89 01 c7 04 25 00 00 00 00 9b 02 00 00 e8 bf 2a 28 03 48 8d 05 8f 6f 4e fc 48 8b 0d c1 b3 72 03 48 89 01 <c7> 04 25 00 00 00 00 70 02 00 00 e8 9e 2a 28 03 48 8d 15 e9 24 50
[171165.508437] __vm_enough_memory: 6 callbacks suppressed
[171165.508440] __vm_enough_memory: pid: 1442321, comm: pool-spawner, no enough memory for the allocation
[171165.508476] __vm_enough_memory: pid: 1442321, comm: pool-spawner, no enough memory for the allocation
[171165.508498] __vm_enough_memory: pid: 1442321, comm: pool-spawner, no enough memory for the allocation
[171165.511514] __vm_enough_memory: pid: 1452853, comm: chromium, no enough memory for the allocation
[171165.511573] __vm_enough_memory: pid: 1452853, comm: chromium, no enough memory for the allocation
[171165.511693] __vm_enough_memory: pid: 1451347, comm: chrome_crashpad, no enough memory for the allocation
[171165.511706] __vm_enough_memory: pid: 1451347, comm: chrome_crashpad, no enough memory for the allocation
[171165.511858] __vm_enough_memory: pid: 1451349, comm: chrome_crashpad, no enough memory for the allocation
[171165.511862] __vm_enough_memory: pid: 1451349, comm: chrome_crashpad, no enough memory for the allocation
[171165.523896] __vm_enough_memory: pid: 1442336, comm: JS Helper, no enough memory for the allocation
[171170.688552] traps: ThreadPoolForeg[1452516] trap invalid opcode ip:55ed3562ee16 sp:7fe5e59fbce0 error:0 in chromium[55ed2f9ea000+d5e1000]
[171171.342759] __vm_enough_memory: 27 callbacks suppressed
[171171.342762] __vm_enough_memory: pid: 1453224, comm: (sd-parse-elf), no enough memory for the allocation
[171171.350324] __vm_enough_memory: pid: 1453224, comm: (sd-parse-elf), no enough memory for the allocation
[171171.350328] __vm_enough_memory: pid: 1453224, comm: (sd-parse-elf), no enough memory for the allocation
[171171.350331] __vm_enough_memory: pid: 1453224, comm: (sd-parse-elf), no enough memory for the allocation

I don't know how these crashes make sense in software that also builds for Windows, which as far as I know does not overcommit memory.

what is the purpose of memory overcommitment on Linux?

6 Answers6

System

dmesg snippet

Linked