Cannot use all memory

Question

I frequently get app crashing due to memory allocation error (reported by dmesg), even during normal browsing with few open tabs, or image processing with gimp. The machine has 16GB of ram. I have tried to test for memory allocation following this question:

$ stress-ng --vm-bytes $(awk '/MemAvailable/{printf "%d\n", $2 * 0.9;}' < /proc/meminfo)k --vm-keep -m 1
stress-ng: info: [28129] defaulting to a 86400 second (1 day, 0.00 secs) run per stressor 
stress-ng: info: [28129] dispatching hogs: 1 vm
stress-ng: error: [28148] stress-ng-vm: gave up trying to mmap, no available memory
stress-ng: info: [28129] successful run completed in 10.02s

Note that the "no available memory" message is not returned if I use 0.55 instead of 0.9 in the awk line, but every value larger than 0.55 give the error. Why I cannot allocate more that 0.55 of available memory?

This is the content of /proc/meminfo

MemTotal:       15919800 kB
MemFree:         5488312 kB
MemAvailable:   12100632 kB
Buffers:          716568 kB
Cached:          6156352 kB
SwapCached:            0 kB
Active:          5465468 kB
Inactive:        4214576 kB
Active(anon):    2985580 kB
Inactive(anon):   117944 kB
Active(file):    2479888 kB
Inactive(file):  4096632 kB
Unevictable:      176576 kB
Mlocked:            2476 kB
SwapTotal:        999420 kB
SwapFree:         999420 kB
Dirty:               156 kB
Writeback:             0 kB
AnonPages:       2983852 kB
Mapped:           713212 kB
Shmem:            294288 kB
KReclaimable:     380216 kB
Slab:             448228 kB
SReclaimable:     380216 kB
SUnreclaim:        68012 kB
KernelStack:       14432 kB
PageTables:        55080 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    16760020 kB
Committed_AS:    9543584 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       31412 kB
VmallocChunk:          0 kB
Percpu:             2848 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      274432 kB
DirectMap2M:     9728000 kB
DirectMap1G:     6291456 kB

and these are my overcommit settings:

$ cat /proc/sys/vm/overcommit_memory
2
$ cat /proc/sys/vm/overcommit_ratio
99
$ cat /proc/sys/vm/swappiness
10

Answer 1

You’re using strict overcommit accounting, so processes aren’t allowed to map more than the available memory. The Committed_AS line shows that 9543584 kiB are currently allocated, and the CommitLimit line shows that the limit is 16760020 kiB, leaving 7216436 kiB which can be allocated; that’s 59.6% of your available memory, so taking into account some overhead for stress-ng, it’s not too surprising that you can’t allocate more than 55% of your available memory to the vm worker.

If you want to allow stress-ng to allocate more of the available memory, you need to allow overcommitting:

sudo sysctl vm.overcommit_memory=0

or increase the amount of swap to raise your commit limit.