"Wrong" total memory in VM

Question

I am running here a couple of Debian 9.3 VMs with the kernel 4.9.0-5-amd64 in VMware Fusion 10.1.1 (in High Sierra).

What concerns me is that the total memory in Free does not reflect correctly the memory I am setting aside for several VMs.

As a test, I have set aside exactly 2GB for the VM and yet, free -m only shows 1986MB:

$ free -m
              total        used        free      shared  buff/cache   available
Mem:           1986          51        1864           1          70        1825

or without the -m:

$ free 
              total        used        free      shared  buff/cache   available
Mem:        2033760       52264     1909584        1108       71912     1869628
Swap:        999420           0      999420

Debugging the situation I also looked to /proc/meminfo and found this:

$ egrep "MemTotal|DirectMap2M" /proc/meminfo
MemTotal:        2033760 kB
DirectMap2M:     2054144 kB

So actually DirectMap2M reflects the 2GB; why has MemTotal aprox. less 20MB then?

Interestingly enough, while Googling for MemTotal/DirectMap2M found this article: Memory / Ram is wrong

If your operating system is showing the wrong RAM allocation via the free –m or top command please be assured that you really do have the correct amount of RAM allocated to your VPS.

It is simply the reporting that is wrong, this is due to us using the latest Xen 4.x.x Versions for performance enhancements which unfortunately can cause this anomaly, more so on 32bit OS templates than others.

Also please keep in mind that more modern kernels do not present some of the kernel reserved memory to the OS.

So what is happening here? Is it the kernel reserving memory as they suggest? For what end?
(yes, they are talking about Xen and this is about VmWare Fusion, yet it might be a clue)

For complementing:

vmstat output:

$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 1909344  13988  57928    0    0    31     0   19   37  0  0 100  0  0

top output:

$ top -b -n 1 | grep Mem
KiB Mem :  2033760 total,  1906228 free,    52836 used,    74696 buff/cache
KiB Swap:   999420 total,   999420 free,        0 used.  1867664 avail Mem

dmidecode output (partial);

Handle 0x0085, DMI type 6, 12 bytes
Memory Module Information
        Socket Designation: RAM socket #0
        Bank Connections: None
        Current Speed: Unknown
        Type: EDO DIMM
        Installed Size: 2048 MB (Single-bank Connection)
        Enabled Size: 2048 MB (Single-bank Connection)
        Error Status: OK

Output of dmesg:

$ sudo dmesg | egrep "Memory|Free|ACPI" | egrep -v "edge|wakeup|noapic|Added|Bug|IRQ|pnp|Plug"
[    0.000000] BIOS-e820: [mem 0x000000007fee0000-0x000000007fefefff] ACPI data
[    0.000000] BIOS-e820: [mem 0x000000007feff000-0x000000007fefffff] ACPI NVS
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x00000000000F6A10 000024 (v02 PTLTD )
[    0.000000] ACPI: XSDT 0x000000007FEEB683 00005C (v01 INTEL  440BX    06040000 VMW  01324272)
[    0.000000] ACPI: FACP 0x000000007FEFEE73 0000F4 (v04 INTEL  440BX    06040000 PTL  000F4240)
[    0.000000] ACPI: DSDT 0x000000007FEEC923 012550 (v01 PTLTD  Custom   06040000 MSFT 03000001)
[    0.000000] ACPI: FACS 0x000000007FEFFFC0 000040
[    0.000000] ACPI: FACS 0x000000007FEFFFC0 000040
[    0.000000] ACPI: BOOT 0x000000007FEEC8FB 000028 (v01 PTLTD  $SBFTBL$ 06040000  LTP 00000001)
[    0.000000] ACPI: APIC 0x000000007FEEC1B9 000742 (v01 PTLTD  ? APIC   06040000  LTP 00000000)
[    0.000000] ACPI: MCFG 0x000000007FEEC17D 00003C (v01 PTLTD  $PCITBL$ 06040000  LTP 00000001)
[    0.000000] ACPI: SRAT 0x000000007FEEB77F 000880 (v02 VMWARE MEMPLUG  06040000 VMW  00000001)
[    0.000000] ACPI: HPET 0x000000007FEEB747 000038 (v01 VMWARE VMW HPET 06040000 VMW  00000001)
[    0.000000] ACPI: WAET 0x000000007FEEB71F 000028 (v01 VMWARE VMW WAET 06040000 VMW  00000001)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x0009ffff]
[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x00100000-0x7fffffff]
[    0.000000] ACPI: PM-Timer IO Port: 0x1008
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] Using ACPI for processor (LAPIC) configuration information
[    0.000000] ACPI: HPET id: 0x8086af01 base: 0xfed00000
[    0.000000] Memory: 2011544K/2096628K available (6196K kernel code, 1159K rwdata, 2848K rodata, 1408K init, 688K bss, 85084K reserved, 0K cma-reserved)
[    0.005465] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    0.005475] ACPI: setting ELCR to 0200 (from 0e80)
[    0.044362] x86/mm: Memory block size: 128MB
[    0.046853] PM: Registering ACPI NVS region [mem 0x7feff000-0x7fefffff] (4096 bytes)
[    0.170037] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f])
[    0.178855] pci 0000:00:07.3: quirk: [io  0x1000-0x103f] claimed by PIIX4 ACPI
[    0.282772] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.995687] Freeing initrd memory: 17572K
[    1.085789] Freeing unused kernel memory: 1408K
[    1.085832] Freeing unused kernel memory: 1980K
[    1.085873] Freeing unused kernel memory: 1248K

Full /proc/meminfo output:

$ cat /proc/meminfo 
MemTotal:        2033760 kB
MemFree:         1906864 kB
MemAvailable:    1868300 kB
Buffers:           14720 kB
Cached:            50220 kB
SwapCached:            0 kB
Active:            45004 kB
Inactive:          28204 kB
Active(anon):       8308 kB
Inactive(anon):     1064 kB
Active(file):      36696 kB
Inactive(file):    27140 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:        999420 kB
SwapFree:         999420 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:          8284 kB
Mapped:            14860 kB
Shmem:              1108 kB
Slab:              23996 kB
SReclaimable:       9756 kB
SUnreclaim:        14240 kB
KernelStack:        3052 kB
PageTables:         1348 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     2016300 kB
Committed_AS:      38852 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       42880 kB
DirectMap2M:     2054144 kB
DirectMap1G:           0 kB

This has been answered before: https://unix.stackexchange.com/questions/30324/why-does-linux-show-both-more-and-less-memory-than-i-physically-have-installed — Terrance, Feb 24 '18 at 16:16
@Terrance Thanks. The answer there just fails to mention kernel, but is a nice complement to the answer here. — Rui F Ribeiro, Feb 24 '18 at 16:25

Stephen Kitt · Accepted Answer · 2018-02-23T22:18:17.270

free, /proc/meminfo etc. only show the memory actually available to user space; the kernel sets aside some memory for its own use. If you look for a Memory: line in your boot logs (/var/log/dmesg.0 or some such, or journalctl), you’ll see something like

Memory: 32818828K/33439808K available (5612K kernel code, 1083K rwdata, 1896K rodata, 1264K init, 832K bss, 620980K reserved, 0K cma-reserved)

The amount of memory available after boot will typically be slightly larger than the amount indicated here, because some of the memory used for initialisation is returned to the system, and the amount of reserved memory can change (e.g. if it’s reserved for an integrated GPU); in my case, MemTotal shows 32062 MiB instead of the 32049 MiB given above.

In your case, only 62MiB are reserved (2048 – 1986); that is quite sufficient to cover the kernel code and data, plus some reserved memory. The boot logs will also include details of the system’s memory map, which should account for most of the reserved memory (it’s reserved for the firmware, ACPI etc., even in a VM).

MemTotal never corresponds to the amount of installed physical memory, or allocated memory for a VM, and that’s perfectly normal.

I got the impression however the memory "missing" increased with size. I used 2MB instead of 512 for it to be more evident. Maybe internal structures and stack too? — Rui F Ribeiro, Feb 24 '18 at 05:14
Yes, the more memory the system has, the more the kernel needs for page tables etc., and IIRC they are counted in “reserved” memory (but I’d have to check). — Stephen Kitt, Feb 24 '18 at 09:49

Rui F Ribeiro · Answer 2 · 2019-03-26T23:43:01.087

Complementing the answer for @StephenKitt, confirming MemTotal is not the the value of the entire available RAM:

From the kernel sources, Documentation/filesystems/proc.txt

MemTotal - Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code)

Also at boot time, x86/boot/compressed/kaslr.c:

enum mem_avoid_index {
    MEM_AVOID_ZO_RANGE = 0,
    MEM_AVOID_INITRD,
    MEM_AVOID_CMDLINE,
    MEM_AVOID_BOOTPARAMS,
    MEM_AVOID_MEMMAP_BEGIN,
    MEM_AVOID_MEMMAP_END = MEM_AVOID_MEMMAP_BEGIN + MAX_MEMMAP_REGIONS - 1,
    MEM_AVOID_MAX,
};

"Wrong" total memory in VM

2 Answers2