What is using memory?

Question

According to htop and free, the machine I'm looking at is currently at 6631 / 11907 MB of RAM. That's more or less half of the RAM available. However when I sort top or htop by MEM%, the most memory being used is by Xorg with 0.8% MEM. gnome-settings is using 0.3%, and a couple of others are using 0.1%. These percentages don't add up to ~50%.

How can I find out what is using ~50% of the memory, and why am I not seeing this memory usage in top/htop?

The output of free -mt:

             total       used       free     shared    buffers     cached
Mem:         11907      10660       1246          0        650       3379
-/+ buffers/cache:       6629       5277
Swap:         3999         26       3973
Total:       15907      10686       5220

EDIT: As requested...

vmstat 1 5

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0  26640 312180 695976 4229988    0    0     4    34    0    1  1  1 96  2  0   
 0  0  26640 312032 695976 4230016    0    0     0     0  157  191  0  0 100  0  0  
 0  0  26640 312156 695976 4230016    0    0     0     0  158  270  0  0 100  0  0  
 0  0  26640 312156 695976 4230016    0    0     0     0  128  189  0  0 100  0  0  
 0  0  26640 312156 695976 4230016    0    0     0     0  212  308  0  0 100  0  0

cat /proc/meminfo

MemTotal:       12192876 kB
MemFree:          311644 kB
Buffers:          695984 kB
Cached:          4230016 kB
SwapCached:         7684 kB
Active:          1378792 kB
Inactive:        3656424 kB
Active(anon):       2192 kB
Inactive(anon):   108500 kB
Active(file):    1376600 kB
Inactive(file):  3547924 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       4095992 kB
SwapFree:        4069352 kB
Dirty:                28 kB
Writeback:             0 kB
AnonPages:        102412 kB
Mapped:            52116 kB
Shmem:              1456 kB
Slab:            5584084 kB
SReclaimable:    4402760 kB
SUnreclaim:      1181324 kB
KernelStack:        3176 kB
PageTables:        17852 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    10192428 kB
Committed_AS:     678004 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      185496 kB
VmallocChunk:   34359540220 kB
HardwareCorrupted:     0 kB
AnonHugePages:     28672 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     5177316 kB
DirectMap2M:     7403520 kB
DirectMap1G:           0 kB

You can show memory usage by using top then SHIFT+m for more options see top then press h — Valentin Bajrami, Jul 18 '14 at 14:41
Could you please provide /proc/meminfo? top sorted by memory usage might also be useful (not all, just the first 15 entries or so) — phemmer, Jul 18 '14 at 15:45

score 1 · Answer 1 · edited Apr 13 '17 at 12:36

1

The issue may be due to shared memory, which isn't taken into account by many utilities because shared between several processes. You can check with the smemstat utility ("report memory usage with shared memory divided proportionally").

EDIT: A significant part of the memory can also be taken by the kernel (included in the used value of free output). Look at the contents of /proc/meminfo. According to Tracking down "missing" memory usage in linux, Slab + Shmem + KernelStack + PageTables gives the kernel space. Slab seems to be rather important in general.

Other users had similar problems:

RAM usage doesn't add up? (Free+used < total) (due to a bug related to NFS)
Discrepancy between reported used memory and sum of application memory usage

edited Apr 13 '17 at 12:36

Community

1

answered Jul 18 '14 at 16:07

vinc17

12,174

Isn't that covered by the "shared" column in free output? – lgeorget Jul 18 '14 at 19:18
Having 0 in the "shared" column of free is strange: no shared libraries at all? On my machine, I have 6.5 MB in this column, but the PSS−USS difference in smemstat output gives 15 MB, and htop and top both say 30 MB for Firefox alone (I thought that it would be a lower bound). I'm a bit lost. – vinc17 Jul 18 '14 at 20:38
1

@lgeorget Correcting my previous comment, free shows the physical memory usage, not the virtual one. So, I think that's a bad command here. – vinc17 Jul 18 '14 at 20:49
@lgeorget But yes, I now think that if there were shared pages in the physical memory, the shared column in free output wouldn't be 0. But 0 also means that the value is not available, as not exported by the kernel (see the free(1) man page); strange, but this is the only explanation I have. – vinc17 Jul 18 '14 at 20:59
1

@lgeorget After more research, the 0 may just be because it is rounded to 0 MB. Now, the shared column in free output is the Shmem value and is part of kernel space according to http://unix.stackexchange.com/questions/56879/tracking-down-missing-memory-usage-in-linux (thus this is unrelated to shared memory). Unfortunately, Shmem is marked as "To be documented" in the proc(5) man page. – vinc17 Jul 19 '14 at 00:31
Thank you for your feedback. That's valuable information – lgeorget Jul 19 '14 at 20:12

score 0 · Answer 2 · answered Jul 18 '14 at 14:59

There are two possible explanation to the discrepancy:

You are on a virtual machine, and there's shared memory.
You have many, many process that consume a minuscule amount of memory, but when aggregated, they consume many more memory.

The first is easy to determine, the second... likewise, but with the caveat that you need to find several process that are using your RAM. ps, uniq and sort can help you:

➜  ~  ps axco command | uniq -c | sort -nr | head
      2 xdg-open <defunct>
      2 mplayer
      1 zsh
      1 zsh
      1 zsh
      1 x-session-manag
      1 xprop
      1 Xorg
      1 writeback
      1 watchdog/0

With this you can see if there's a rogue process calling itself, and using your memory.

For the second bullet, this commandline will add up the percentages of memory used by the processes: ps -e -o pmem | awk 'BEGIN { mem = 0; } NR != 1 { mem = mem + $0 } END { print mem; }' — lgeorget, Jul 18 '14 at 15:29

klerk · Answer 3 · 2014-07-18T20:08:19.533

You already got your answer with free command so,

free -m
-/+ buffers/cache:       6629       5277

this is the line what you need so those values are real memory used, so you have 5277 of real free memory and that's way you dont see it with top or htop

First field from free:

             total       used       free     shared    buffers     cached
Mem:         11907      10660       1246          0        650       3379

This shows memory which is actually reserved by OS(kernel) and not real used without buffers, you can consult vmstat for more info and mdb -k module to check used memmory by kernel and calculate it with vaulues from free command.

As i already pointed your memory is reserved (cached) by kernel and it stays there until some of the apps request it. So that's why you cannot see which process uses it.

Please give output of cat /proc/meminfo and give us cache field that is actually what is used by kernel!

score -2 · Answer 4 · edited Apr 13 '17 at 12:36

-2

You have 1246 MB of completely free memory, BUT the 5277 mb you see is without 'cached' memory. This means that the OS has put some stuff in your memory to be quicker. It calls this "used" (therefore your 'free' number is only 1246), but it is actually available for you if you need it.

see this answer: `free`: output format

edited Apr 13 '17 at 12:36

Community

1

answered Jul 18 '14 at 14:52

Rabin

3,883
1
22
23

2

@lgeorget Incorrect answer. The OP is talking about the number minus cache ("currently at 6631 / 11907 MB of RAM","How can I find out what is using ~50% of the memory"). Somebody else has evidently downvoted you for this already, I won't double that. – goldilocks Jul 18 '14 at 14:54
@goldilocks indeed, I don't know what I exactly was looking at but you're obviously right – lgeorget Jul 18 '14 at 14:58

What is using memory?

4 Answers4