1

Many people use ps_mem.py script in order to know how much RAM processes use. In this case the result of the script was something like this:

---------------------------------
                        278.4 MiB
=================================

So the whole system uses 278.4 MiB, but free says something really different:

# free
              total        used        free      shared  buff/cache   available
Mem:           1.8G        756M        980M         57M        131M        1.0G
Swap:          2.5G         11M        2.5G
Total:         4.3G        767M        3.4G

So here, the system utilizes 756M. It's not the cache and it's not because of the tmp files.

I also tried:

# echo "3" > /proc/sys/vm/drop_caches

in order to see whether there will be any difference, but nothing changed.

So how to release the pages that are taken for some reason? I have no idea what and why utilizes the space, and I don't know how to recover it. For now the only option is to reboot the machine.

Here's the photo where you can see what processes left. Can you explain the RAM utilization based on that?

enter image description here

Mikhail Morfikov
  • 10,549

1 Answers1

2

Due to complexity of virtual memory management (all of which is for the benefit of using the least amount possible), it's practically impossible to determine how much RAM is actually in use. See this link.

So whatever your python script reports, it won't reflect the actual state.

What's cached is actually free, no worries about that (dropping caches doesn't actually free anything, it just flushes the pages kernel would reuse if it needed them).

What free reports is what the kernel knows about the system. It can't be wrong, as the kernel is the one that's serving the memory. But it doesn't equal the sum of used memory of each individual process, because of various mechanisms: shared memory (libraries), copy-on-write memory (after forking, only pages that are touched are actually duplicated), uninitialized (zeroed out) pages, loaded program code (also shared), virtual memory not corresponding to RAM, swapped out pages, interprocess shared memory, kernel memory (reserved by kernel modules), kernel memory (the main kernel itself, including the page table), and so on...

The point is... if kernel reports pages as used, they must be used for something. If you want to free up some memory, it must come from somewhere: each running process, module and the kernel itself, may have mechanisms to release some memory they may not need now and will reload later (it's up to application writers if they saw the need to implement needed complexity to do that). But the kernel will take care of it if you really need new memory. It'll drop filesystem caches if you request more memory, it will push stale pages to swap if you are using it, if you use zram or something like that, it'll compress pages instead... and at the end, if you really run out of space, OOM killer will find kill something to prevent the system from locking up. But the process is way too complicated for you to think you know better than whatever is going on inside.

orion
  • 12,502
  • So is there no way to recover the space and to know what uses it? You know, I think all the processes that runs right now on my machine are the same when the pc starts. So after starting there's ~300MiB, now 750M+ and I don't know why. – Mikhail Morfikov May 08 '16 at 18:45
  • You can find out who's using how much from ps output, it's just that you don't know what's counted twice, thrice, or maybe zero times (lazy allocation). But it should show you which process is hogging the resources if you observe the change in usage over time. If it's not that (if your tool really shows realistically all that may be used by userspace processes), then it may be some kernel module. In that case, read up on this: https://bbs.archlinux.org/viewtopic.php?id=108421 – orion May 08 '16 at 18:48
  • http://stackoverflow.com/questions/662526/memory-usage-of-a-kernel-module – orion May 08 '16 at 18:48
  • It's none of it. I just updated the question. There's a picture from TTY depicting processes that left. I just wanted to kill every process that would free the RAM space, and I failed. Any idea? – Mikhail Morfikov May 08 '16 at 19:14
  • Run lsmod and tell us which module uses the most memory? – orion May 08 '16 at 19:27
  • I had to reboot the machine and I have no idea what module could that be if any. When the problem occurs the next time, I check that. – Mikhail Morfikov May 08 '16 at 19:34
  • Pay attention to the graphics driver. – orion May 08 '16 at 19:38
  • I have just one question. Does the size of the loaded modules can grow and shrink when a module is loaded? – Mikhail Morfikov May 08 '16 at 19:39
  • Depends on how the module works (the module can free the memory, just like a program can). However, if the module isn't crucial for working of the system and seems to be leaking memory, you can unload it and load it again (briefly disrupting the service it was providing). – orion May 08 '16 at 20:21