Actual memory usage of a process

Question

The following are the memory usage of mysql and apache respectively on my server. As per the output of pmap say, mysql is using about 379M and apache is using 277M.

[root@server ~]# pmap 10436 | grep total
 total           379564K

[root@server ~]# pmap 10515 | grep total
 total           277588K

Comparing this with the output of top, I see the values are almost matching.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
10515 apache    20   0  271m  32m 3132 S  0.0  6.6   0:00.73 /usr/sbin/httpd
10436 mysql     20   0  370m  21m 6188 S  0.0  4.3   0:06.07 /usr/libexec/mysqld --basedir=....

Now these values definitely are not the current memory usage of those two processes, since if it were it would've exceeded the 512M ram on my system and I understand the fact that these are the size of the pages assigned to these two processes and not really the size of the memory actively used by them. Now, when we use pmap -x, I see an extra coloumn Dirty which shows far less memory usage for the process. As seen in the example show below, the Dirty coloumn shows 15M as opposed to 379M in the first coloumn. My question is: Is the value under coloumn Dirty is the 'real' amount of memory actively used by that process? If its not, then how can we find out the real memory usage of a process? Not ps and top for the same reasons above. Do we have anything under /proc that will give this info?

[root@server ~]# pmap -x 10436 | grep total
total kB          379564   21528   15340
[root@server ~]#


[root@server ~]# free -m
             total       used       free     shared    buffers     cached
Mem:           489        447         41          0         52        214
-/+ buffers/cache:        180        308
Swap:         1023          0       1023
[root@server ~]#

Answer 1

There is no command that gives the “actual memory usage of a process” because there is no such thing as the actual memory usage of a process.

Each memory page of a process could be (among other distinctions):

• Transient storage used by that process alone.
• Shared with other processes using a variety of mechanisms.
• Backed up by a disk file.
• In physical memory or swap.

I think the “dirty” figure adds up everything that is in RAM (not swap) and not backed by a file. This includes both shared and non-shared memory (though in most cases other than forking servers, shared memory consists of memory-mapped files only).

The information displayed by pmap comes from /proc/PID/maps and /proc/PID/smaps. That is the real memory usage of the process — it can't be summarized by a single number.

Answer 2

I'll cite something I wrote in the man page for an application that does analysis similar to top and drawing information from the same sources as pmap (e.g. /proc/[N]/maps):

VIRTUAL ADDRESS SPACE VS. PHYSICAL MEMORY

It is important to understand the difference between virtual address space and physical memory in interpreting some of the above statistics. As the name implies, virtual address space is not real; it's basically a map of all the memory currently allocated to a process. The limit on the size of this map is the same for each processes (generally, 2-4 GB), and it is not accumulated (ie, you may have dozens or hundreds of processes, each with its own 2-4 GB virtual address space, on a system that only actually has 512 MB of physical memory).

Data cannot actually be stored or retrieved from virtual address space; real data requires real, physical memory. It is the kernel's job to manage one in relation to another. Virtual space stats (VirtualSz, Data+Stack, and Priv&Write) are useful for considering the structure of a process and the relationship to physical memory use, but with regard to amount of RAM actually used, the physical memory stats (ResidentSz, Share, and Proportion) are what counts.

pmap is mostly reporting to you information about virtual address space. Your observation that "the values are almost matching" in top output presumably refers to the VIRT figure, which is very different than the RES figure. These correspond exactly to what above I've labelled "VirtualSz" and "ResidentSz" (the VIRT is for virtual, the RES is for resident).

Now, when we use pmap -x, I see an extra coloumn Dirty which shows far less memory usage for the process. As seen in the example show below, the Dirty coloumn shows 15M as opposed to 379M in the first coloumn. My question is: Is the value under coloumn Dirty is the 'real' amount of memory actively used by that process?

No, but sort of. "Dirty" memory refers to data which has been loaded from disk and subsequently modified; since it has been modified, it must be part of resident memory because these changes are currently stored in RAM. However, it is not synonymous with it.

Answer 3

Virtual memory is like speed dial numbers, except there are around 3 billion or them (for 32 bit system, 4 billion for 32 bit app on 64bit kernel, much more for 64 bit application), and you can not dial numbers direct, they have to be mapped to speed dial.

Several processes can have different mappings (speed dial numbers) for the same address (phone numbers). For example they may share several libraries, so have virtual addresses for the whole library (you can see that in pmap). They may even share the same executable e.g. 2 instances of bash.

So far this explains how the sub of all the virtual address can fit, but there is more. One process can have so much virtual memory that it should not fit, how? Some parts of a library, or executable may not be used, they will not get copied from disk in to ram, or ram gets full and bits that where loaded from disk are dropped, because they can be re-fetched from disk if needed, or memory that is not backed my disk is mapped in to swap, copied out to swap and then dropped. It can then be read from swap if and when needed. If any of these latter strategies are use too much then the system becomes slow.