So, I'm trying to do some investigation on where does swap use come from in a system with high swap usage:
# free
total used free shared buffers cached
Mem: 515324 508800 6524 0 4852 27576
-/+ buffers/cache: 476372 38952
Swap: 983032 503328 479704
Adding up swap used per process:
# for proc in /proc/*; do cat $proc/smaps 2>/dev/null | awk '/Swap/{swap+=$2}END{print swap "\t'`readlink $proc/exe`'"}'; done | sort -n | awk '{total+=$1}/[0-9]/;END{print total "\tTotal"}'
0 /bin/gawk
0 /bin/sort
0 /usr/bin/readlink
28 /sbin/xxxxxxxx
52 /sbin/mingetty
52 /sbin/mingetty
52 /sbin/mingetty
52 /sbin/mingetty
56 /sbin/mingetty
56 /sbin/mingetty
60 /xxxxxxxxxxx
60 /usr/sbin/xxx
84 /usr/sbin/xxx
108 /usr/bin/xxx
168 /bin/bash
220 /sbin/init
256 /sbin/rsyslogd
352 /bin/bash
356 /bin/bash
360 /usr/sbin/sshd
496 /usr/sbin/crond
672 /usr/sbin/sshd
12972 /opt/jdk1.6.0_22/bin/java
80392 /usr/libexec/mysqld
311876 /opt/jdk1.6.0_22/bin/java
408780 Total
Which gives a lower value for total used swap. Where is the remaining used swapspace? Is it vmalloc()'ed memory inside the kernel? Something else? How can I identify it?
Output of meminfo:
# cat /proc/meminfo
MemTotal: 515324 kB
MemFree: 6696 kB
Buffers: 5084 kB
Cached: 28056 kB
SwapCached: 157512 kB
Active: 429372 kB
Inactive: 65068 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 515324 kB
LowFree: 6696 kB
SwapTotal: 983032 kB
SwapFree: 478712 kB
Dirty: 100 kB
Writeback: 0 kB
AnonPages: 399456 kB
Mapped: 8792 kB
Slab: 7744 kB
PageTables: 1820 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 1240692 kB
Committed_AS: 1743904 kB
VmallocTotal: 507896 kB
VmallocUsed: 3088 kB
VmallocChunk: 504288 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 4096 kB
mm/swapfile.c
:free_swap_and_cache()
callsswap_entry_free()
which incrementsnr_swap_pages
(SwapFree
), but only callsdelete_from_swap_cache()
(which decrementstotal_swapcache_pages
-SwapCached
) when the swap page is not still mapped elsewhere or the swap is full. – ninj Apr 08 '13 at 16:06task
directory containing few more running tasks. They also consume swap spaces. Try adding them too.e.g. /proc/12345/task/12346/smaps
– SHW Oct 21 '13 at 07:27readlink
and is causing parse errors in yourawk
script, and you are effectively not counting processes whose binaries are no longer present in your total. The ongoing discussion about swap/caching/etc. going on here is a red herring. The more accurate question would have been "Why is my measurement of total swap usage different than whatfree
reports?". – Jason C Oct 23 '13 at 00:55