4

we have moon server - version rhel 7.5

the behavior of consuming swap on this server is very strange

we configured the /proc/sys/vm/swappiness to 1 and restart the server

but we can see that server is eating 15G , when the available is 44G !!

HOW IT CAN BE??

[root@moon01 network-scripts]# more /proc/sys/vm/swappiness
1
[root@moon01 network-scripts]# free -g
              total        used        free      shared  buff/cache   available
Mem:            125          80          38           0           6          44
Swap:            15          15           0

from my understanding only if available is near last few giga ram then swap will increased

but this isn't the situation

yael
  • 13,106
  • But is this straight after restart (which is not necessary for swappiness change), or after some time of server action? With 80GB USED it looks like the second. So there could have been a spike, and the 15 GB swap are just lying there at the moment. –  May 17 '20 at 07:46
  • @rastafile no spike is required, see here. Linux will start swapping a long time before "free" hits zero. – Philip Couling May 17 '20 at 07:51
  • You had this 24-hour phase of activity. Same result. –  May 17 '20 at 07:56
  • @rastafile yes and provably no spike large enough to cause it. A low memory process running a lot of disk IO can cause it (as it did for me). There's been a few other question here on unix.so who've all had the same result. – Philip Couling May 17 '20 at 08:00

1 Answers1

6

Even with swappiness=1 Linux will continue to use swap if available. Your user-space programs do not need to exhaust free RAM for the kernel to start swapping. I first discovered this because I was having problems on an Ubuntu Linux desktop.

In answers and comments to my question, someone pointed out that disk caching is the probable cause. "Free" space in memory is very rarely empty. The kernel will quietly use it for caches including disk caches, safe in the knowledge that it can abandon the cache whenever applications require more memory.

I'm hunting for the reference in the kernel docks. But there's a good description somewhere of the way the majority of programs will have a lot of memory (including code) which is only used during startup and then never again. So particularly on a server, you will have a lot of "junk" sitting in memory, stealing space from useful things like disk caches. Linux knows this and will chose to swap out the junk rather than abandon pages from a disk cache. Over all this has the effect of gradually increasing the swap usage and slowly growing the size of caches. It does this even though "free" memory remains relatively high.

In short, this is expected behaviour and there's no simple way to turn it off.

Philip Couling
  • 18,733