2

I have an Oracle Solaris 11.3 which has 128 GB of Memory (RAM) and 80 GB of Swap.

I want to limit the use of swap space (not its size just the usage).

I know in more modern linux distros like Ubuntu, Fedora, CentOs , ... there is a file under /proc/sys/vm/swappiness which you can edit and change its number from 0 to 100. 0 means less usage and 100 means more usage from swap space. But unfortunately there is no directory in /proc named as sys (either on solaris 10 and 11.3).

For now:
Where should I find /proc/sys or swappiness file? If this file does not exist on solaris How should I limit the use of swap?

PS:
At the moment my memory usage is about 30% but whenever I start a new process it allocates swap to it and does not start the process correctly.

echo "::memstat" | mdb -k result is as follows:

Page Summary                 Pages             Bytes  %Tot   
----------------- ----------------  ----------------  ----   
Kernel                      852964              6.5G    5%   
ZFS Metadata                156226              1.1G    1%   
ZFS File Data              2675261             20.4G   16%   
Anon                       4342304             33.1G   26%   
Exec and libs                 1764             13.7M    0%   
Page cache                   28121            219.6M    0%   
Free (cachelist)             18607            145.3M    0%   
Free (freelist)            8687248             66.2G   52%   
Total                     16777216              128G   

and swap -l result is:

swapfile             dev    swaplo   blocks     free   
/dev/zvol/dsk/rpool/swap 303,1        16  8388592  8388592

and prstat -Z result is:

ZONEID    NPROC  SWAP   RSS MEMORY      TIME  CPU  ZONE
 0        96     97G    33G    26%  26:29:09  0.3% global
Vahid F
  • 143
  • Have you try to redirect TMPFS to different place to free some RAM/Swap usage? – Romeo Ninov Jan 05 '19 at 06:46
  • My /tmp is about 2.0 GB and only 4% used. I'm not seeking a temporary answer because this machine is under high load. – Vahid F Jan 05 '19 at 06:51
  • What is result of swap -l and echo "::memstat" | mdb -k – Romeo Ninov Jan 05 '19 at 06:56
  • Edit your question, not answer. And crop the listing to the most consuming programs – Romeo Ninov Jan 05 '19 at 08:48
  • 1
    Doesn't look like anything is using swap. Are you sure it's not better to let the kernel do its thing? – Chris Davies Jan 05 '19 at 09:37
  • The problem is that I can't start new processes because my swap(100 GB) is full and the memory is 70% free!! – Vahid F Jan 05 '19 at 10:47
  • Your swap is empty! and memory is 52% free! and Your swap is 4GB, not 100G: https://docs.oracle.com/cd/E26502_01/html/E29031/swap-1m.html – Romeo Ninov Jan 05 '19 at 11:40
  • But prstat -Z command tells me that my swap is 97GB Full. I add it to my question so please take a look! – Vahid F Jan 05 '19 at 11:53
  • Have a look at the Swapping-Related Parameters in the Oracle Solaris Tunable Parameters Reference Manual. It is available online. – fpmurphy Jan 05 '19 at 13:16
  • @VahidF The problem is that I can't start new processes because my swap(100 GB) is full and the memory is 70% free!! That's because Solaris doesn't play "I hope this will work" games with overcommitting memory. What kind of process(es) are you trying to start? Also see https://unix.stackexchange.com/questions/151139/lots-of-free-memory-but-java-wont-start-sunos-5-10 You likely have a lot of processes that allocate a lot of memory but don't actually use it - yet. Those allocations cause the OS to reserve swap space and there is no tuning you can do to prevent that. – Andrew Henle Jan 05 '19 at 15:22
  • This question: https://unix.stackexchange.com/questions/32857/is-swap-an-anachronism?rq=1 and this answer: https://unix.stackexchange.com/a/32877/111943 are also relevant. – Andrew Henle Jan 05 '19 at 16:26
  • @AndrewHenle I have about 20 java microservices that each of them are configured to use about 5 GB of memory. I have 128 GB of total Memory and I have tested this scenario on another Solaris server with the same configuration. The problem is that on system.d linux distros like cent, fedora , ... you have an option to specify swappiness of your operating system but on system.v distros like solaris I cant find this os parameter. – Vahid F Jan 06 '19 at 10:29
  • @AndrewHenle I'm still wondering because Oracle has stated in this article that the swappiness exists on Solaris but I can't find it: https://docs.oracle.com/cd/E24290_01/coh.371/e22838/tune_perftune.htm#COHAG223 – Vahid F Jan 06 '19 at 10:29

1 Answers1

3

I want to limit the use of swap space (not its size just the usage).

You can not limit swap reservations in Solaris.

Just about every byte of RAM used by a process is guaranteed by Solaris to have some sort of persistent backing store available. (There are some exceptions to that on Solaris, such as some types of shared memory, which do not need swap reservations because they can't be swapped out. Implementations like Oracle database SGAs use these features.)

Memory-maped files, such as executables and shared objects, usually use the actual file on disk as the backing store, and thus don't use swap space at all. Most other memory usages need a backing store. For example, if a process asks the kernel for a 2 GB heap but never actually uses it, that will cause a 2 GB reservation against swap space, because Solaris guarantees that if a process asks for memory, it will get it. There's no OOM killer that kills your database process on your production database server or your web server on your online web server processing customer orders...

On Solaris, if you ask for memory and the kernel says you can have it, you get it. Even if you don't actually use it until after a long, long period of time. That means if you ask for it, the kernel has to make sure there's a place to put it if it has to be swapped out in the future for any reason.

There's a cost to running under a "I said you can have this memory, and that means you WILL be able to access it no matter what" paradigm instead of a "I said you can have this memory but I lied to you and now that you've tried to use it I'm going to kill you" paradigm, and that cost is what might seem like exorbitant swap usage.

To see the swap usage of a process, you can use the pmap -S command:

bash-4.1$ pmap -S $$
4622:   /usr/bin/bash
         Address     Kbytes       Swap Mode Mapped File
0000000000400000       1412          - r-x----  bash
0000000000571000         40         40 rw-----  bash
000000000057B000         24         24 rw-----  bash
0000000EC09E6000        236        236 rw-----    [ heap ]
00007FF0C8590000        304          - r-x----  libcurses.so.1
00007FF0C85EC000         20         20 rw-----  libcurses.so.1
00007FF0C85F1000         16         16 rw-----  libcurses.so.1
00007FF0C8600000       6756          - r-x----  en_US.UTF-8.so.3
00007FF0C8CA9000          8          8 rw-----  en_US.UTF-8.so.3
00007FF0C8CD0000         32          - r-x----  libgen.so.1
00007FF0C8CE8000          4          4 rw-----  libgen.so.1
00007FF0C8CF0000         64         64 rwx----    [ anon ]
00007FF0C8D10000         64         64 rwx----    [ anon ]
00007FF0C8D2D000          4          - rwxs---    [ anon ]
00007FF0C8D30000         64          - r-x----  methods_unicode.so.3
00007FF0C8D40000          4          4 rw-----  methods_unicode.so.3
00007FF0C8D50000         24         24 rwx----    [ anon ]
00007FF0C8D60000       1816          - r-x----  libc.so.1
00007FF0C8F36000         68         68 rw-----  libc.so.1
00007FF0C8F47000          8          8 rw-----  libc.so.1
00007FF0C8F50000         64         64 rw-----    [ anon ]
00007FF0C8F6C000        352          - r-x----  ld.so.1
00007FF0C8FD4000         16         16 rwx----  ld.so.1
00007FF0C8FD8000          4          4 rwx----  ld.so.1
FFFF80E6271AF000         20         20 rw-----    [ stack ]
---------------- ---------- ----------
        total Kb      11424        684

Note this line:

0000000000400000       1412          - r-x----  bash

The Swap column indicates that particular memory mapping requires no swap usage at all. That's the part of the bash executable that's backed by the on-disk /usr/bin/bash file itself.

But this line:

0000000000571000         40         40 rw-----  bash

uses 40 kb of swap. It's likely a data segment mapped from /usr/bin/bash, but since it's modifiable data (note the rw permissions), the backing store can't be the non-modifiable /usr/bin/bash disk file, so it gets a 40 kb swap reservation.

Note that all of the heap, anon, and stack mappings are backed by swap space in their entirety.

Bottom line:

On Solaris, if you don't want to use up all your swap space to the point where you can't start new processes, don't have your processes ask for memory they don't actually use.

Or make a bigger swap partition.

Andrew Henle
  • 3,780
  • Dear andrew please take a look at this article: https://docs.oracle.com/cd/E24290_01/coh.371/e22838/tune_perftune.htm#COHAG223 If you are right so why oracle has introduced this article in its tuning section? – Vahid F Jan 06 '19 at 10:34
  • @VahidF The Linux tuning of "swappiness" has nothing to do with swap reservations on either Linux or Solaris. You can't "tune" swap reservations - if your process asks for a 16 GB heap, that requires a 16 GB swap reservation. On Linux you get away with not having enough swap space to ensure all memory has a proper backing store because by default Linux does memory overcommit - Linux LIES to you about whether or not you can REALLY use all that memory. Solaris doesn't lie about that - if you ask for it, you WILL get it. Which means if there isn't enough swap, your process fails. – Andrew Henle Jan 06 '19 at 13:12
  • @VahidF Again, you have two solutions for this: 1. Stop having your processes ask for huge amounts of memory that they don't actually use 2. Add more space to your swap partition(s). Period. Full stop. That's it. You can't tune your way out of that. – Andrew Henle Jan 06 '19 at 13:19
  • Is there any way to add swap space on solaris without reinstalling OS? – Vahid F Jan 06 '19 at 13:45
  • @VahidF This describes one way: https://docs.oracle.com/cd/E53394_01/html/E54801/ggvlr.html Reading the swap man page (via man swap) should also help. – Andrew Henle Jan 06 '19 at 13:54