Is there a way to limit the amount of memory a particular process can use in Unix?

Question

I need to test a process for memory management.

• I do not have the source, so I need to do all the testing from the OS side.
• I want to be able to say something like limitmemory 400k -p <pid>

Is there a way to do this in unix? Any common unix utility would be excellent.

Answer 1

ulimit -v, it's a shell builtin, but it should do what you want.

I use that in init scripts sometimes:

ulimit -v 128k
command
ulimit -v unlimited

It seems however, that you want ways of manipulating the maximum allocatable memory while the program is running, is that correct? Probably something like renice for manipulating the Priority.

There is, however, no such tool to my knowledge.

Answer 2

On Linux systems with kernel >=2.6.36 and util-linux >=2.21, you can use the prlimit command to set a process resource limits:

prlimit --as=400000 --pid <pid>

This will restrict the max amount of virtual memory the process can use to 400000. You can find the full list of resources you can limit for a given process in man 2 prlimit.

As noted in the comments and in the manpage, limiting the amount of RSS does not work in kernels >2.4.0 and <2.4.30.

Answer 3

On Linux systems you can use the memory controller from Control Groups version 1 or version 2.

systemd makes it easy to control resource usage, especially on systems with cgroups-v2:

MemoryHigh=bytes

  Specify the throttling limit on memory usage of the executed processes in this unit. Memory usage may go above the limit if unavoidable, but the processes are heavily slowed down and memory is taken away aggressively in such cases. This is the main mechanism to control memory usage of a unit. […]

MemoryMax=bytes

  Specify the absolute limit on memory usage of the executed processes in this unit. If memory usage cannot be contained under the limit, out-of-memory killer is invoked inside the unit. It is recommended to use MemoryHigh= as the main control mechanism and use MemoryMax= as the last line of defense. […]

MemoryLimit=bytes

  Specify the limit on maximum memory usage of the executed processes. […]

  This setting is deprecated. Use MemoryMax= instead.

If instead of creating and customizing a full blown service, you just want to run an ad-hoc command, then you can use the systemd-run utility as suggested by user Hi-Angel:

systemd-run --scope -p MemoryMax=1G firefox

--scope

  Create a transient .scope unit instead of the default transient .service unit (see above).

--property=, -p

  Sets a property on the scope or service unit that is created. This option takes an assignment in the same format as systemctl(1)'s set-property command.

Answer 4

To set the limit when starting the program, use ulimit -v 400, as indicated by polemon. This sets the limit for the shell and all its descendants, so in a script you might want to use something like (ulimit -v 400; myprogram) to limit the scope.

If you need to change the limit for a running process, there's no utility for that. You have to get the process to execute the setrlimit system call. This can often be done with a debugger, although it doesn't always work reliably. Here's how you might do this with gdb (untested; 9 is the value of RLIMIT_AS on Linux):

gdb -n -pid $pid -batch -x /dev/stdin <<EOF
call setrlimit(9, {409600, -1})
detach
quit
EOF

Answer 5

If you're using systemd, you can set some additional options in a .service file. The complete list of the options you can set is described here.

Below is a short example that shows how to use this feature of systemd:

# cat /etc/systemd/system/qbittorrent-nox.service
[Unit]
Description=qbittorrent-nox
Documentation=man:qbittorrent-nox
DefaultDependencies=yes
Requires=media-Kabi.mount
After=media-Kabi.mount network-online.target
Before=multi-user.target
Conflicts=umount.target

[Service]
User=morfik
Group=p2p
Type= simple
RemainAfterExit=no
ExecStart=/usr/bin/qbittorrent-nox
Nice=19
IOSchedulingClass=idle
PrivateNetwork=no
CPUShares=256
MemoryLimit=50M
BlockIOWeight=128
Slice=p2p.slice
StandardError=null
StandardOutput=null

[Install]
WantedBy=multi-user.target

Of course, you don't need all of the options I used. If you just want to limit memory usage, add MemoryLimit=50M, which limits to 50MiB.

And this is the result:

# systemctl status qbittorrent-nox.service
● qbittorrent-nox.service - qbittorrent-nox
   Loaded: loaded (/etc/systemd/system/qbittorrent-nox.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2015-05-07 19:25:20 CEST; 1s ago
     Docs: man:qbittorrent-nox
 Main PID: 21712 (qbittorrent-nox)
   Memory: 9.4M (limit: 50.0M)
   CGroup: /p2p.slice/qbittorrent-nox.service
           └─21712 /usr/bin/qbittorrent-nox

May 07 19:25:20 morfikownia systemd[1]: Started qbittorrent-nox.
May 07 19:25:20 morfikownia systemd[1]: Starting qbittorrent-nox...

So far, this works only for system daemons/services, and you can't limit, for instance, a firefox processes as a regular user in this way. But maybe this will change some day.

Answer 6

There is the setrlimit() function, which allows to configure a process' limits in C. Write a C program to call setrlimit then to exec the command you want to be limited. setrlimit cannot change other processes' limits.

Luckily someone already wrote something similar. It can be downloaded from freshmeat. I had a quick look at the source code and it seems to be fine. Use rlimit at your own discretion. Note that rlimit also cannot change other processes' limits.

Edit: Gilles proposed a nice hack with gdb: Attach to the process with gdb then make the process call setrlimit. This would perhaps solve the problem to limit an already running process.

Answer 7

If you just want to test and measure the memory usage of your program please look at time. You can measure the resource usage of your program in many aspects including the terms of CPU Time and memory usage. Following command will give you the memory usage and CPU time usage of myProgram:

/usr/bin/time myProgram

(Be sure to give the absolute path to distinguish it from bash built-in time command.)

If you just want to limit the resources of your process, i recommend you to create a test user for this specific task. Limit the resources of this user according to your needs and run the process by the user. It seems that, in *nix world, resource management based on users is much more advanced than resource management based on processes.

You can check /etc/security/limits.conf to limit the resources of a user. Or you can use ulimit after logged in with the to-be-limited user.