Limit a process's memory by name forever

Question

I am developing using webpack. The problem is, my current project is huge, and takes around 6GB of memory (~30% of RAM according to top).

Is it possible to run a command restricting it's memory persistently? It's important to note, it starts as root (tools we use - I know - it's very weird)

The problem is, that when there is too little memory, my machine just freezes. I am looking for any way to avoid the freezing. So maybe reserve 500mb for the system itself. I'm open to suggestions

Answer 1

When establishing hard and/or soft limits with ulimit [-H|S], once set for a given shell a hard limit cannot be modified by a non-root user. To do so a non-root process owner needs to stop execution, set a new limit and start again.

You'll need to specify the limit within your current project's execution shell. If you do so in a calling shell the limit set will apply to ALL processes started or spawned in that shell, combined. So with that in mind you might be interested in specifically setting the max stack size (ulimit -H -s <limit>) and/or the max resident set size (ulimit -H -m <limit> for max RSS).

RSS is the portion of memory occupied by a process that is held in physical memory.

You may or may not have access to the latter, depending on your system. (I do not have access to an Ubuntu OS right now to check... you might be out of luck.)

A caveat is that setting that limit, the behavior you get is not fully deterministic over replicated executions. Depending on what your system does at any given moment, it may decide to swap chunks of memory at different times from one execution to the next. In other words, it may reach a different values of RSS at different times across runs, e.g. when executing other unrelated processes concurrently.

Apart from the ulimit man page, also look at answer and comments here.

Answer 2

If your project(or the program which runs it) is a systemd service, slice, mount, or socket you will need to set resource limits within its service/slice/mount/socket file rather than setting it using /etc/security/limits.conf or ulimit.

You can find your systemd unit files in /usr/lib/systemd/system/ or /etc/systemd/system depending on your distribution.

To quickly make edits you can use systemctl edit <unit name> to make 'drop-in' edits. Specifying --full with the previous command will allow you to view the full service file while you edit(Essentially editting a copy and a replacing).

For your particular question it would be advisable to set:

MemoryHigh=<Bytes>

You can think of this directive as specifying the throttling point of memory usage for your program. You can add K,M,G, or T as a suffix to the to specify Kilobytes, Megabytes, Gigabytes, or Terabytes respectively.

You can also set:

MemoryMax=<Bytes>

Upon exceeding this directives value, an out-of-memory killer will invoked within the unit. The manpage for regarding these directives(systemd.resource-control) recommends using this as a 'last line of defense`. You may also specify K,M,G,T as with the previous directive.

There's also a deprecated setting that preceded the two above directives called MemoryLimit which may be useful if you're not on a newer systemd release.

The two mentioned directives apply limits to the entire control group, which will prevent forking processes from being able to escape from the limits; However you can also set limits to the singular process using the LimitAS (Limit address space) directive.

An example service file using MemoryHigh= and MemoryMax=

[Unit]
Description=A foobar service that does foo and bar
After=network.target
[Service]
Type=simple
MemoryHigh=2500M
MemoryMax=3000M
ExecStart=/usr/bin/foobar-daemon start
ExecStop=/usr/bin/foobar-daemon stop
[Install]
WantedBy=multi-user.target

For full description of directives or for additional information;

see systemd.resource-control or man systemd.resource-control and systemd.exec or man systemd.exec