11

I reckon it's not such an uncommon problem: one process allocates massive amounts of memory (be it due to a memory-leak bug, because you try to process an infeasibly large input file, or whatever). The RAM fills up, and at some point Linux has to switch over to swap. Well, sometimes this is just the last resort: if I have an expensive computation going, I do not want to lose data if towards the end I run out of RAM.

Rather more often however (in my experience), the memory consumption is unbounded, by a rogue, perhaps buggy process. I.e., I do not just end up with some less urgently needed data moved to swap, but the OS is forced to panically swap loads of data. And that unfortunately does not just heavily break the offending process, but can bring the whole system to almost a standstill (it's not quite as bad anymore on machines with SSD, but OTOH it makes me worry whether writing gigabytes and gigabytes of garbage data may in the long term harm the flash cells).
Until I notice the problem and manually kill the process (once it actually took minutes until I even got myself logged into a virtual terminal!), half my running session is in swap, and I need to wait quite a while until the system runs smooth again.

There is one draconic solution to the problem: enforce a hard memory limit. But doing this system-wide would sometimes kill processes that I rather need still, and if I have to manually ulimit before starting an offending process... well, I'll often forget until it's too late.

Possible kinds of solution I'd be happier with:

  • If any process exceeds a certain memory usage, it's artificially throttled down so the rest of the system stays responsive.
  • If any process exceeds a certain memory usage, it's SIGSTOPped so I have time to figure out what to do next.
  • If a process approaches the RAM limit, I get a warning, before the great swapping starts.

Is there any way to get such a behaviour, or similar?

leftaroundabout
  • 664
  • 2
    You can limit the amount of RAM used by a process or group of processes, by using cgroups. http://stackoverflow.com/questions/3043709/resident-set-size-rss-limit-has-no-effect/6365534#6365534 – Mark Plotnick Nov 25 '15 at 20:35
  • 2
    This is literally the exact sort of thing ulimit is for. – DopeGhoti Nov 25 '15 at 21:07
  • 1
    Ulimit -m would be the thing to use, except it hasn't worked on Linux since 2.4.30, and only worked in certain situations before that. http://unix.stackexchange.com/questions/129587/does-ulimit-m-not-work-on-modern-linux – Mark Plotnick Nov 25 '15 at 21:34
  • niceload --noswap yourprg – Ole Tange Nov 26 '15 at 07:57

3 Answers3

6

niceload --noswap yourprg is made for exactly that situation: It looks at swapping activity:

  • If swapping out: Let process run
  • If swapping in: Let process run
  • If swapping in and out: Suspend process until swapping stops and resume the process when swapping has stopped

It does not suspend the process before the swapping starts, but lets swapping run for 1 second before acting.

niceload --mem 1G yourprg works similar: If less than 1GB is free youprg is suspended. When more then 1GB is free yourprg is resumed.

Ole Tange
  • 35,514
  • I would have to experiment with this command for a game that runs inside of Wine. How would I go about limiting the memory of that particular game inside of Wine? I ask this because simply running the command for Wine itself wouldn't fix it, since the game creates its own separate process with separate PID. – Winampah May 19 '21 at 09:55
0

Yes. It is quite easily done with practically any modern shell.

sh -c 'ulimit -S -m "$1"
       shift; exec your_command "$@"
' -- some_size args

You can use the -l option for locked memory limits. Your process will be signalled if the limit is exceeded.

mikeserv
  • 58,310
-1

Cronjob to clear the cache: How to clear memory cache in Linux

I have actually similar problems. I have a bunch of users who runs their own custom scripts and every once in a while their scripts consumes all available memory and brings the redhat server down. The reason for the mass consumption of RAM was that their scripts can run for days just waiting on a event, thereby hogging resources when in reality it is not using any. So what I did was just simply force clear the cache with a cronjob and haven't had a problem since.

Simple and lazy.

Community
  • 1
D.Zou
  • 405
  • Hm. Interesting suggestion, though I'm afraid it does nothing for most of the problems I'm experiencing. My main issue is with single processes (scientific applications) which can rapidly allocate many gigabytes of memory. – leftaroundabout Nov 25 '15 at 22:19
  • Clearing caches usually introduce performance problems. Very occasionally they solve performance problems. But they won't cause memory to become available when it isn't there. – Gilles 'SO- stop being evil' Nov 25 '15 at 23:29