2

How to setup limit for process to memory usage?

Similar to open files limit in /etc/security/limits.conf:

ubuntu           soft    nofile          4096
ubuntu           hard    nofile          8192

E.g. while I'm launching python script with raw eval of json data from 1.1G file, python takes whole of RAM, while creating objects around each dict and list in json.txt. It hung my machine for 20-30 minutes. Thereafter:

# python read_data.py
Killed

Ubuntu System is very stable today. Its recovery from hung. Swap going to 8Gb of usage. RAM completely empty, script going off.

I'm trying to find, is this limit, which killed my script configurable? Can I tune my system in such way, that every process, which takes more than 70% of current RAM size would be just killed or stopped or something.

youblind
  • 23

1 Answers1

1

Using the same file limits.conf you can limit:

  • data - max data size (KB)
  • memlock - max locked-in-memory address space (KB)
  • rss - max resident set size (KB)
  • as - address space limit (KB)
  • stack - max stack size (KB)

So it's pretty much you can do. After setting up limits you can check this level with ulimit -a → you can also use this program to set them. To use limits.conf you need pam_limits.so.

And last thing is what killed your script. I think it could be OOM killer. A feature in linux kernel which kills misbehaving processes when they eat all real memory ;) Use dmesg to find out if killer was used. There will be lots of info about candidates to kill and what process precisely was killed.

Cheers,

Scyld de Fraud
  • 2,692
  • Good answer but you might also want to mention the as (address space) limitation. – Bratchley Aug 29 '14 at 11:36
  • Thank you, i'm found by myself: ulimit -v 4000000 protect my system from hunging, while I'm playing with python. – youblind Aug 29 '14 at 11:41
  • Note that you cannot limit RSS. http://linux.die.net/man/2/setrlimit - "This limit only has effect in Linux 2.4.x, x < 30". – phemmer Aug 29 '14 at 14:25