13

I have 3 processes on a single core machine. Each process is exactly the same and burns CPU as fast as it can (tight loop). Two of them (A and B) are launched with cgexec in separate groups with shares set to 512 and 512. The third (C) is launched regularly, not with cgexec.

Once all are up and running, C gets 66% of the CPU while A and B split 33%. If I kill C then A and B get 50% each (as expected).

Why does C get 66%? I would have expected 33% each in this scenario, or maybe 50% (C), 25% (A) and 25% (B). 66% going to C doesn't work out no matter how I do the math though.

In general, I want to understand how processes launched without cgexec interact with processes launched with cgexec when it comes to resource sharing (CPU in particular, but a more general answer would be appreciated if it isn't too complex).

Micah Zoltu
  • 231
  • First of all, I wonder how do you measure CPU usage percentage? What are the priorities you run A, B and C? – MAQ Feb 27 '16 at 12:22
  • I believe I was measuring CPU usage with top and I believe they were started fairly simply from a command prompt: cgexec -g cpu:foo myprogram and ./myprogram. It has been a while so I don't remember with certainty. – Micah Zoltu Feb 27 '16 at 18:28
  • Can it be the case that myprogram uses more than single thread /process? Btw are you still interested in solving this problem? – MAQ Feb 27 '16 at 20:24
  • The program was a test application written just to test this behavior. It was intentionally single threaded to narrow the results. I am still interested in an answer to this question. – Micah Zoltu Feb 27 '16 at 21:51
  • This page from RedHat may be of assistance here. cpu.shares is a very peculiar configuration option; I recommend splitting things up by CPUs if possible using cpuset.cpus instead. – Wildcard May 31 '16 at 21:27

1 Answers1

5

Cgroups are hierarhical and they are inherited by all subprocesses. So all processes must be in some cgroup. By default it's the root cgroup and by default it has 1024 shares which is twice as A and B in Your example.

CPU time is shared betwean cgroups according to weight assigned to them in cpu.shares.

If A had 1024 shares and B 512 and C 256 and D 256, the cpu time distribution whould be A - 50%, B - 25%, C and D 12,5%.

Lazy404
  • 181
  • So the any process not launched with cgexec is in the root cgroup with 1024 shares, divided evenly among all immediate children. One such immediate child is the cgroup generated by calling cgexec. So the non-cgexec process would get 50% and the cgrouped processes would all share the remaining 50%. Within the cgrouped processes they share their 50% evenly, meaning both get 25%. This would make sense, but it isn't the behavior I observed. What I saw is 66%, 33% and 33%. Can you update the answer to include more details and perhaps an example distribution? – Micah Zoltu Jun 01 '16 at 01:40
  • Oh, I think I see. Any process on the OS launched without cgexec gets 1024 shares. Any process launched with cgexec gets the shares specified. So in this case, one process gets 1024 shares and the other two get 512 each, resulting in the distribution I saw.

    Would you mind updating your answer to give a little more clarity, e.g., with an example?

     – Micah Zoltu Jun 01 '16 at 01:42
  • Actually, that still doesn't add up. 1024 + 512 + 512 = 2048. 512/2048 == 25%. I don't believe this answer is actually correct, at least not on the surface. – Micah Zoltu Jun 02 '16 at 23:51
  • do You have any other cgroups ? It might affect the numbers. You can also check redhat docs, ther's a simmilar example https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html#sect-cfs – Lazy404 Jun 03 '16 at 00:42
  • Nothing else of significance running, the box was idle before and after the test run. All of the examples only show what happens when all processes are started with cgroups setup. It doesn't discuss what happens when some processes aren't using cgroups, which is what I am trying to figure out. – Micah Zoltu Jun 03 '16 at 02:03
  • @MicahZoltu is it possible your system is configured to assign 2048 shares to the root cgroup? – stewbasic Dec 11 '19 at 05:30
  • @stewbasic Unfortunately this was from a few years ago and I no longer remember the setup that lead to the question. I am reasonably confident that things were "default" for the given OS/distribution, whatever that means. – Micah Zoltu Dec 12 '19 at 06:28