3

My debian 8 machine has 2 drives. Sometimes I move/copy files from drive A to drive B. Drive B is slow, so the file system cache is full causing the system to be slow/unresponsive when copying large files. To avoid that i wanted to limit the file system cache to be used for device B. Therefore i changed the setting /sys/class/bdi/MAJOR:MINOR/max_ratio from 100 to 20. However that doesn't seem to take effect because dirty pages are still filled the same way as before according to /proc/meminfo.

What am I missing? Do I need to do anything to reload those settings? Is there another way to limit file system cache for a specific device?

I have already tried mount option "sync". That works, but makes drive B terribly slow.

EightBitTony
  • 21,373
Martin
  • 41
  • "However that doesn't seem to take effect." do you mean you know it's not being honoured, or do you mean it doesn't alleviate the issue and so you assume it's not being honoured? – EightBitTony Nov 15 '17 at 10:47
  • I am also experiencing mysterious write slowdown (also for a single drive), and I recently found that echo 3 | sudo tee /proc/sys/vm/drop_caches helps to get the speed back to normal ranges, sometimes immediately, sometimes after a short time. No idea about the root cause; I suspect locking issues for several cores. – dirkt Nov 15 '17 at 10:52
  • @EightBitTony I know it's not being honoured because it still fills dirty pages same as before according to /proc/meminfo. – Martin Nov 15 '17 at 11:06
  • @dirkt As soon as the copy operation has finished, dirty pages will decrease and everything is fine again. So that doesn't seem to be the problem here? – Martin Nov 15 '17 at 11:20
  • Check out https://www.reddit.com/r/linux/comments/4ero1o/if_you_are_sick_of_linux_bogging_down_when/ for a detailed coverage. – Ferenc Wágner Nov 21 '17 at 15:08

1 Answers1

1

These settings only take effect after the total write-back cache has filled up to (dirty_ratio + dirty_background_ratio / 2). References:

  • LKML post by Jan Kara
  • commit 5fce25a9df48 in v2.6.24. "We allow violation of bdi limits if there is a lot of room on the system. Once we hit half the total limit we start enforcing bdi limits..." This is part of the same kernel release that added the internal per-device "limits". So the "limits" have always worked like this, except for pre-releases v2.6.24-rc1 and -rc2.
  • Smarter write throttling by LWN.net - if you want a reference for the original patch series. It does not describe this specific limitation.

For simplicity, let us assume the default values of the vm.dirty* sysctls: dirty_background_ratio = 10 and dirty_ratio = 20. In this case, processes are allowed to dirty pages without getting slowed down, until the system as a whole reaches the 15% point.

sourcejedi
  • 50,249