3

Linux's disk caching (page cache) is used to cache file data and uses as much RAM as possible. After upgrading my server's RAM I was able to download huge files faster than my disks would allow. But only if the file has been downloaded once before which is absolutely logical for me:

First download:
enter image description here

Second download (only partial from cache?):
enter image description here

Third download:
enter image description here

But does it mean, that a file that is as big or bigger as my buffer size, that it will overwrite all the nice cached files that are accessed much more often? Or is there a smart mechanism that "re-crawls" those "hot" files after the huge file is not accessed anymore?

mgutt
  • 467
  • 1
    But does it mean, that a file that is as big or bigger as my buffer size, that it will overwrite all the nice cached files that are accessed much more often? That's exactly what will happen. That's one good reason for using direct IO for something where you don't reread the same data, like downloading a large file - by bypassing the page cache you don't evict other data from the cache. – Andrew Henle Sep 05 '19 at 10:20
  • @AndrewHenle It seems you are wrong ;) – mgutt Sep 07 '19 at 12:22
  • That's why it was an off-the-top-of-my-head comment, and not a full answer. Note that the file you downloaded wasn't an order of magnitude or more larger than the RAM on your system. Look at it this way: if you read every byte of a 200 GB file on a system with 4 GB of RAM with a full page cache and you know you won't read those bytes again, you will evict a lot of data that might be read again from the cache. Just because it hasn't been read again yet doesn't mean it wouldn't have been. – Andrew Henle Sep 07 '19 at 13:31

1 Answers1

6

But does it mean, that a file that is as big or bigger as my buffer size, that it will overwrite all the nice cached files that are accessed much more often?

No, the cache is somewhat smarter than that. Pages in the page cache are tracked in two lists, the inactive list and the active list. When a page is faulted in (i.e., data is read from disk), it is initially added to the inactive list; if it’s accessed again, it is promoted to the active list. Pages are only evicted from the inactive list.

In particular, this means that large files which are read once won’t evict smaller files which are used more than once.

This also explains the behaviour you saw. When you first downloaded the large file, it was read into the cache, but progressively evicted as the web server progressed through the file. Your second download thus didn’t start in the cache; but it eventually caught up with pages which were still in the cache. This second download resulted in the corresponding pages becoming more desirable to keep in the cache, and your third download found everything in the cache.

You’ll find a detailed explanation of this approach in the kernel source code.

Or is there a smart mechanism that "re-crawls" those "hot" files after the huge file is not accessed anymore?

This however is the sort of smart mechanism which the kernel doesn’t have (a similar example is the persistent idea that the kernel moves unused pages to swap when the system is quiet; it doesn’t). The kernel doesn’t try to predict the future much; one of the exceptions to this “rule” is that it does perform read-ahead on block devices, but that’s about it.

Stephen Kitt
  • 434,908