How to disable the Linux page cache

Question

The Linux kernel implements the page cache to accelerate I/O operations.

It would be helpful to be able to turn off and on the page cache for research and testing.

How can the Linux page cache be diabled?

Update 1: If it is not possible to turn off the page cache globally, it may be possible to mount Linux file systems in write-through mode. When I understand it right, the mount option dax of EXT2, EXT4, and XFS does implement write-through mode. Is this a valid option to avoid the page cache (at least for a file system)?

https://www.kernel.org/doc/Documentation/filesystems/dax.txt

Update 2: Obviously, DAX is more or less dead, and the page cache cannot be switched off really well (globally, not at all, and for a single application only in a limited manner). But is there really no way to set a Linux file system (e.g., via mount parameter) to write-through mode?

Update 3: The tool dd is one of the application examples that allow easily to bypassed the page cache of the Linux kernel via oflag=direct.

https://man7.org/linux/man-pages/man1/dd.1.html

Update 4: Obviously, file systems that make use of the FUSE (Filesystem in Userspace) module can be used in direct-io mode where the page cache is completely bypassed for reads and writes.

https://www.kernel.org/doc/Documentation/filesystems/fuse-io.txt

Update 5: The mount option -o sync implements write-through mode and bypasses the page cache for some file systems (ext2, ext3, FAT, VFAT, UFS)

https://linux.die.net/man/8/mount

Answer 1

It is not possible, even temporarily, to disable the page cache.

One particular application can however indeed try its best to bypass caches by open_ing files with the O_DIRECT flag set :

   O_DIRECT (since Linux 2.4.10)
          Try to minimize cache effects of the I/O to and from this
          file…
          File I/O is done directly to/from user-space buffers.
          The O_DIRECT flag on its own makes an
          effort to transfer data synchronously, but does not give
          the guarantees of the O_SYNC flag that data and necessary
          metadata are transferred.  To guarantee synchronous I/O,
          O_SYNC must be used in addition to O_DIRECT.

While this implies that on must be able to rebuild the source code modified accordingly, nocache, which works intercepting the open and close system calls can be used to launch a non modified binary.
It has got some limitations and will of course slow the process down and… I never tested it.

---

One of course can always empty the cache using the following command :

# echo 1 > /proc/sys/vm/drop_caches

Answer 2

I think that we shouldn't try to bypass the page cache on a block device like HDDs and SSDs. That's because the granularity of such disks is block (512B, 4KB or something else), however, the r/w system call is performed on bytes.

The inconsistency means that the kernel needs to combine the data that you want to write to the disk with something else that forms the block, and then write it to the disk at once (if we ignore cross-block write, but it will be the same). For read, the kernel will read a whole block that contains the data you want, and then give you the part that you need.

Once we understand this inconsistency, we can find that the page cache is not just a speed-up trick. Instead, the page cache is quite vital on the I/O stack of disks. More precisely, the inconsistency is handled by the page cache and the buffer head, which is a little bit complex.

Anyway, If you want to bypass the page cache, here are some tips:

• DAX: If you are working on a byte-addressable device (like a PMEM or something), DAX is the best choice for you. Because the device itself can be accessed in bytes, the kernel doesn't need to bother with the inconsistency we discussed above. In DAX mode, the file system will simply bypass the page cache and read from / write to the device directly. Note that DAX CANNOT be used on a block device! You can refer to Direct Access for files for more details.

• O_DIRECT: You can also open a file with O_DIRECT flag to bypass the page cache. This is designed for applications that have their own high-performance caches in userspace. However, as far as I know, you can only perform aligned operations on it. Also, this does not ensure that when it returns the data is already on the disk. And the performance can be much lower if you don't have any other cache mechanism implemented. Refer to the man page open(2), or this discussion Clarifying Direct IO's Semantics, or just give it a try. Also, @MC68020 has listed the relevant part.

Further, if you just want to evaluate the performance of a file system without the influence of page cache, you can try these:

• sync: This is for write. Open the file with O_SYNC, then each write will only return when the data is on the disk. open(2) The write performance under O_SYNC will be close to that the page cache is 'disabled' (though there are some extra overheads like copying data to the page cache, the main cost will be disk I/O).

• drop cache: This is for read. As @MC68020 says, you can drop the system cache before you do something. This gives you a blank page cache, so when you read something the file system will fetch it from the disk. The command is like: echo 1 > /proc/sys/vm/drop_caches or sysctl -w vm.drop_caches=1. Check it here /proc/sys/vm, search 'drop_cache' in it. The defect is that the read for a position will only turn to the disk at the first time. The next time it will be in the page cache.

Hope these will help :)