397

Prior to doing some benchmarking work how would one free up the memory (RAM) that the Linux Kernel is consuming for its buffers and cache?

Note that this is mostly useful for benchmarking. Emptying the buffers and cache reduces performance! If you're here because you thought that freeing buffers and cache was a positive thing, go and read Linux ate my RAM!. The short story: free memory is unused memory is wasted memory.

Gilles 'SO- stop being evil'
  • 829,060
slm
  • 369,824
  • 1
    What is the interest in open dup question and answer self? http://unix.stackexchange.com/questions/58553/how-to-clear-memory-cache-in-linux & http://unix.stackexchange.com/questions/17936/setting-proc-sys-vm-drop-caches-to-clear-cache – innocent-world Aug 23 '13 at 13:59
  • 2
    @innocent-world - never saw these when I was searching, so it appeared to be a hole in the knowledge base on this site, and was attempting to fill it in. – slm Aug 23 '13 at 14:01
  • 2
    @innocent-world - in looking at those 2 Q's I think there is still room for this Q&A. This one acts as a canonical Q&A on the site. Those are fairly specific in what they're addressing. Additionally this one shows information about analyzing the buffers & cache and also how to set the parameters using sudo. Neither of the other 2 questions address any of this. – slm Aug 23 '13 at 14:09
  • 3
    Please read this article http://www.linuxatemyram.com/ If you understand the risk then you can do free && sync && echo 3 > /proc/sys/vm/drop_caches && free –  Mar 23 '15 at 04:36

1 Answers1

597

Emptying the buffers cache

If you ever want to empty it you can use this chain of commands.

# free && sync && echo 3 > /proc/sys/vm/drop_caches && free

         total       used       free     shared    buffers     cached



Mem:       1018916     980832      38084          0      46924     355764
-/+ buffers/cache:     578144     440772
Swap:      2064376        128    2064248
             total       used       free     shared    buffers     cached
Mem:       1018916     685008     333908          0        224     108252
-/+ buffers/cache:     576532     442384
Swap:      2064376        128    2064248

You can signal the Linux Kernel to drop various aspects of cached items by changing the numeric argument to the above command.

  • To free pagecache:

    # echo 1 > /proc/sys/vm/drop_caches

  • To free dentries and inodes:

    # echo 2 > /proc/sys/vm/drop_caches

  • To free pagecache, dentries and inodes:

    # echo 3 > /proc/sys/vm/drop_caches

The above are meant to be run as root. If you're trying to do them using sudo then you'll need to change the syntax slightly to something like these:

$ sudo sh -c 'echo 1 >/proc/sys/vm/drop_caches'
$ sudo sh -c 'echo 2 >/proc/sys/vm/drop_caches'
$ sudo sh -c 'echo 3 >/proc/sys/vm/drop_caches'

NOTE: There's a more esoteric version of the above command if you're into that:

$ echo "echo 1 > /proc/sys/vm/drop_caches" | sudo sh

Why the change in syntax? The /bin/echo program is running as root, because of sudo, but the shell that's redirecting echo's output to the root-only file is still running as you. Your current shell does the redirection before sudo starts.

Seeing what's in the buffers and cache

Take a look at linux-ftools if you'd like to analyze the contents of the buffers & cache. Specifically if you'd like to see what files are currently being cached.

fincore

With this tool you can see what files are being cached within a give directory.

fincore [options] files...

--pages=false      Do not print pages
  --summarize        When comparing multiple files, print a summary report
  --only-cached      Only print stats for files that are actually in cache.

For example, /var/lib/mysql/blogindex:

root@xxxxxx:/var/lib/mysql/blogindex# fincore --pages=false --summarize --only-cached * 
stats for CLUSTER_LOG_2010_05_21.MYI: file size=93840384 , total pages=22910 , cached pages=1 , cached size=4096, cached perc=0.004365 
stats for CLUSTER_LOG_2010_05_22.MYI: file size=417792 , total pages=102 , cached pages=1 , cached size=4096, cached perc=0.980392 
stats for CLUSTER_LOG_2010_05_23.MYI: file size=826368 , total pages=201 , cached pages=1 , cached size=4096, cached perc=0.497512 
stats for CLUSTER_LOG_2010_05_24.MYI: file size=192512 , total pages=47 , cached pages=1 , cached size=4096, cached perc=2.127660 
stats for CLUSTER_LOG_2010_06_03.MYI: file size=345088 , total pages=84 , cached pages=43 , cached size=176128, cached perc=51.190476 
stats for CLUSTER_LOG_2010_06_04.MYD: file size=1478552 , total pages=360 , cached pages=97 , cached size=397312, cached perc=26.944444 
stats for CLUSTER_LOG_2010_06_04.MYI: file size=205824 , total pages=50 , cached pages=29 , cached size=118784, cached perc=58.000000 
stats for COMMENT_CONTENT_2010_06_03.MYI: file size=100051968 , total pages=24426 , cached pages=10253 , cached size=41996288, cached perc=41.975764 
stats for COMMENT_CONTENT_2010_06_04.MYD: file size=716369644 , total pages=174894 , cached pages=79821 , cached size=326946816, cached perc=45.639645 
stats for COMMENT_CONTENT_2010_06_04.MYI: file size=56832000 , total pages=13875 , cached pages=5365 , cached size=21975040, cached perc=38.666667 
stats for FEED_CONTENT_2010_06_03.MYI: file size=1001518080 , total pages=244511 , cached pages=98975 , cached size=405401600, cached perc=40.478751 
stats for FEED_CONTENT_2010_06_04.MYD: file size=9206385684 , total pages=2247652 , cached pages=1018661 , cached size=4172435456, cached perc=45.321117 
stats for FEED_CONTENT_2010_06_04.MYI: file size=638005248 , total pages=155763 , cached pages=52912 , cached size=216727552, cached perc=33.969556 
stats for FEED_CONTENT_2010_06_04.frm: file size=9840 , total pages=2 , cached pages=3 , cached size=12288, cached perc=150.000000 
stats for PERMALINK_CONTENT_2010_06_03.MYI: file size=1035290624 , total pages=252756 , cached pages=108563 , cached size=444674048, cached perc=42.951700 
stats for PERMALINK_CONTENT_2010_06_04.MYD: file size=55619712720 , total pages=13579031 , cached pages=6590322 , cached size=26993958912, cached perc=48.533080 
stats for PERMALINK_CONTENT_2010_06_04.MYI: file size=659397632 , total pages=160985 , cached pages=54304 , cached size=222429184, cached perc=33.732335 
stats for PERMALINK_CONTENT_2010_06_04.frm: file size=10156 , total pages=2 , cached pages=3 , cached size=12288, cached perc=150.000000 
---
total cached size: 32847278080

With the above output you can see that there are several *.MYD, *.MYI, and *.frm files that are currently being cached.

Swap

If you want to clear out your swap you can use the following commands.

$ free
             total       used       free     shared    buffers     cached
Mem:       7987492    7298164     689328          0      30416     457936
-/+ buffers/cache:    6809812    1177680
Swap:      5963772     609452    5354320

Then use this command to disable swap:

$ swapoff -a

You can confirm that it's now empty:

$ free
             total       used       free     shared    buffers     cached
Mem:       7987492    7777912     209580          0      39332     489864
-/+ buffers/cache:    7248716     738776
Swap:            0          0          0

And to re-enable it:

$ swapon -a

And now reconfirm with free:

$ free
             total       used       free     shared    buffers     cached
Mem:       7987492    7785572     201920          0      41556     491508
-/+ buffers/cache:    7252508     734984
Swap:      5963772          0    5963772
slm
  • 369,824
  • 15
    never heard of linux-ftools after so many years, though I knew how to drop cache. you are really a guru. Thanks for sharing! – johnshen64 Aug 23 '13 at 12:53
  • 6
    The things you say about sync are wrong: according to the linux doc, writting to drop_cache will only clear clean content (already synced). Besides, even if it drops unsynced data, saying that typing the sync command just before clearing cache would save your data is wrong: there is a non zero time between the sync command drop_cache write, so any data could be added during this time lapse. There is nothing atomic here. – Congelli501 Nov 13 '14 at 22:06
  • @congelli501 - please cite where this is stated. – slm Nov 13 '14 at 22:51
  • 3
    I got that information here: https://www.kernel.org/doc/Documentation/sysctl/vm.txt - "This is a non-destructive operation and will not free any dirty objects.". It also says that sync could be used to increase the droped cache size (for writeback caches). – Congelli501 Nov 14 '14 at 05:06
  • 1
    I agree with @Congelli501. sync && echo 3 > /proc/sys/vm/drop_caches is not atomic, so suggesting that they must be run "together" or you'll lose data would be crazy. +1 for the link to the docs. @slm can you please edit your answer to reflect that? – mgalgs Dec 10 '14 at 18:43
  • 1
    @mgalgs - thanks for reminding me, updated. – slm Dec 10 '14 at 18:56
  • 39
    Prefer "sudo sysctl vm.drop_caches=1" over "sudo sh -c 'echo 1 >/proc/sys/vm/drop_caches'" – Raúl Salinas-Monteagudo Apr 13 '15 at 14:10
  • 1
    Is linux-ftools maintained anymore? Their repo on google code is frozen because google shut everything down. – Joseph Garvin Oct 30 '15 at 13:43
  • 2
    @JosephGarvin - I believe they're part of the Linux kernel now. https://github.com/torvalds/linux/tree/master/mm. All the tools, fallocate, fadvise, and mincore are listed there. – slm Oct 30 '15 at 15:19
  • @slm - I'm on Ubuntu 14.04 but I only seem to have fallocate in my $PATH. Do you know if the others are available without me running a newer kernel? Or can I download the kernel sources and only compile those utils? – Joseph Garvin Oct 30 '15 at 19:15
  • Just a note here, you might wanna think twice about doing this if you're running zfs. I didn't think about it beforehand, but now that I'm doing it, arc_reclaim is chewing up my cpu, and well, I imagine zfs doesn't much like having it's arc emptied... hope my machine comes back. – stu Dec 17 '15 at 19:36
  • 11
    @slm i like using tee to write as root: echo 3 | sudo tee /proc/sys/vm/drop-caches – pqnet Dec 29 '15 at 14:21
  • Might I recommend adding echo 1 > /proc/sys/vm/compact_memory after dropping the caches? –  Jul 26 '16 at 21:11
  • @esperanto: Under UNIX-likes, everyting is a file. Which as a huge range of advantages, since everything can interact with files, and you don’t need to learn a new basic interface for everything. As they say “Those who do not understand Unix, are doomed to re-invent it. Badly.” :) (Which apparently even Linus doesn’t get anymore, considering the network driver implementation.) –  Jul 26 '16 at 21:14
  • @Evi1M4chine: I find the sysctl command requires less tokens, and in addition to that, it could for example check arguments and give meaningful error messages. I like simplicity but overengineering is as bad as underengineering. – Raúl Salinas-Monteagudo Jul 27 '16 at 07:41
  • 2
    @esperanto: Of course a limited interface requires less tokens. Because it’s limited. But you need one of them for every walled garden you create. And even worse, over time, you lose the ability to solve such problems in an universal manner, without knowing which tool to pick first. Which is why on Apple systems, you need an app for everything, and if you don’t have one, you are stuck. It is the stuck part… the self-crippling out of childish fear from being overwhelmed or just staight-up laziness… that’s the problem. –  Jul 28 '16 at 12:04
  • @esperanto: There is no reason a file interface can’t give any amount of error messages you like. sysctl is nothing more than using those files and picking the errors out of the kernel log. So prepending the echos with tail -f /proc/kmsg & cd /proc/sys/ or something like that should give the same results. –  Jul 28 '16 at 12:05
  • and if you don't have access to root, like in so many shared servers? How to clear the cache of your own session? – João Pimentel Ferreira May 11 '17 at 20:06
  • 1
    @JoãoPimentelFerreira Why? The kernel will not let other users access it, and if there were an exploit to do so, it wouldn't be difficult to use that exploit to gain root privileges and read anything it likes. – doug65536 Oct 06 '19 at 05:37
  • @sim How to empty coaches on regular basis from linux systems when the processes are running. without hampering process ? like if you are running machine learning tasks . – Swap Dec 25 '19 at 04:13
  • Just a BIG note about swapon and swapoff. Today I tested them on a Kubuntu LTS. It worked somehow by crashing the system after freezing the graphical environment: it damaged something -- now I know -- at the user's cache level. After a few reboots I was able to log in first in text mode, then starting Unity instead of plasma, reconfiguring all qt, kde plasma and x related packages with dpkg ... and wasting a couple of hours and all the window positions. So to be used with a fair amount of caution. – Hastur Nov 17 '21 at 14:24