5
~$ free
             total       used       free     shared    buffers     cached
Mem:       3891076    1044972    2846104          0      82744     463876
-/+ buffers/cache:     498352    3392724
Swap:      4035580          0    4035580
  1. How to investigate what kind of data is buffered and cached.(not answered yet)
  2. How to clear buffers and cached? (answered)

EDIT

This command provided by Chris did the trick: 

sudo bash -c 'echo 3 > /proc/sys/vm/drop_caches && sleep 2 && free'

2 Answers2

12

The memory represented by "buffers/cache" in free is your disk and filesystem cache, respectively, which Linux caches to speed up reading data from your disk, as hitting the disk is generally a fairly slow way to access data repeatedly. As such, they are cached in memory, and transparently served from there if available.

You can see which blocks are currently in your cache by using fincore. Here is an example from the project page:

# fincore --pages=false --summarize --only-cached * 
stats for CLUSTER_LOG_2010_05_21.MYI: file size=93840384 , total pages=22910 , cached pages=1 , cached size=4096, cached perc=0.004365 
stats for CLUSTER_LOG_2010_05_22.MYI: file size=417792 , total pages=102 , cached pages=1 , cached size=4096, cached perc=0.980392 
stats for CLUSTER_LOG_2010_05_23.MYI: file size=826368 , total pages=201 , cached pages=1 , cached size=4096, cached perc=0.497512 
stats for CLUSTER_LOG_2010_05_24.MYI: file size=192512 , total pages=47 , cached pages=1 , cached size=4096, cached perc=2.127660 
stats for CLUSTER_LOG_2010_06_03.MYI: file size=345088 , total pages=84 , cached pages=43 , cached size=176128, cached perc=51.190476

As for how to clear them, from man 5 proc:

/proc/sys/vm/drop_caches (since Linux 2.6.16)

Writing to this file causes the kernel to drop clean caches, dentries, and inodes from memory, causing that memory to become free. This can be useful for memory management testing and performing reproducible filesystem benchmarks. Because writing to this file causes the benefits of caching to be lost, it can degrade overall system performance.

  • To free pagecache, use: echo 1 > /proc/sys/vm/drop_caches
  • To free dentries and inodes, use: echo 2 > /proc/sys/vm/drop_caches
  • To free pagecache, dentries and inodes, use: echo 3 > /proc/sys/vm/drop_caches

Because writing to this file is a nondestructive operation and dirty objects are not freeable, the user should run sync(8) first.

You generally don't want to flush the cache, as its entire purpose is to improve performance, but for debugging purposes you can do so by using drop_caches like so (note: you must be root to use drop_caches, but sync can be done as any user):

# sync && echo 3 > /proc/sys/vm/drop_caches
Community
  • 1
Chris Down
  • 125,559
  • 25
  • 270
  • 266
6

fincore is a useful tool to see how much of specific files are present in the page cache. In order to find which files are getting added to the page cache, I've found the kernel tracepoint filemap:mm_filemap_add_to_page_cache to be useful.

ex:

# perf record -a -e filemap:mm_filemap_add_to_page_cache sleep 60
# perf script
[...]
        perf_5.7   889 [005]    17.077676: filemap:mm_filemap_add_to_page_cache: dev 0:3 ino 0 page=0x234161 pfn=2310497 ofs=12897742848
        perf_5.7   889 [005]    17.078127: filemap:mm_filemap_add_to_page_cache: dev 0:3 ino 0 page=0x228440 pfn=2262080 ofs=13017223168
        perf_5.7   889 [005]    17.078457: filemap:mm_filemap_add_to_page_cache: dev 254:1 ino c23cf page=0x22d236 pfn=2282038 ofs=0
        perf_5.7   889 [005]    17.078460: filemap:mm_filemap_add_to_page_cache: dev 254:1 ino c23cf page=0x228447 pfn=2262087 ofs=4096
[...]

The dev <major>:<minor> numbers can be mapped to device nodes using lsblk.

# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
fd0      2:0    1    4K  0 disk
sr0     11:0    1 1024M  0 rom
vda    254:0    0   20G  0 disk
└─vda1 254:1    0   20G  0 part /

(I'm not sure what the "0:x" entries are, maybe anonymous mappings also trigger that tracepoint?)

The inode numbers can be mapped to files using find [...] -inum <decimal>.

# find / -mount -inum $((0xc23cf))
/usr/bin/sleep

In a pinch, the output can be made more pallatable like this:

# perf script | grep -v "dev 0:" | \
    awk '
        {
            match($0, "ino ([^ ]+)", a)
            print a[1]
        }
    ' | sort | uniq -c | sort -n | \
    while read count inode; do
        echo "$count $inode $(find / -mount -inum $((0x$inode)))"
    done

[...]
20 c1950 /usr/lib/apt/methods/rred
27 c194a /usr/lib/apt/methods/gpgv
36 4203f
36 60070
36 60080 /var/lib/apt/lists/deb.debian.org_debian_dists_unstable_InRelease
36 c1f24 /usr/bin/cp
40 c04a6 /usr/bin/gpg-connect-agent
47 c194c /usr/lib/apt/methods/http
50 60340 /var/lib/apt/lists/deb.debian.org_debian_dists_unstable_contrib_source_Sources
[...]

The first column corresponds to the number of pages swapped in. Presumably, the inodes without file names have been deleted.

gobenji
  • 231