I was wondering if there is any caching done with these utilities. I assume not, but not positive. Are there any typical similar utilities that do use caching to speed up results on subsequent runs?
Asked
Active
Viewed 3,658 times
5
-
I don't think that dd/df are doing the caching themselves but it rather some part of the kernel does that automatically – Bonsi Scott May 04 '14 at 20:49
-
Maybe you ask the question in view of an other problem. When a program writes large logfiles and keep the filedescriptors open, deleting the files will not help making space free. The ugly program still has 'locked'the filespace. – Walter A May 04 '14 at 21:24
-
@WalterA I am only asking out of curiosity, I dont have an issue with open file descriptors or unexpected output, but thank you. – Gregg Leventhal May 04 '14 at 22:12
-
Take a look at this question and my answer – Yuval Sep 17 '19 at 00:10
-
@yuval Thanks! I have learned so much since I asked this ill-informed question (it's a little embarrassing to be reminded of this question, haha) – Gregg Leventhal Sep 19 '19 at 00:32
3 Answers
2
There is no need for caching in df as df does a single statfs() call (per filesystem). And obviously this call does read files on disk and sum up their sizes... The filesystems (e.g. the kernel) keep track of the free space.
du uses (without being aware of that) the page cache all applications use. In order to cache explicitly between runs there would have to be a "du daemon" anyway.
Hauke Laging
- 90,279
-
ducould have a cache without a daemon. All it has to do is store the directory sizes in a file somewhere, and then read back that file on next run. (not saying it does, just saying a daemon is not needed) – phemmer May 04 '14 at 21:56 -
@Patrick But without something like inotify this would make caching a probability game. – Hauke Laging May 04 '14 at 21:59
-
Ah, thats why you meant daemon, so it could monitor real time. Yes, agree :-) – phemmer May 04 '14 at 22:00
0
There is caching at some level (command or perhaps OS or FS-driver level - probably one of the latter two if you ask me), although forgive me for not knowing more details:
d@s7/mp3Ϡϡ time du -sh /mp3/
27G /mp3/
du -sh /mp3/ 0.01s user 0.03s system 32% cpu 0.112 total
d@s7/mp3Ϡϡ time du -sh /mp3/
27G /mp3/
du -sh /mp3/ 0.00s user 0.01s system 82% cpu 0.015 total
d@s7/mp3Ϡϡ time du -sh /mp3/
27G /mp3/
du -sh /mp3/ 0.00s user 0.01s system 86% cpu 0.014 total
d@s7/mp3Ϡϡ time du -sh /mp3/
27G /mp3/
du -sh /mp3/ 0.01s user 0.01s system 78% cpu 0.020 total
Results obtained on Ubuntu 15.04 with ext4 filesystem, kernel version 3.19.0-15-generic.
djvs
- 101
-
those margins are really tiny to meaningfully demonstrate caching performance. I would use a much larger directory – axolotl Apr 18 '22 at 19:56
-2
confirm, it caches
see this:
$du -sh testUpload.txt
104M testUpload.txt
$ dd if=/dev/zero of=testUpload.txt bs=1M count=50
50+0 records in
50+0 records out
52428800 bytes (52 MB, 50 MiB) copied, 0.0248501 s, 2.1 GB/s
$ du -sh testUpload.txt
104M testUpload.txt
<b>
$ ls -al testUpload.txt
-rw-rw-rw- 1 alfred alfred 52428800 Jul 4 11:50 testUpload.txt
-
1Cannot reproduce. What system is this? What filesystem is
/tmp? Whichduis it? – muru Jul 04 '19 at 04:15 -
1
-
showing du is caching it, and it does not reflect the "new" size immediately. – system programmer Jul 04 '19 at 09:50
-
du is showing you the amount used. You wrote zeros to it, so it's a sparse file and didn't add any addition usage. To see the apparent size, use du --apparent-size testUpload.txt – Jacob Brown Mar 02 '22 at 22:00