32

I'm working on an embedded system with the busybox version of dd. I'm trying to test an erase to the drive from some outside utility, however dd does not read from the disc again after the erase, but shows me the cached data.

I've narrowed it down to dd as when I do an initial dd, see the data, restart my system to flush the cache, did the erase, and then ran dd again it came up with all zeros.

However, if I do dd on factory settings, erase the drive, and do dd again without restarting it won't show me all zeros until a restart.

I've read in the GNU manpage that dd supports the iflag opt, with a nocache flag, but busybox does not support that option so that's out of the question.

My question is how can I force dd to read from the disk again rather than from cache?

Gilles 'SO- stop being evil'
  • 829,060
ardent
  • 445
  • 2
    Potential nitpicking, but... dd does not cache it. The kernel/disk system does. This is why Frostschutz's command (which does not get send to dd) works. – Hennes Jul 12 '13 at 22:26
  • I need this to keep the drive awake instead of waiting to spin up. – neverMind9 Jun 03 '18 at 20:54
  • For people looking for answer for question in the title rather than an answer for the problem statement in the description, see: https://unix.stackexchange.com/questions/379947/how-to-prevent-dds-progress-from-being-meaningless-on-linux/704649 – kravemir Jun 01 '22 at 18:13

4 Answers4

27

Based on @sendmoreinfo's answer:

dd if=/dev/device iflag=direct bs=1M

It does not affect read-cache.

Ole Tange
  • 35,514
13

You could try

sync
echo 3 > /proc/sys/vm/drop_caches

which drops all sorts of caches.

For details see /usr/src/linux/Documentation/sysctl/vm.txt on drop_caches.

Note: the question was about busybox dd which did not support iflag=direct at the time. It was added in busybox v1.33.0 (2020-12-29), see busybox dd: support for O_DIRECT i/o. See the other answers for usage examples.

frostschutz
  • 48,978
  • 1
    Just a note: that only works with Linux. Also, the /proc filesystem is disabled in some embedded configurations of Linux. –  Jul 12 '13 at 20:28
  • @EvanTeitelman Right, It seems to work here though, so I'm going to upvote and accept this answer. – ardent Jul 12 '13 at 20:40
  • 2
    I wondered if this has a once-off effect, or is ongoing. According to this, the effect is a once-off. – Craig McQueen Feb 07 '17 at 04:10
  • This will drop the caches, but does nothing at all to prevent dd from trashing them in the process if transferring large amount of data. – Jan Hudec Feb 19 '19 at 09:10
12

A small flourish to the the iflag=direct answer; provide a progress bar too:

dd iflag=direct if=~/source.iso | pv | dd oflag=direct of=/dev/sdb bs=8M

Without the iflag/oflag the pv reports it's finished and then looks like it's hung; but the dd to the device is still working on the cache.

awltux
  • 241
  • 2
  • 2
  • 2
    dd >= 8.24 also supports the status=progress flag which removes the need for pv – 1N4001 Oct 28 '20 at 14:52
  • @IN4001 This is only partially true. dd did implement progress, but it's very basic, and if you want proper progress, you still need a secondary tool like pv. – xZero Jun 12 '21 at 11:10
9

Direct I/O (open mode O_DIRECT) should work, but your kernel and/or dd may not support it.

sendmoreinfo
  • 2,573
  • 4
    This is the right answer: dd if=/dev/device iflag=direct bs=1M; it does not affect read-cache. – Ole Tange Feb 04 '17 at 20:42
  • 1
    @OleTange: Why don't you post this as your own answer then? This is the #1 hit on google for "dd no cache", and the actual correct answer is in a comment to some non-accepted answer... – mic_e Oct 11 '17 at 10:41