0

I have been facing this issue ever since I started using Linux distributions. While copying/moving either graphically or with cp, one/many big content (anything like a big text file, tar.gz archive, ISO image file, and movies), some part of the content is written to disk and some part cached in memory (RAM). During the copying time, the amount of shared and cached memory dramatically increased (checked with free -m).

After some time the file manager (like Dolphin or PCManFM) or cp shows that copying is finished, but data is actually not written to disk until I do a sync

I think this is not a hardware issue.

  • I checked with many internal and external hard drives, and USB flash drives of various brands, but all with the same result.

  • Not a hard drive APM issue. I always disable hard drive power management.

  • The problem is same with dd and cat, like dd if=live.iso of=/dev/sdb.

  • Not a distribution-specific issue. I checked with Debian, Fedora, Ubuntu, Slax, etc.

  • I have not crosschecked with other Unix-like OSes. If anyone have/had the same issue with FreeBSD, OpenBSD, etc. please let me know.

What is the problem and how can I solve it?

Peter Mortensen
  • 1,025
Arnab
  • 1,581
  • 2
    It is not a problem, but a feature. http://linuxatemyram.com/ – Basile Starynkevitch Jun 20 '15 at 05:32
  • okay , got it , but I want that data to be written on the disk, like making a live USB, taking a movie from a friend etc and I want it as fast as possible. Is there any workaround to avoid/disable it ? – Arnab Jun 20 '15 at 05:38
  • You might use some mount options. You should umount a media (e.g. your USB key) before unplugging it (that would flush the data to the disk). You could continue using sync – Basile Starynkevitch Jun 20 '15 at 05:47
  • Everything explained very well at http://www.linuxatemyram.com/play.html , I will stick with sync . Thank you for your comment , will you consider to write it as an answer ? – Arnab Jun 20 '15 at 09:09

2 Answers2

2

It is by design that the kernel tells userspace utilities that a file operation completed successfully while the caches are buffered.

Try editing /etc/hdparm.conf to disable write caching with the option write_cache = off. However this will reduce performance, and it is usually preferred to use sync if you want to ensure write buffers are flushed to disk.

Rachel Frei
  • 494
  • /etc/hdparm.conf says write_cache = off for IDE drives , though disabled write-cache temporarily with hdparm -W 0 /dev/sda but this is even worst, laggy performance. Anyway I will stick with sync. but what is wrong wiyh USB flash drives , they dont support write-caching ? – Arnab Jun 20 '15 at 05:57
  • 4
    write_cache = off in /etc/hdparm.conf only disables the write cache in the drive, it doesn't affect the kernel's caches. – Stephen Kitt Jun 20 '15 at 08:44
1

As I commented (and for obvious performance reasons) the kernel is using a page cache. So this is a feature, not a problem. See http://linuxatemyram.com/ for more.

You could (but I don't recommend doing that) using some mount options (to disable, or lower the use of, the page cache), and you need to umount any device (e.g. an USB key) before unpluggging or removing it. Then the kernel would flush all the data before unmounting.

You can also do a sync.

Basile Starynkevitch
  • 10,561