74

I have a really strange situation here. My PC works fine, at least in most cases, but there's one thing that I can't deal with. When I try to copy a file from my pendrive, everything is ok -- I got 16-19M/s , it works pretty well. But when I try to copy something to the same pendrive, my PC freezes. The mouse pointer stops moving for a sec or two, then it moves a little bit and it stops again. When something is playing, for example, in Amarok, the sound acts like a machine gun. The speed jumps from 500K/s to 15M/s, average 8M/s. This occurs only when I'm copying something to a pendrive. When the process of copying is done, everything backs to normal.

I tried everything -- other pendrive, a different USB port on front panel or those ports from back, I even changed the USB pins on motherboard (front panel), but no matter where I put my USB stick, it's always the same. I tried different filesystem -- fat32, ext4. I have no problem with the device on Windows, on my laptop. It has to be my PC or something in my system. I have no idea what to look for. I'm using Debian testing with standalone Openbox. My PC is kind of old -- Pentium D 3GHz, 1GiB of RAM, 1,5TB WD Green disk. If you have something that would help me to solve this issue, I'd be glad to hear that.

I don't know what else info I should provide, but if you need something, just ask, I'll update this post as soon as possible.

I tried to reproduce this problem on ubuntu 13.04 live cd. I mounted my encrypted partition + encrypted swap and connected my pendrive to a usb port. Next I tried to start some apps, and now I have ~820MiB in RAM and about 400MiB in SWAP. There's no problem with copying, no freezing at all, everything is as it should be. So, it looks like it's a fault of the system, but where exactly? What would cause such a weird behavior?

slm
  • 369,824
Mikhail Morfikov
  • 10,549
  • When I encountered something similar to this it was because I had hard drive issues on my laptop. The disk had some bad areas and every time I was trying to read anything from those areas it would freeze until it was done. Just an idea to look into. Maybe you have bad sectors where you're trying to read from. – slybloty Jan 03 '14 at 16:22
  • My hdd is ok, at least smart say so (after full scan). – Mikhail Morfikov Jan 03 '14 at 16:26
  • Try reducing the IO priority of the copying process, e.g. ionice -c3 cp something.tgz /media/pendrive. This will put the newly spawned cp process in the third (=lowest) priority class "idle". – n.st Jan 03 '14 at 18:00
  • I tried this, but it has no effect. – Mikhail Morfikov Jan 03 '14 at 18:22
  • @MikhailMorfikov FYI, this issue was addressed in linux 4.9. Still haven't tested the fix myself. YMMV. – Seamus Connor Jan 24 '17 at 20:27
  • Could you give some link? – Mikhail Morfikov Jan 25 '17 at 00:27

3 Answers3

109

Are you using a 64-bit version of Linux with a lot of memory? In that case the problem could be that Linux can lock for minutes on big writes on slow devices like for example SD cards or USB sticks. It's a known bug that should be fixed in newer kernels.

See http://lwn.net/Articles/572911/

Workaround: as root issue:

echo $((16*1024*1024)) > /proc/sys/vm/dirty_background_bytes
echo $((48*1024*1024)) > /proc/sys/vm/dirty_bytes

I have added it to my /etc/rc.local file in my 64bit machines.

TANSTAAFL; this change can (and probably will) reduce your throughput to these devices --- it's a compromise between latency and speed. To get back to the previous behavior you can 

echo 0 > /proc/sys/vm/dirty_background_bytes
echo 0 > /proc/sys/vm/dirty_bytes

...which are the default values, meaning that the writeback behavior will be controlled by the parameters dirty_ratio and dirty_background_ratio.

Note for the not-so-expert-with-linux people: the files in /proc are pseudofiles --- just communication channels between the kernel and user space. Never use an editor to change or look at them; get instead a shell prompt --- for example, with sudo -i (Ubuntu flavors) or su root and use echo and cat).

Update 2016/04/18 it seems that, after all, the problem is still here. You can look at it at LWN.net, in this article about writeback queues.

Rmano
  • 3,425
  • 4
    I have 64bit but only 1GiB of RAM, and I have to tell you this solution works! I've just tested it, and after setting the two parameters, there's no freezing anymore. :) – Mikhail Morfikov Jan 03 '14 at 19:10
  • +1 - It worked fine. !!! Thank you so much. It was a real head ache – Abid Rahman K Feb 16 '14 at 17:26
  • +1: it worked perfectly for me too. I always wondered why my system became unusable when copying stuff to an external HDD, now has a workaround. – Renan Mar 08 '14 at 01:59
  • Do I need to restart the PC after running these commands? – a06e Aug 07 '14 at 15:07
  • How can I revert this change? Does anyone report a decrease in copying speed after applying this fix? – a06e Aug 07 '14 at 15:13
  • @becko Writing "0" in the two pseudofiles above should reset the thing as it was before. Notice that since kernel 3.13 the above workaround should not be needed. – Rmano Aug 07 '14 at 15:38
  • @Rmano Is kernel 3.13 in Ubuntu 14.04? – a06e Aug 07 '14 at 15:53
  • 1
    In my 14.04 installation, uname -a returns 3.13.0-32-generic, so yes. But I have not checked if the patch for the issue was finally integrated in the kernel or not. I have a 16GB machine and it seems to work ok without the workaround, although I have to say that I didn't try with especially slow devices. – Rmano Aug 07 '14 at 15:56
  • On Ubuntu 14.04, I get bash: /proc/sys/vm/dirty_background_bytes: Permission denied -- however I saved the file using VIM. Any idea about this error? – Ionică Bizău Oct 05 '14 at 17:26
  • 1
    @IonicăBizău --- that is a pseudo-file, do not edit it with vim ever. Get a root shell (with sudo -i) and use the aforementioned commands. – Rmano Oct 05 '14 at 17:52
  • 1
    @Rmano It worked! However, I did edit it with VIM. Thanks! – Ionică Bizău Oct 05 '14 at 18:49
  • I'm on Ubuntu 16.04. Is this solution better than setting vm.dirty_background_ratio = 5 and vm.dirty_ratio = 10 as suggested here? – Joschua May 15 '16 at 09:38
  • @Joschua, it will probably depend on the specific case --- workload, available ram, device speed. Try both --- they're not dangerous. – Rmano May 15 '16 at 15:35
  • I know some who'd be much interested in Mikhael question and @Ramno hack! E.g. Copying files and using the computer should not be painful rocket science and why Linux feels slow, and how to fix that, by Rudd-O, 2007-2013. Great thread! Though I can't remember having this kind of freeze in 15 yrs --11 of which with =<2 gig RAM. – tuk0z Sep 04 '16 at 23:35
  • 3
    I'm using ubuntu 16.04 on a brand new laptop (with 16 GB of RAM). I was really angry at this issue. Your solution worked like a charm! Maybe you could add that this could still be needed with kernel 4.8.0-45 . – LGenzelis Apr 14 '17 at 18:58
  • 1
    To make it persistent see this answer – PeterM Sep 14 '17 at 13:06
  • A bit more explanation here: https://wiki.archlinux.org/index.php/sysctl#Virtual_memory – esskov Nov 21 '17 at 01:28
2

hardware vs. software

I've run into strange problem similar to this with USB thumbdrives, and in my research it's almost always either a driver issue or the specific hardware within the PC/Motherboard.

I know this because I've got several systems that are identical hardware, and on one, I can do this operation without issue, while on another the problem shows up.

What to do?

You options are really limited here. About the only things you can do are make sure you have the latest BIOS/firmware installed on your system, and make sure you have the latest versions of your disto's packages.

Beyond that all I can suggest is making sure that you avoid this situation by not attempting to copy files while another copy is in progress.

If you have the type of personality where things like this irk you, you could try another live distro of Linux and repeat the steps that lead to your problem. This would just eliminate whether it's a distro specific issue or a hardware issue as I've described above. It would be a small consolation, but I always like to know things rather than bury my head in the sand, and not.

Anything else?

If you're truly obsessive you could try running the application that you're doing the copy with through strace in the hopes of catching the system in whatever system call is freezing. You should be able to do this from the command line as well.

Example

$ strace -o cp1.log cp -r /path/to/dir1 /path/to/usb/.

Then while that's running start another one.

$ strace -o cp2.log cp -r /path/to/dir2 /path/to/usb/.

The system will hopefully freeze during this operation and maybe you'll get lucky and find some smoke in either of those log files.

slm
  • 369,824
  • I always use the only one instance of file copying. I have BIOS updated (2008), and there's no newer version since then. I think it's not the BIOS. My debian distro is also updated to the testing branch. I tried using strace and it starred freezing almost instantly, so I waited a few sec and killed the process. I got 1Mb log, but I can't read it, I don't know what to look for. You can check it here http://pastebin.com/u29RvqgC -- it's not the full log (limited to 500Kb), but there were only similar lines to those at the end. I will try to reproduce this issue with ubuntu live cd. – Mikhail Morfikov Jan 03 '14 at 17:08
  • I updated the question as to live cd testing. – Mikhail Morfikov Jan 03 '14 at 17:51
  • @MikhailMorfikov - I think you're pretty much at the end of what you can expect to do. Your hardware is pretty old (2008) and there's really not much else you can do beyond what I've outlined above. – slm Jan 03 '14 at 18:03
  • But even older pcs are able to copy files without problems. – Mikhail Morfikov Jan 03 '14 at 18:20
  • 1
    @MikhailMorfikov - Age isn't the only factor, but the likely hood of getting any updates to firmware or updates to software for old hardware is low, is what I meant. – slm Jan 03 '14 at 18:22
2

The reason can be write amplification, as the system tries to write in chunks that are smaller, than erase block (doing read/mod/write) + block misalignment.

To check yours current setting do:

cat /sys/block/sd**X**/device/max_sectors

You can tune hall rules for those devices:

Change value of USB "max_sectors" for an entire family of devices

In this case I had replaced max_sectors for all devices, that used default of 240 (USB storage) to 32K sectors or 2K sectors.

On my system (Mageia 4, 3.14.24 core i7) I had to do this due terribly slow write speeds (2MB/sec) on Kingston DT101 G2 16GB: 

vi /usr/lib/udev/rules.d/81-udisks_maxsect.rules

and add:

SUBSYSTEMS=="scsi", ATTR{max_sectors}=="240", ATTR{max_sectors}="32678"

And the dd write speed went up 3x times. mc cp probably 10-20x up (after I had started first partition @8192'th sector and reformatted with 64k aligned clusters): 

fdisk -u /dev/sdh # make DOS compat off if on
mkfs.vfat /dev/sdh1 -n KINGSTON16G -s 128 **-R 4592*** and use *fsck.vfat -v /dev/sdh1

to check alignment (check [data start sector] should be multiple of 128 (cluster size)). Adjust number of the reserved sectors (-R) if needed.

Default max_sectors (240) seem to cause high write amplification on some of the cheap new drives. But be very careful with such high setting, the similar effect is achieved at 2048 sectors (probably 1M erase blocks:

SUBSYSTEMS=="scsi", ATTR{max_sectors}=="240", ATTR{max_sectors}="2048"

Test all yours old USB devices, that they still work well. Use vendor/model attributes in the rules files to be more specific.

Community
  • 1
Mark
  • 29