1

I'm copying files to a USB 3.0 stick. The USB stick is plugged to a blue USB port.

I'm copying 22 files of 1.5GB.

When I do the copy (simple drag and drop), the copy dialog reports a speed of ~60MB/s, then, after copying the first 1.5 GB (surely the first file), progression stops for several minutes and after that, transfer speed is reported to be less than 3MB/s. The original transfer time estimated to a few minutes is now estimated to few hours...

enter image description here enter image description here

lsusb -t

reports:

/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M
    |__ Port 1: Dev 6, If 0, Class=Mass Storage, Driver=usb-storage, 480M
    |__ Port 9: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 2: Dev 4, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 3: Dev 5, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 10: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M

The device is on Bus 01, reported 480M, is that a speed? If so, I see Bus 02 has a higher rate of 5000M, but I tried all ports accessible and I could not have the USB stick appear under this Bus.

lsb_release -a

reports

Distributor ID: LinuxMint
Description:    Linux Mint 18.3 Sylvia
Release:    18.3
Codename:   sylvia

Note:

  • Same files are copied in less than an hour using the same USB stick on a Windows computer, takes more than 6 hours using my Linux machine. So I really suspect the Linux system does not handle USB 3 correctly (as it shows the key as 480M, while it IS a USB 3 memory stick: has a blue USB connector and is sold as USB 3)
  • Tried a different USB 3 stick. It appears under Bus 02 5000M this time and copy is much faster (even if estimation remains hazardous). So apparently my computer can handle some USB 3 devices correctly but not the one I'm using in my original post. As apparently Windows is able to it, hopefully there is something that can be done for Linux to manage it a better way.

How can I make the copy faster?

jpo38
  • 91
  • 1
    "480M" is typical of USB2.0 ports. What you're observing is normal and expected for that USB standard (the apparent high speed at the beginning is a sort of RAM buffering). – ChanganAuto Apr 24 '23 at 23:05
  • OK thank you, does the lsusb output mean I'm supposed to have USB3 ports? How can I confirm this from the hardware and make those available in my system? – jpo38 Apr 25 '23 at 05:17
  • The only option I could find in my BIOS is "XHCI Hands-off" and it's enabled – jpo38 Apr 25 '23 at 05:52
  • 1
    3 MB/s can't be the limit of the bus, but it can be the writing speed of a cheap USB stick. It starts with 60 MB/s (which indicates USB3, because you won't get this as effective rate on USB2), until the stick's cache is filled, then it stalls to continue with the actual writing speed. What does the data sheet of the stick say? – Philippos Apr 25 '23 at 08:28
  • I agree with the previous comments. If you wish, you can look closer at the real write speed according to this link and this link and this link – sudodus Apr 25 '23 at 09:00
  • 1
    There's no "cache" for 99.999% of USB flash sticks out there (only rare "SSD" USB flash sticks have it), you're solely looking at dirty buffers being filled up. – Artem S. Tashkinov Apr 25 '23 at 09:16
  • 1
    @Philippos, you say it indicates "USB 3" but lsusb shows I'm on a USB 2 bus, doesn't it? I have no USB stick datasheet, it comes from Amazon... it's cheap. – jpo38 Apr 25 '23 at 10:23
  • @jpo38, You cannot increase the write speed of existing hardware, but you can watch it (as described in the links in my previous comment). What you can do to improve the speed is check carefully what to buy, to select USB drives with fast hardware (memory cells and other internal electronics). This is described in the following link: Installation From USBStick - Prerequisites. – sudodus Apr 25 '23 at 11:11
  • @sudodus seemy comment, same USB disk and same files are copied much faster under Windows, so I don't believe the USB drive is actually the issue. – jpo38 Apr 25 '23 at 12:35
  • @jpo38, you know your system better than I, so if Windows is copying the files faster than Linux I can only say 'too bad for Linux'. Please remember to include the time for flushing the buffers (by 'safe removal of USB drive' in Windows). - I can think of another cause of speed difference: I have noticed that Linux cooperates best with Linux file systems, it is slower (can be much slower) with NTFS compared to ext4). So if you want fast write speed with Linux, you should have fast hardware and a suitable file system, for example ext4. – sudodus Apr 25 '23 at 21:42
  • I ended up returing the memory stick and buying a new one, payed a few more euros, the new one is recognized as USB3 by my OS and works much faster.... – jpo38 Apr 30 '23 at 16:03

1 Answers1

3

progression stops for several minutes and after that, transfer speed is reported to be less than 3MB/s

First dirty buffers are being filled up and you're not seeing the real write speed, and once they are filled up, you're starting to get the real write speed.

More info here:

Why were "USB-stick stall" problems reported in 2013? Why wasn't this problem solved by the existing "No-I/O dirty throttling" code?

https://lwn.net/Articles/572911/

Here's how I've solved it for myself:

/etc/sysctl.d/dirty_buffers.conf
# Per Torvalds advice
vm.dirty_background_bytes = 33554432
vm.dirty_bytes = 134217728

For some reasons this is still relevant even with Linux 6.2.

Artem S. Tashkinov
  • 29,025
  • Thank you for your help. What do you mean by "I've solved it". Will this make the estimated speed and remaining time shown more accurate or will it actually make the copy be faster? – jpo38 Apr 25 '23 at 10:21
  • 1
    This "solution" will make your applications show the real estimated speed. Performance cannot be "fixed", it's a property of your USB flash drive. – Artem S. Tashkinov Apr 25 '23 at 10:52
  • Ok thank you for your help – jpo38 Apr 25 '23 at 12:30
  • Thing is the same files are copied in less than an hour using a Windows computer, takes more than 6 hours using my Linux machine. So I really suspect the Linux system does not handle USB 3 correctly (as it shows the key as 480M, while it IS a USB 3 memory stick) – jpo38 Apr 25 '23 at 12:33
  • See more notes in my OP, looks like Linux handles this specific USB stick as USB 2 while it's USB 3 and while it's able to handle others correctly. Is there any firmware in those USB sticks that could be incompatible with some OS? – jpo38 Apr 25 '23 at 15:16
  • This could be a kernel bug: please try version 6.2.12 and if it doesn't work, consider filing a bug report here bugzilla.kernel.org – Artem S. Tashkinov Apr 26 '23 at 08:47
  • 1
    I don't understand the version of Linux you are talking about. uname -r reports me 4.15.0-96-generic and software update proposes me ``4.15.0-142`, this is far from 6.2... – jpo38 Apr 26 '23 at 21:42
  • Google: ubuntu how to install mainline kernel – Artem S. Tashkinov Apr 27 '23 at 06:20
  • @jpo38, Your Linux kernel series is old. Which Linux distro and version are you running? You may want to try a newer version of the whole operating system, which might manage writing to your USB drive better. – sudodus Apr 27 '23 at 08:34
  • 1
    @sudodus. It's Linux Mint 18.3 – jpo38 Apr 28 '23 at 09:57
  • 2
    Version 18.3 is unsupported, you really could update: https://linuxmint.com/download_all.php – Artem S. Tashkinov Apr 28 '23 at 10:28
  • I'm on Ubuntu 22.04 and that dirty_buffers.conf code worked perfectly for me! My usb sticks are exfat formatted and were hovering around 3MB/s, but after adding that code and restarting, they're back up to 54MB/s which is around what was expected! – chimeraha Dec 19 '23 at 03:49