Is cp faster than rsync during the first run?

Question

I am currently using rsync to copy a 73GB file from a Samsung portable SSD T7 to an HPC cluster.

rsync -avh path/to/dataset me@server.somewhere:/path/to/dest

The following applies:

1. My local machine (where my T7 is connected) is a VirtualBox VM running Ubuntu 20.
2. The T7 transfer speeds should be up to approx. 1000MB/s.
3. Network gives me an approximate upload speed of 7.9Mbps.
4. Rsync transfer speed is probably bottlenecking this to 1-5MB/s according to this answer.

The problem is that the move is still not done after 9 hours. According to 1, using cp instead is better with an empty directory (for the first time). I do not understand this or whether it is actually true. Can someone explain this?

Answer 1

You say that,

1. Your network speed is approximately 8 Mb/s.
2. Your rsync runs at about 1-5 MB/s.

Given that 1 MB/s is approximately 10 Mb/s I'd say that rsync is doing you a big favour.

Me, I'd probably have added compression with -z, and since you're using rsync with sensible flags over a network connection it's probably with interrupting it and restarting with compression. It'll just pick up where it left off.

Quick calculation: 73 GB is 73,000 MB^*, which is 730,000 Mb^‡. Approximately. You've got a network speed of 8 Mb/s, so that means the copy should take around 730,000/8 = 91,250 seconds. 25 hours, assuming theoretical maximum use of network bandwidth.

^{* Why 1000:1 instead of 1024:1? Partly because GB:MB strictly is 1000:1, but largely because this is an approximation calculation
‡ Why 10:1 instead of 8:1 as suggested by 8 bits to 1 byte? Two reasons: (a) it's an O(n) approximation, and (b) there's packet/protocol overhead to consider.}

---

Now to try and answer the question strictly as asked, which is, "Is cp faster than rsync during the first run?". If you're using cp you need to have something managing the transport between the local and remote servers; that could be something like sshfs or NFS, or alternatively you might mean scp.

1. cp over an NFS mounted filesystem. With a well-tuned network this is probably fairly efficient.
2. cp over sshfs. This will include not only encryption overheads of ssh but also the translation between kernel and userspace for the FUSE implementation of the filesystem. Inefficient.
3. scp (implicitly over ssh). This will include encryption overhead from ssh. Acceptable, and might futher benefit from link compression (-C).
4. rsync (implicitly over ssh since we've not mentioned rsyncd). This will include encryption overhead from ssh. Acceptable, and might futher benefit from protocol compression (-z).

I've not taken quantitative performance metrics; these are qualitative assessments. However, note the comments on a similar answer Why is scp so slow and how to make it faster?, although to be fair that's discussing comparative transfer rates of multiple files rather than one large one.

However, where rsync wins across a network connection is that it's restartable. With the right flags (i.e. --partial) even mid-way through a transfer. For a single 73GB file taking around 25 hours to transfer that's a massive potential advantage.