Is it possible to see cp speed and percent copied?

Question

I'm having problems when copying large files using nautilus (it gets stuck). I need to copy, using cp. I would like to know if there are any parameters that shows the % copied and also the transfer speed.

Answer 1

rsync version 3.0.9+ has a --progress flag, which shows progress per file:

rsync --progress SOURCE DEST

rsync of any version has a --info=progress2 flag, which shows the overall percentage:

rsync --info=progress2 SOURCE DEST

Answer 2

If you allow other tools than cp it's surely possible. For a single file you can use pv. It's a small tool providing nice statistics.

pv inputfile > outputfile

If you have multiple files or directories you can use tar:

tar c sourceDirectory | pv | tar x -C destinationDirectory

You can wrap it in a shell function. It's less to type and you get semantics close to the ones of cp. Here's a very simple (and not error-proof!) function:

cpstat () {
  tar c "$1" | pv | tar x -C "$2"
}

Note that some versions of tar don't support the abovementioned syntax (e.g. Solaris tar) and you have to use the following variant:

cpstat () {
  tar cf - "$1" | pv | (cd "$2";tar xf -)
}

You call it like this

cpstat sourceDirectory destinationDirectory

You can enhance it further, so that pv provides an estimation of the remaining time.

Another solution (as frostschutz mentioned in a comment) is to use rsync with the --progress option:

rsync --progress -a sourceDirectory destinationDirectory

Answer 3

rsync works the best for showing the progress during the copying progress.

ex:

rsync -avh --progress sourceDirectory destinationDirectory

Answer 4

Is it possible to see cp speed and percent copied?

Yes, it is possible to display the progress of the command cp on the terminal.

---

Method #1: (use progress)

Open a new terminal and type progress it will display the progress of cp command and others command such as:

cp, mv, dd, tar, cat, rsync, grep, fgrep, egrep, cut, sort, md5sum, sha1sum, sha224sum, sha256sum, sha384sum, sha512sum, adb, gzip, gunzip, bzip2, bunzip2, xz, unxz, lzma, unlzma, 7z, zcat, bzcat, lzcat, split, gpg

If you didn't install progress, you can install it from repository, for example in the CentOS with this command: yum -y install progress.

Then after you run cp command in the background, or after run cp normally, open another terminal, so just try this command:

# progress -m

See output:

[30904] cp /home/user/file.mp4
        2.3% (200.9 MiB / 8.4 GiB) 3.6 MiB/s remaining 0:38:58

---

Method #2: (use patch cp)

Or download the GNU coreutils and patch it.

cd /tmp
wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.21.tar.xz
tar -xvJf coreutils-8.21.tar.xz
cd coreutils-8.21
wget https://raw.githubusercontent.com/atdt/advcpmv/master/advcpmv-0.5-8.21.patch
patch -p1 -i advcpmv-0.5-8.21.patch

Run:

export FORCE_UNSAFE_CONFIGURE=1
./configure
make

Copy the two files cp and mv loacated under coreutils-8.21/src/ to /usr/local/bin:

cp src/cp /usr/local/bin/cp
cp src/mv /usr/local/bin/mv

Or copy it using your file manager as root

Add the following lines to your ~/.bashrc:

alias cp='cp -gR'
alias mv='mv -g'

Logout and Login

The progress bar can be displayed without the –g or –progress-bar as recommended by the Author

See output:

0 files copied so far...                                              194.4 MiB /   8.4 GiB
[||--------------------------------------------------------------------------------]  2.3 %
Copying at   3.3 MiB/s (about 0h 5m 47s remaining)
/home/user/file.mp4                                                   194.4 MiB / 446.1 MiB
[||||||||||||||||||||||||||||||||||||----------------------------------------------] 43.6 %

Source:linuxhelp

Answer 5

As explained in other answers, cp doesn't provide that functionality. However, progress (formerly called cv) is a small tool which is very helpful in providing this missing feature.

This tool can be described as a Tiny Dirty Linux Only* C command that looks for coreutils basic commands (cp, mv, dd, tar, gzip/gunzip, cat, ...) currently running on your system and displays the percentage of copied data.

I've been using it for the past few months and it's solid.

Answer 6

Noticed this posting was somewhat old. But as I made a solution to the issue at hand, I thought I'd post my solution.

Scenario: We have a very large directory-structure we want to copy, so we issue the command cp -R sourcedir destdir in one terminal window.

Open another terminal window or push the process to the background (Ctrl+Z,bg<CR>), then let's make a progress indicator the ghetto way:

We first get the total size of the directory structure we want to copy, you do this with:

du -sh /path/sourcedirectory/

Convert the result to gigabytes (G), then substitute the n for the result you got in the variable sourcesize. Once you've inserted the sourcesize and destdir (destination directory), run the command below.

while true; do sourcesize=n destdir=/path/destinationdirectory/ copyprogress="$(export | du -sh $destdir | awk '{print $1}' | sed 's/[^0-9.]*//g' )" ; echo "scale=3 ; $copyprogress / $sourcesize * 100" | bc | xargs echo -n ; echo % completed ; sleep 10 ; done

This will print the progress every 10 seconds with messages like:

0.100% completed
0.200% completed
0.300% completed 
... etc

I'm aware that this solution did not address the speed of copying, but it would be possible to add that by by expanding the commands above, possibly making it into a bash script so it would be easier to read. For instance by checking how much data is copied pr. n time unit, the speed should be trivial to calculate and display.

Answer 7

To improve on Marco's answer, today I wrote this version of cpstat, so you can copy many directories and use the last parameter as the destination path. Note that if you use a relative deep path as source, you can, thanks to tar, automatically create the same dependent directory tree.

function cpstat () {

    tar -cf - "${@: 1: $#-1}" |
    pv -s "$( du -cs -BK --apparent-size "${@: 1: $#-1}" |
              tail -n 1 |
              cut -d "$(echo -e "\t")" -f 1)" |
    ( cd "${@: $#}"; tar -xf - )

;}

Answer 8

dd status=progress

Option added in GNU Coreutils 8.24+ (Ubuntu 16.04):

dd if=src of=dst status=progress

Stdout is a periodically updated line like:

462858752 bytes (463 MB, 441 MiB) copied, 38 s, 12,2 MB/s

See also: How do you monitor the progress of dd? | Ask Ubuntu

Answer 9

I found this to use, worked great:

sudo watch lsof -p`pgrep -x cp`

from a comment by Mike on the question how to check progress of running cp

Answer 10

Not in a standard version of cp.

You could background the cp and list the size while waiting for cp to finish.

I use this perl to monitor growth of a single file but it runs externally to the cp (or whatever else I'm doing)

Answer 11

Another simple solution is to just watch the output folder as follows:

Linux/Cygwin: watch -n 10 --differences du -sk --apparent-size <WATCHED_FOLDER>

FreeBSD: gnu-watch -n 10 --differences du -sk <WATCHED_FOLDER>

This handy utility will run du and highlight the differences.

One other potential solution would be to write a simple script that loops, calling du and spitting out a rough estimate of the throughput.

Answer 12

Even though rsync will give you some progress stats, if you want a progress bar you should use pv.

You can use cpv, a wrapper for pv with an easier syntax, more similar to that of cp that you can use to recursively copy directories

You can get it here

Answer 13

Using a tool like mc (midnight commander) can be a good alternate option to other answers.

It's a simple file manager that shows you copy / move progress.

Answer 14

I just had a case where cp was more or less blocked due to a FUSE bug in the receiving filesystem.

If you can't just switch from cp (i.e. because --sparse=always) and want something that is actually reliable for progress, go via /proc.

Every 2.0s: cat /proc/20977/io                                                      Thu Sep 28 04:08:57 2017

rchar: 268573800
wchar: 268500992
syscr: 4112
syscw: 4097
read_bytes: 268697600
write_bytes: 0
cancelled_write_bytes: 0

Answer 15

Use rsync. However, rsync --info=progress2 source dest produces error skipping directory source. You must add a -r or -a option, or similar, to fix this. I recommend -a and also adding --stats to show a nice output summary at the end. Lastly, add time in front to see how long the total process took.

So, do this instead:

time rsync -a --info=progress2 --stats source destination 

See my other answer here for more details.

Using this command I just duplicated a 42GB directory within a fast SSD drive in ~2.5 minutes! Incredible!

Here's the output. Notice that the "real" time this run took was 2.798 seconds. That's because I forgot to use time on the first run where it actually did the copy, so this is just what the output looks like when everything is already the same in source and destination.

$ time rsync -a --info=progress2 --stats mydir mydir_copy
              0   0%    0.00kB/s    0:00:00 (xfr#0, to-chk=0/289187)
Number of files: 289,187 (reg: 266,595, dir: 22,580, link: 12)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 0
Total file size: 48,621,942,935 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 2,096,688
File list generation time: 0.017 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 11,036,784
Total bytes received: 24,420
sent 11,036,784 bytes  received 24,420 bytes  4,424,481.60 bytes/sec
total size is 48,621,942,935  speedup is 4,395.72
real    0m2.798s
user    0m1.115s
sys 0m1.935s

UPDATE:

Here's what it looks like doing a full copy of a 48GB folder. It took 3 min 13.408 sec!

$ time rsync -a --info=progress2 --stats mydir mydir_bak
 48,621,942,066  99%  239.88MB/s    0:03:13 (xfr#266595, to-chk=0/289188)
Number of files: 289,188 (reg: 266,595, dir: 22,581, link: 12)
Number of created files: 289,188 (reg: 266,595, dir: 22,581, link: 12)
Number of deleted files: 0
Number of regular files transferred: 266,595
Total file size: 48,621,942,935 bytes
Total transferred file size: 48,621,942,066 bytes
Literal data: 48,621,942,066 bytes
Matched data: 0 bytes
File list size: 12,515,323
File list generation time: 0.034 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 48,656,219,349
Total bytes received: 5,209,801
sent 48,656,219,349 bytes  received 5,209,801 bytes  251,480,254.01 bytes/sec
total size is 48,621,942,935  speedup is 1.00
real    3m13.408s
user    2m40.937s
sys 1m0.627s

Related:

1. [my answer] Detailed description and examples of many ways to use rsync, including mirroring (with deletion not just addition of files): https://superuser.com/questions/1271882/convert-ntfs-partition-to-ext4-how-to-copy-the-data/1464264#1464264
2. [my answer] https://askubuntu.com/questions/17275/how-to-show-the-transfer-progress-and-speed-when-copying-files-with-cp/1275972#1275972

Answer 16

VCP – copy files in a curses interface (Show the Progress Bar in Linux).

DESCRIPTION: vcp copies files and directories in a curses interface, with text only output available.  its options and output are similar to BSD's cp while adding some new features.

It provides information on:

• files copied and left to copy
• data written and total data size
• data being written every second
• two status bars, one showing current file status, the other total status (except with 1 file, both show current), and percentage

when output is sent to the console:

• a status bar
• size copied and speed

VCP man page: https://linux.die.net/man/1/vcp

Download vcp-2.2.tar.gz from:

http://distfiles.gentoo.org/distfiles/vcp-2.2.tar.gz

Dependence: curses.h (libncurses-dev).

On Ubuntu, install:

sudo apt-get install libncurses-dev

or

sudo apt-get install libncurses5-dev

---

INSTALL vcp:

make && make install

---

Manual:

man vcp

Answer 17

With BSD cp, if you use ctrl+t you will get an update on the percentage, if you must use cp

Answer 18

On specific appliances like VMWare ESXi where you either have no control over what's installed or have no rights to install, you can see percentage per file and speed by using scp on localhost instead of plain cp:

scp -rv item-to-copy your-user@127.0.0.1:/path/to/destination