cp command does not overwrite existing files

Question

I'm copying files from a git repository into a virtualhost folder, although none of the following commands do overwrite existing files:

sudo cp -R /home/git/repo /var/www/website
sudo cp -R -f /home/git/repo /var/www/website

How can I make sure the cp command overwrites existing files?

Edit: Quick test, copying files from the webdir to the repo does overwrite files, so apparently something (www-data?) is locking the files in the webdir. The mount output:

~$ mount | column -t
sysfs       on  /sys                             type  sysfs       (rw,nosuid,nodev,noexec,relatime)
proc        on  /proc                            type  proc        (rw,nosuid,nodev,noexec,relatime)
udev        on  /dev                             type  devtmpfs    (rw,nosuid,relatime,size=1005964k,nr_inodes=251491,mode=755)
devpts      on  /dev/pts                         type  devpts      (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs       on  /run                             type  tmpfs       (rw,nosuid,noexec,relatime,size=204840k,mode=755)
/dev/vda1   on  /                                type  ext4        (rw,relatime,errors=remount-ro,data=ordered)
securityfs  on  /sys/kernel/security             type  securityfs  (rw,nosuid,nodev,noexec,relatime)
tmpfs       on  /dev/shm                         type  tmpfs       (rw,nosuid,nodev)
tmpfs       on  /run/lock                        type  tmpfs       (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs       on  /sys/fs/cgroup                   type  tmpfs       (ro,nosuid,nodev,noexec,mode=755)
cgroup      on  /sys/fs/cgroup/systemd           type  cgroup      (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore      on  /sys/fs/pstore                   type  pstore      (rw,nosuid,nodev,noexec,relatime)
cgroup      on  /sys/fs/cgroup/hugetlb           type  cgroup      (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup      on  /sys/fs/cgroup/freezer           type  cgroup      (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup      on  /sys/fs/cgroup/cpu,cpuacct       type  cgroup      (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup      on  /sys/fs/cgroup/memory            type  cgroup      (rw,nosuid,nodev,noexec,relatime,memory)
cgroup      on  /sys/fs/cgroup/devices           type  cgroup      (rw,nosuid,nodev,noexec,relatime,devices)
cgroup      on  /sys/fs/cgroup/net_cls,net_prio  type  cgroup      (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup      on  /sys/fs/cgroup/pids              type  cgroup      (rw,nosuid,nodev,noexec,relatime,pids)
cgroup      on  /sys/fs/cgroup/blkio             type  cgroup      (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup      on  /sys/fs/cgroup/cpuset            type  cgroup      (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup      on  /sys/fs/cgroup/perf_event        type  cgroup      (rw,nosuid,nodev,noexec,relatime,perf_event)
systemd-1   on  /proc/sys/fs/binfmt_misc         type  autofs      (rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
debugfs     on  /sys/kernel/debug                type  debugfs     (rw,relatime)
mqueue      on  /dev/mqueue                      type  mqueue      (rw,relatime)
hugetlbfs   on  /dev/hugepages                   type  hugetlbfs   (rw,relatime)
fusectl     on  /sys/fs/fuse/connections         type  fusectl     (rw,relatime)
tmpfs       on  /run/user/1000                   type  tmpfs       (rw,nosuid,nodev,relatime,size=204840k,mode=700,uid=1000,gid=1000)

And:

df -h /var/www
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        30G   20G  8.9G  69% /

Did the above command give you an error or a message? Try adding the -v option to understand better the problem. — Giacomo Catenazzi, Oct 19 '16 at 07:29
No errors are given. -v option shows all files being copied, but again, not overwritten. — Kurt Van den Branden, Oct 19 '16 at 22:06
The output of mount may help a lot. I suspect a network mount that may be configured in strange ways. — grochmal, Oct 20 '16 at 22:35
Are you using the genuine cp command (i.e. /usr/bin/cp), or is cp an alias, such as cp -n? — countermode, Oct 21 '16 at 06:37
cp is not an alias. Tested it with \cp and alias cp That last one returned -bash: alias: cp: not found — Kurt Van den Branden, Oct 21 '16 at 07:23
Do you have a selinux installed? getenforce should tell you. — Kalavan, Oct 21 '16 at 09:36
No. ~$ getenforce The program 'getenforce' is currently not installed — Kurt Van den Branden, Oct 21 '16 at 09:39
Might be stupid question, please don't feel offended, but are you sure you are copying data to the location you really want to? Commands you specified should copy repo folder to /var/www/website/repo. Hope this is what you wanted to achieve. I don't believe linux is copying data and silently drops it. Any message? You could also add -v switch to your copy command to see what is going on. — Kalavan, Oct 21 '16 at 12:49
@roaima ls -l index.html -rw-r--r-- 1 root root 98151 Oct 20 12:56 index.html — Kurt Van den Branden, Oct 21 '16 at 13:49
@Kalavan: Yes, for testing purpose we copy everthing to a subfolder of /var/www/website. — Kurt Van den Branden, Oct 21 '16 at 13:50
Typo edit: df -h /var/www/website/repo/index.html returns /dev/vda1 30G 20G 8.8G 69% / — Kurt Van den Branden, Oct 23 '16 at 11:13
... So what happens instead? Error messages, cp -v, strace, lsattr? — frostschutz, Oct 23 '16 at 11:27
Reduce the problem to a single or a few files (so they fit sanely in the question), the do an ls -l on the source files, and on the destination files before and after the copy. And post the commands you entered exactly. Something like 'ls -l /home/git/repo/foo ; ls -l /var/www/website/repo/foo ; cp -a /home/git/repo /var/www/website/ ; ls -l /var/www/website/repo/foo` — ilkkachu, Oct 23 '16 at 14:04
As it is now, the question doesn't even show the files involved. Comments are a really bad way of adding information, as the formatting doesn't work well. (Also the ls -l listing a couple of comments up doesn't show where that index.html is located.) — ilkkachu, Oct 23 '16 at 14:07
Is the server also running an NFS (server)? If so please show /etc/exports. Or is Samba running? Please also check what could be locking the file: fuser -v /var/www/website/repo/index.html. — Joshua Huber, Oct 25 '16 at 16:31
From someone who's opened a bounty, I would expect a little more effort to include the details. Change your prompt, if you like, with PS1='\$ ', and then show the entire terminal output concisely that illustrates the difficulty you are having. Many good suggestions in the comments on how to do this, but the point is that it should be possible from reading the terminal output alone to see what your problem is. — Wildcard, Oct 26 '16 at 02:04

score 3 · Answer 1 · edited Apr 13 '17 at 12:36

Instead of `cp` I prefer use `tar` for big tree..

Yes, I use near never cp -r syntaxe (except for hardlinking backups, with cp -al, but it's very specific).

Syntax of tar do permit a lot of finest configuration for copying big tree, about what could be or not copied (which files, permissions, owner...) see man tar.

I prefer to be familiar with tar command:

On localhost:

 tar -cpC /path/to/source . | tar -xpC /path/to/target

(note the dot . which specify what to save).

Be familiar with tar could be usefull for remote copy:

ssh user@remotesource tar -zcpC /path/to/source . | tar -zxpC /path/to/target

or

tar -zcpC /path/to/source . | ssh user@targethost tar -zxpC /path/to/target

or even

ssh user@remotesource tar -zcpC /path/to/source . |
    ssh user@targethost tar -zxpC /path/to/target

On remote and/or different hosts, while copying users account, you may like --numeric-owner options...

--one-file-system stay in local file system
--overwrite overwrite existing files
-T, --files-from FILE get names to extract or create from FILE
-z use gzip compression

Instead of `tar`, why not using `cpio`

As imz -- Ivan Zakharyaschev commented out, cpio could do same, but not with same syntax:

cpio require list of inode (file, dir, dev, socket, fifo) that must be stored in archive to be submitted by STDIN.

But cpio offer a mode --pass-through wich let you copy file to another directory.

This make syntaxe stronger, but as we use find to build this list, we could use power of find to copy exactly what we need:

cd /path/to/source
find . \( -type f -o -type d \) ! -name '*.foo' -print0 | 
    cpio --null -pvd /path/to/destination

Used remotely, you could:

ssh user@remotesource /bin/sh <<<'cd /path/to/source;
    find . \( -type f -o -type d \) ! -name '*.foo' -print0 |
    cpio --null -o |
    gzip' |
  ssh user@targethost 'cd /path/to/destination && gunzip | cpio -id'

@imz--IvanZakharyaschev Hmm No: cpio require list of node to by backed up by standard input. Don't have a recursive mode. But yes, you could do same by using cpio... with some more work: ssh source /bin/sh <<<'cd /source/path && find . | cpio -o | gzip' | gunzip | cpio -i — F. Hauri - Give Up GitHub, Oct 23 '16 at 12:38
@user1133275 yes, but tar and cpio is installed with linux standard base. rsync is a strong tool not installed by default everywhere. But yes, this could be done with rsync in very insteresting ways. (This is no more only copying, but syncing!) — F. Hauri - Give Up GitHub, Oct 23 '16 at 13:54

score 1 · Answer 2 · answered Oct 19 '16 at 06:57

1

Since you are running this as root (using sudo), have you checked if there is an alias for cp in root's .bashrc or .profile? Something like:

alias cp="cp -i"

Try to run the commands with '\', like this:

sudo \cp -R /home/git/repo /var/www/website

The \ in \cp will make cp run without any alias.

answered Oct 19 '16 at 06:57

2

I doubt it's about that, cp -i should ask about overwriting, instead of silently skipping, and -f should override -i. Also, most shells don't expand aliases when running non-interactively, in bash it's controlled by shopt expand_aliases which is set by default (only) on interactive shells. Also, .bashrc isn't read by non-interactive shells either. You can test that with doing something like bash -c 'alias ls="ls -l"; ls'. Actually I don't think sudo even runs a shell to execute a command line like that, but runs the command by itself. (try sudo alias) – ilkkachu Oct 19 '16 at 08:35
Yeah, you're right. cp -i should prompt before overwriting. – Oct 19 '16 at 09:37
\cp does not do the trick. The files are not overwritten. – Kurt Van den Branden Oct 19 '16 at 22:07
@KurtVandenBranden Sorry buddy, at this point I am out of ideas! – Oct 20 '16 at 06:54
cp -u but this only works, when the Files that you are copy are newer, than the existing. – Oct 25 '16 at 12:22

Josip Rodin · Answer 3 · 2016-10-23T09:06:31.733

1

It sounds like you want the files to end up in /var/www/website/index.html, but with the above syntax, they'll end up in /var/www/website/repo/index.html.

You can verify if this is indeed happening by checking whether you have a /var/www/website/repo/ directory with new files.

The solution would be to use something like cp -R source/* dest/, or maybe a rsync -a source/ dest/ to be more efficient.

edited Oct 23 '16 at 09:06

answered Oct 22 '16 at 19:58

Josip Rodin

1,138

Yes. And additionally to cp -R source/* dest/ or rsync -a source/ dest/, one other could help here: cp -R source -T dest – imz -- Ivan Zakharyaschev Oct 23 '16 at 09:17
As stated before in one of the comments: For testing purpose, we copy everything to a subfolder of /var/www/website. – Kurt Van den Branden Oct 23 '16 at 11:02
@KurtVandenBranden you stated in a comment earlier you want to operate on /var/www/website/index.html. Is that not true? Please clarify your question, it's apparent that several people now don't understand what you're actually asking. – Josip Rodin Oct 23 '16 at 11:07
@KurtVandenBranden a good place to start would be pasting the exact output of a command that shows the previous state of the directory you want to update, and the exact output of the command that shows you the new state after the update. – Josip Rodin Oct 23 '16 at 11:13

score 1 · Answer 4 · answered Oct 26 '16 at 01:51

Permissions and ownership may matter on your files, but if you don't need anything but 644 and 755 all by the same owner, you could just check out directly from Git:

git --git-dir=/home/git/repo/.git --work-tree=/var/www/website checkout -f HEAD

The -f flag forces the checkout, causing Git to overwrite any changes in /var/www/website with the version saved in Git.

HEAD is of course the pointer to Git's idea of the "current" commit.

You could also explore the git worktree command, but since you won't be doing development in /var/www/website, the above is likely more appropriate.

score 0 · Answer 5 · answered Oct 26 '16 at 00:51

0

I recommend trying sudo cp -Rf /home/git/repo/* /var/www/website, I've always had better luck myself when copying contents of directories instead of directories.

answered Oct 26 '16 at 00:51

Aaron Franke

955

cp command does not overwrite existing files

5 Answers5

Instead of cp I prefer use tar for big tree..

Instead of tar, why not using cpio

Instead of `cp` I prefer use `tar` for big tree..

Instead of `tar`, why not using `cpio`