218

I have a folder structure with a bunch of *.csv files scattered across the folders. Now I want to copy all *.csv files to another destination keeping the folder structure.

It works by doing:

cp --parents *.csv /target
cp --parents */*.csv" /target
cp --parents */*/*.csv /target
cp --parents */*/*/*.csv /target
...

and so on, but I would like to do it using one command.

Wilfred Hughes
  • 145

7 Answers7

226

find has a very handy -exec option:

find . -name '*.csv' -exec cp --parents \{\} /target \;
Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
msb
  • 2,654
  • 82
    Probably because of this {} ; – igo Mar 04 '15 at 13:56
  • 18
    '{}' works just as well – OrangeDog Dec 07 '15 at 17:44
  • Reading the find manual I just found out it's recommended to use -execdir instead of -exec. The manual says: There are unavoidable security problems surrounding use of the -exec action; you should use the -execdir option instead. – ArianJM Apr 15 '16 at 09:42
  • 3
    Even though -execdir is safer than -exec, simply replacing one with the other does not preserve the folder structure as intended. – sshine Aug 23 '16 at 11:33
  • 3
    Why I get message 'Omitting directory' when I try to copy them with your command ? – Vicky Dev Sep 06 '16 at 07:32
  • 8
    Can you explain why the braces have to be escaped here? – Noumenon Oct 23 '16 at 03:26
  • @VickyDev your directory hidden, i.e., it's name began with .? – KernelPanic Jan 05 '17 at 13:59
  • This is much slower than the xargs method, at least on Windows: 4 seconds vs 4 minutes. – piedar May 05 '17 at 15:51
  • "Why I get message 'Omitting directory' ?" It's because at each level that it's copying, there are subdirs; and cp is telling you that these subdirs are not being copied. You adapted the command to your needs, and whatever you have in -name includes subdirs, which in the "csv" example, it didn't. You would need -r to recursively copy the subdirs; but you don't want that, because copying the subdirs is being covered by the find itself. Depending on what you're doing, obviously. Maybe you want. But most probably not. :$ – msb Mar 05 '18 at 18:52
  • 2
    @Noumenon the help page for find says: "-exec command ; Execute command; true if 0 status is returned. All following arguments to find are taken to be arguments to the command until an argument consisting of ; is encountered. The string {} is replaced by the current file name being processed everywhere it occurs in the arguments to the command, not just in arguments where it is alone, as in some versions of find. Both of these constructions might need to be escaped (with a '\') or quoted to protect them from expansion by the shell." – Paul Rougieux Sep 14 '18 at 11:37
  • 2
    When using find in a terminal on a mac, the --parents option is not recognized. I'm using cpio as per the answer from @iain there. – parvus Jun 11 '19 at 07:10
  • 2
    This answer saved my day! – Jinhua Wang Oct 03 '19 at 13:54
  • This answer does not save the structure. – Miguel Nov 16 '22 at 12:42
  • @Miguel Maybe it's your version of "find", or "cp", or shell. This works fine in a standard linux system with bash. Let us know what are your specs and maybe someone can help... or you can post a new question, with all the details. – msb Nov 28 '22 at 19:15
  • How can this be modified to copy a folder rather than a file. – mindlessgreen Sep 05 '23 at 16:31
96

You could also use rsync for this.

$ rsync -a --prune-empty-dirs --include '*/' --include '*.csv' --exclude '*' source/ target/

If you want to keep empty directories from the source tree, skip the --prune-empty-dirs option:

$ rsync -a --include '*/' --include '*.csv' --exclude '*' source/ target/

If you do not want symlinks, modification dates, file permissions, owners etc. preserved, please replace -a with another combination of -rlptgoD. ;-)

Dubu
  • 3,723
  • 3
    -m is a shortcut for --prune-emty-dirs. – Geremia Sep 24 '16 at 00:59
  • 1
    Also, the -R option can be added to copy the parent directory structure of source. (cf. my answer here.) – Geremia Sep 24 '16 at 01:52
  • Unfortunately it seems that rsync is excruciating slow at finding matching files though. It takes like 100 more time than find. Is there anything that can be done about this? – Kvothe May 11 '23 at 14:38
67

You can use find and cpio in pass through mode

find . -name '*.csv' | cpio -pdm  /target

This will find all .csv files in the current directory and below and copy them to /target maintaining the directory structure rooted in ..

If you use

find /path/to/files -name '*.csv' | cpio -pdm /target

it will find all of the file in /path/to/files and below and copy them to /target/path/to/files and below.

  • 4
    I tried all the answers top down up to this one, and this was the only one that worked on the first attempt – OscarRyz Jun 24 '16 at 19:10
  • 2
    This should be the accepted answer. – Philippe Remy Aug 25 '20 at 00:30
  • 1
    I found the GNU docs helpful: "In copy-pass mode, [requested by the -p option, cpio] reads the list of files to copy from the standard input; the directory into which it will copy them is given as a non-option argument." ... "-d Create leading directories where needed." ... "-m Retain previous file modification times when creating files." – Nickolay Sep 02 '20 at 21:48
  • I want to do the same as the first line but copy all the contents of a folder and not only the csv files. – seralouk Nov 15 '22 at 12:39
40

The cp command permits multiple source arguments:

cp **/*.csv --parents ../target

CAVEAT: I'm using a recursive glob here; this is the globstar option in Bash 4+ and ksh, and is supported by default in zsh. Recursive globs do not match hidden files and folders, and the some implementations follow symlinks while others do not.

If your shell doesn't support recursive globs, or if you'd prefer not to use them, you can do the following:

  • *.csv */*.csv */*/*.csv */*/*/*.csv -- this is of course very redundant and requires knowing how deep your directory structure is.
  • $(find . -name '*.csv') -- This will match hidden files and folders. find also supports specifying whether or not symlinks are followed, which may be useful.
Community
  • 1
Kyle Strand
  • 739
  • this is exactly what i tried (the recursive glob) and it found some but not all? pretty weird. I got the same exact result when using the npm copyfiles script but if i use the find command it finds everything... – Randyaa Sep 02 '16 at 15:22
  • @Randyaa I'll need some more details on which files, exactly, weren't found in order to help you. You may find the discussion here and continued here about the precise behavior of the recursive glob useful. – Kyle Strand Sep 02 '16 at 15:50
  • 1
    turns out recursive glob wasn't enabled for some reason... I've never run into this before but i corrected it with just an execution of shopt -s globstar immediately before my command and all is well. Thanks for the follow up! – Randyaa Sep 02 '16 at 15:52
  • 3
    --parents was what I was looking for. thanks – Josh Oct 03 '19 at 22:35
31

This one worked for me:

find -name "*.csv" | xargs cp --parents -t /target

If you have file names with spaces, add options -print0 and -0 like suggested in one of the comments:

find -name "*.csv" -print0 | xargs -0 cp --parents -t /target

marko
  • 419
  • 2
    Best answer with most simple syntax. Works just fine and is easy to remember. In many cases find [things] | xargs [do stuff] is very powerfull. – oh really Dec 04 '18 at 07:56
  • Agreed. Very straightforward compared to some of the other answers. – Tim B Mar 21 '19 at 12:40
  • 2
    Breaks if you have spaces in filenames – Michele Piccolini Sep 16 '19 at 14:58
  • 2
    @MichelePiccolini Spaces in filenames can be handled with find -print0 and xargs -0. – pasztorpisti Sep 19 '19 at 15:22
  • 1
    @pasztorpisti, is there any downside to that? If not should the answer not simply be edited to include your suggestion. – Kvothe Mar 09 '21 at 18:10
  • @Kvothe Unix pathnames are zero-terminated byte arrays. All byte values are allowed (even newline) and the only special byte values are slash and zero. For this reason -print0 with -0 is probably the most reliable way to transfer pathnames from find to xargs. Note that without -0 the xargs command interprets the input in a special way that is described in the specification of xargs. (It allows single and double quoted strings and escaping.) This default behavior (that is rarely what people want) can be changed not only with -0 but also with the gnu specific -d. – pasztorpisti Mar 11 '21 at 10:21
12

From rsync's manpage:

-R, --relative

Use relative paths. This means that the full path names specified on the command line are sent to the server rather than just the last parts of the filenames. This is particularly useful when you want to send several different directories at the same time. For example, if you used this command:

rsync -av /foo/bar/baz.c remote:/tmp/

... this would create a file named baz.c in /tmp/ on the remote machine. If instead you used

rsync -avR /foo/bar/baz.c remote:/tmp/

then a file named /tmp/foo/bar/baz.c would be created on the remote machine, preserving its full path. These extra path elements are called "implied directories" (i.e. the "foo" and the "foo/bar" directories in the above example).

So, this would work, too:

rsync -armR --include="*/" --include="*.csv" --exclude="*" /full/path/to/source/file(s) destination/
Geremia
  • 1,183
  • Thanks for providing this answer. Compatible with BSD userland (macOS), and paths with spaces in them. – myxal Feb 14 '20 at 09:57
9

Assuming you want to replicate this structure from ./source to ./destination:

cd source
find . -name "*.csv" | xargs tar cvf - | (cd ../destination ; tar xfp -)

I'm prepared to count that as one line, the cd source being a shell builtin.

MadHatter
  • 644