0

How do I find all files in a folder and its subfolders terminated with 3 digits and move them to a new place while keeping the directory structure?

Alternatively, how can I find all files whose names don't end with three digits?

terdon
  • 242,166
wsdzbm
  • 2,836
  • By “keeping the structure” do yo mean moving into subdirectories, recursively? – Incnis Mrsi Sep 11 '15 at 19:06
  • 1
    Unless I'm missing something, all files matching or not matching a pattern = all files. – don_crissti Sep 11 '15 at 19:14
  • @IncnisMrsi Yes, keep the directory tree, e.g. move /old/a/a001 to /new/a/a001, /old/a/b/b012 to /new/a/b/b012 – wsdzbm Sep 11 '15 at 19:18
  • @don_crissti hehe... two independent cases they are. Maybe it's clearer to say "matching (or not matching)". In fact, what I wanna do now is to remove all files not ended with 3 digits, the same effect as moving all files ended with 3 digits and deleting all old files. – wsdzbm Sep 11 '15 at 19:21
  • http://unix.stackexchange.com/q/2161 – don_crissti Sep 11 '15 at 19:48
  • What do you mean by "keeping the structure"? Aren't all files in a single folder? Could you give an example? – terdon Sep 12 '15 at 11:13
  • @terdon referring to "the structure", I mean the tree of subdirectories and files. Not all files in a single folder. – wsdzbm Sep 14 '15 at 09:37
  • Ah, right. Updated my answer then, but I'd use alienth's. – terdon Sep 14 '15 at 10:04

2 Answers2

4

Much cleaner solution, based on an answer linked by @don_crissti. (Rsync filter: copying one pattern only)

rsync -av --remove-source-files --include='*[0-9][0-9][0-9]' --include='*/' --exclude '*' /tmp/oldstruct/ /tmp/newstruct/

And the negation:

rsync -av --remove-source-files --exclude='*[0-9][0-9][0-9]' /tmp/oldstruct /tmp/newstruct/

Original answer:

This should do it. It will find any file in the structure you cd into ending in 3 digits, create a destination folder in the /tmp/newstruct, and move the file.

cd /tmp/oldstruct
find ./ -type f -regextype posix-basic -regex '.*[0-9]\\{3\\}' | 
  while read i; do 
    dest=/tmp/newstruct/$(dirname $i)
    mkdir -vp $dest
    mv -v $i $dest
  done

I'd recommend prepending the mkdir and mv with echo before you actually run it, just to ensure it does what you're expecting.

To negate the 3 digits, simply place do ! -regex instead.


Here is a simpler method which relies upon rsync. However, it does call rsync for every file it finds, so definitely not very efficient.

find ./ -type f -regextype posix-basic -regex '.*[0-9]\{3\}' --exec rsync -av --remove-source-files --relative {} /tmp/newstruct
alienth
  • 2,197
  • The indent for while..done is necessary? – wsdzbm Sep 11 '15 at 19:25
  • For this setup, yes. Although you might be able to find something that won't require loops by using a combination of --include and --exclude flags with rsync. – alienth Sep 11 '15 at 19:29
  • `$ls -R .: a001 aaa b

    ./b: b001 bbb c

    ./b/c: c001 ccc $ find ./ -regextype posix-basic ! -regex '.*[0-9]{3}' -print | while read i; do d=/tmp/newstruct/$(dirname $i); mkdir -vp $d; mv -v $i $d; done mkdir: created directory ‘/tmp/newstruct’ mv: cannot move ‘./’ to ‘/tmp/newstruct/./.’: Device or resource busy ‘./b’ -> ‘/tmp/newstruct/./b’ mv: cannot stat ‘./b/c’: No such file or directory mv: cannot stat ‘./b/c/ccc’: No such file or directory mv: cannot stat ‘./b/bbb’: No such file or directory ‘./aaa’ -> ‘/tmp/newstruct/./aaa’ `

    – wsdzbm Sep 11 '15 at 19:42
  • Ah, since you're negating, it's trying to move the directories as well. Add -type f to the find params. – alienth Sep 11 '15 at 19:44
  • I didn't notice the '!' ...... Perfect now!!! – wsdzbm Sep 11 '15 at 19:47
  • @alienth: You disuse find’s -exec because of overhead? Or? – Incnis Mrsi Sep 11 '15 at 20:03
  • Didn't use -exec because I couldn't easily think of a way to duplicate the structure. Specifically the parts where I'm fetching the dirname and creating that directory. I could be missing something, tho. – alienth Sep 11 '15 at 20:05
  • @don_crissti there is no problem. I didn't mean ONLY three digits. – wsdzbm Sep 11 '15 at 20:08
  • BTW, find finds the regexp in the full path but not the filename only? I found '.*[0-9]\{3\}' matches filename like 111 and the output filename is ./111 – wsdzbm Sep 11 '15 at 20:11
  • Without the -type f, the find will match directory names ending in 3 digits (not sure that is what you're referring to, tho). – alienth Sep 11 '15 at 20:15
  • @don_crissti Came up with an -exec solution which relies on rsync and added it above. Although that's a lot of rsync calls, potentially. – alienth Sep 11 '15 at 20:17
  • @don_crissti Just read up on that, and it is likely a better answer. One thing I'm trying to figure out on it is the regex syntax of --include, which isn't included in that answer. The rsync docs don't seem to indicate that it supports quantifiers? – alienth Sep 11 '15 at 20:23
  • @don_crissti yeah, looks as though quantifiers aren't supported. Not a big issue for this solution, though. Answer updated. – alienth Sep 11 '15 at 20:28
  • Please don't add edit N to your answer. Just edit it. We don't need to know the history. Think of this site as a wiki, other people will read this answer and they don't need to know that it has gone through N revisions. – terdon Sep 12 '15 at 10:19
  • Also, please note that your find approach will fail if any of your file names contain newlines or glob characters. There's no need to loop over find's output, you can use -exec instead. – terdon Sep 12 '15 at 10:31
0

You can do this with bash :

## Make ** match all files and 0 or more dirs and subdirs
shopt globstar
## Iterate over all files and directories
for f in **; do 
    ## Get the name of the parent directory of the 
    ## current file/directory
    dir=$(dirname "$f"); 
    ## If this file/dir ends with 3 digits and is a file
    if [[ $f =~ [0-9]{3} ]] && [ -f "$f" ]; then 
        ## Create the target directory
        mkdir -p targetdir1/"$dir"
        ## Move the file
        mv "$f" targetdir1/"$f" 
    else 
        ## If this is a file but doesn't end with 3 digits
        [ -f "$f" ] && 
        ## Make the target dir
        mkdir -p targetdir2/"$dir" &&
        ## Move the file
        mv "$f" targetdir2/"$f" 
    fi 
done
terdon
  • 242,166
  • @don_crissti ah, yes, you may well be right. The OP also said that all files are in a folder so I'm not sure what they want. And thanks, second command fixed. – terdon Sep 12 '15 at 11:14
  • No problem. I had initially wanted to vote to close it as a dupe of the other one but due to the fact the question is quite confusing I ended up leaving just a comment (although it's almost a duplicate imo). – don_crissti Sep 12 '15 at 11:18