45

I have this huge folder with thousands of unordered files. Is it feasible to move the first 5000s to a subfolder via the mv command? For now I move files with 

 mv *some_pattern* ./subfolder1/

As for now, I move images quite randomly, it's not really important if there aren't exactly 5000 files in each subfolder. Is there a better way to do it?

Anthon
  • 79,293
Fabinout
  • 553

7 Answers7

70
mv `ls | head -500` ./subfolder1/
Stéphane Chazelas
  • 544,893
schaiba
  • 7,631
  • 22
    (assuming none of the filenames contain space, tab, newline, star, open square bracket, question mark characters or start with - or . and assuming subfolder1 itself does not show up in that list.) – Stéphane Chazelas Dec 13 '13 at 17:05
  • 1
    @StéphaneChazelas if the file contains those, how do we modify the command? – Peiti Li Nov 10 '17 at 23:19
  • 2
    $ sudo mv `ls | head -50000` 01/ sudo: unable to execute /bin/mv: Argument list too long – Brian Thomas Oct 07 '19 at 18:38
  • For me in MacOS the solution worked removing the .

    Something like: mvls | head -500/subfolder1/

     – Flavio Dec 09 '20 at 12:41
  • Regarding the comment above in dealing with spaces, tabs, etc. This command worked nicely after changing $IFS, as my files contained spaces. – s3dev Aug 23 '21 at 08:21
  • mkdir ./subfolder2 | mv ls *.xml | head -500 ./subfolder2/ – Eugene Kaurov Jun 03 '22 at 14:21
19

With zsh and its glob qualifiers:

mv -- *(D.oN[1,5000]) ./subfolder1

To move up to 5000 regular files in the order they are in the directory.

For the first 5000 in the lexicographically sorted list:

mv -- *some_pattern*(D.[1,5000]) ./subfolder1

If you get an error about arg list too long. You can use zsh's buitin mv command by issuing:

zmodload zsh/files

first.

POSIXly:

set --
for f in .*some_pattern* *some_pattern*; do
  [ "$#" -lt 5000 ] || break
  [ -f "$f" ] || continue
  [ -L "$f" ] && continue
  set -- "$@" "$f"
done
[ "$#" -eq 0 ] || mv -- "$@" subfolder1/

On a GNU system, you could also do:

find . ! -name . -prune -name '*some_pattern*' -type f -print0 |
  head -zn 5000 |
  xargs -r0 mv -t ./subfolder1 --
Stéphane Chazelas
  • 544,893
15

A version that is simple and supports special chars, spaces, etc.

ls -Q dir1 | head -1000 | xargs -i mv dir1/{} dir2/

For this to work as-is dir2 must exist and you have to execute it from the parent directory of dir1 and dir2.

This will move 1000 files from dir1 to dir2.

Luis Antolín Cano
  • 603
  • 5
  • 13
  • nice one! ls -Q -S dir1 | head -1000 | xargs -i mv dir1/{} dir2/
    for moving 1000 largest files in dir1 (-S lists file by size)     – oneklc May 03 '18 at 23:05
  • 2
    Note that ls -Q does not produce an output compatible with xargs's expected input format. It helps for file names containing space characters, but not for double quotes or backslashes and harms for file names containing control characters including TAB. – Stéphane Chazelas Jun 29 '18 at 13:17
8
  1. Go to the directory which you want to move files from.
  2. Run below command: 
    find . -name 'Hello*.gz' | head -n 5000 | xargs -d $'\n' mv -t /data01/path/

In the find command, . (dot) denotes current directory

Finds files which start with Hello and end with .gz, first 5,000 files will be moved to /data01/path/.

Walf
  • 1,321
Pratik Hiremath
  • 81
3

You might need to do something like this:

x=1
for file in *
do
    if [ "X$x" = "X#####" ]; then
        break
    fi
    mv $file <destination>
    x=`expr $x + 1`
done

This script works in bash, ksh, sh and multiple UNIX variants.

Karlson
  • 5,875
  • 1
    (provided none of the filenames contain space, tab, newline, star, open square bracket, question mark characters or start with - or . and provided destination itself does not show up in that list.) – Stéphane Chazelas Dec 13 '13 at 17:25
  • @StephaneChazelas True. This is not a complete solution just a method of dealing with the problem. – Karlson Dec 13 '13 at 17:39
2

I was able to succesfully do a move of 50,000 without the mv bash error like this 

 ls | head -50000 | xargs -I{} sudo mv {} 01/

Funny enough, this was on a samba share, so the 50k limit is because Windows Explorer GUI does not like more than 60k files in a dir in general.

Brian Thomas
  • 714
  • Yes solve error zsh: argument list too long: mv in my case ls i | head -500000 | xargs -I {} mv i/{} o/{} – Ax_ Nov 08 '22 at 11:14
2

Version working even in extremely large directories: moving thousand files from directory containing half million:

ls -U . | head -1000 | xargs mv -t 01/

(Note that the syntax with {} is spawning new mv for every file, while this one doesn't)

Honza
  • 129