4

I have a folder A with hundreds of thousands of files. I would like to move these files to new subfolders S_i, with, say, 100 files in each (the last folder may have less than 100 files)

In other words, if my folder A has:

file1
file2
...
file1000

I would later have:

S_1:    
    file_1
    ...
    file_100    
S_2:
    file_101
    ...
    file_200
...

Before I write a Zsh script manually to do this task (e.g. using variables to count files), I was wondering if there are any readily available tools (like split) that would facilitate this task.

Amelio Vazquez-Reina
  • 41,161
  • 1
    I think this is the best answer: http://stackoverflow.com/questions/17669176/how-can-i-split-a-folder-with-thousands-of-images-into-multiple-sub-folders – Robin Winslow Oct 08 '15 at 16:23

3 Answers3

8

If the filenames don't contain spaces:

i=1;while read l;do mkdir $i;mv $l $((i++));done< <(ls|xargs -n100)

Using parallel:

ls|parallel -n100 mkdir {#}\;mv {} {#}
Lri
  • 5,223
  • The first one assumes file names don't contain whitespace characters, quotes or backslashes (and if used in shells other than zsh, wildcard characters), don't start with -. The second that they don't start with - and don't contain newline characters that the braceccl or extendedglob (common) options are not enabled. You'd also want to use && to separate the two commands (and maybe -p for mkdir to still be successful if the directory already existed beforehand). – Stéphane Chazelas Oct 12 '21 at 05:31
4

With zsh:

autoload zmv
zmv 'file_(<->)' 'S_$((1 + ($1 - 1) / 100))'

If the files are not numbered, but you just want to split that list:

n=0; zmv -Q 'file_*(n)' 'S_$((n++/100+1))'

(n) is to toggle numerical ordering for the list (and you need -Q for that globbing qualifier).

Those call one mv per file. You can make it a bit more efficient by making mv builtin (zmodload zsh/files), or you could do:

files=(file_*(nN))
for ((n=1; $#files; n++)) {
  mv -- $files[1,100] S_$n
  files[1,100]=()
}

The (nN) above are zsh globbing qualifiers which further qualify the glob. n is for numerical sorting, N is to turn on the nullglob option for that glob, that is for the glob to expand to an empty list when there is no match.

(with that one, you could even throw in a mkdir S_$n in case those directories didn't exist beforehand).

Stéphane Chazelas
  • 544,893
0

There is a linux utility for that dirsplit https://linux.die.net/man/1/dirsplit

Talespin_Kit
  • 567