Move specific files to another folder based on partial pattern in .txt file

Question

I have a folder with various file types, but I am interested in the files with .img as extension, with the following pattern:

ppi_noTD_d0_P_76con_0001.img
ppi_noTD_d0_P_104con_0001.img
ppi_noTD_d0_P_150con_0001.img
ppi_noTD_d0_P_201con_0001.img
etc.

The only changing bit of the file names is the P_XXX part.

I have created two .txt files, stable.txt and recurring.txt, each containing a list of a subset of P_XXX that I would like to use to move the subsets to separate folders (stable and recurring, respectively). For example, P_76 and P_201 are listed in stable.txt, while P_104 and P_154 are listed in recurring.txt.

I tried a for loop to return the relevant P_XXX from the .txt file, so that I can use that to retrieve the matching .img in another for loop from the folder, which should then be moved to the stable folder:

for P in $(< stable.txt); do
     for f in *"$P"*.img; do
         echo mv - "$f" "./stable/$f"
     done
done

It returns the correct number of P_XXX listed, but $f does not return the full filename (only the *P_XXX bit). Strangely enough, it does return the full filename for the last P_XXX in the .txt file (so ppi_noTD_d0_P_201_con_0001.img)

As there seems to be something going wrong with calling $f, I can't move the files to their respective folders (stable and recurring).

How do I solve this?

EDIT:

This is the output that I'm getting:

*.img ./stable/*P_76
*.img ./stable/*P_86
*.img ./stable/*P_89
*.img ./stable/*P_90
*.img ./stable/*P_91
*.img ./stable/*P_99
*.img ./stable/*P_121
*.img ./stable/*P_128
*.img ./stable/*P_132
*.img ./stable/*P_136
*.img ./stable/*P_140
*.img ./stable/*P_144
*.img ./stable/*P_153
*.img ./stable/*P_156
*.img ./stable/*P_162
*.img ./stable/*P_180
*.img ./stable/*P_203
*.img ./stable/*P_205
*.img ./stable/*P_208
*.img ./stable/*P_211
*.img ./stable/*P_215
*.img ./stable/*P_229
*.img ./stable/*P_250
*.img ./stable/*P_256
mv - ppi_noTD_d0_P_257con_0001.img ./stable/ppi_noTD_d0_P_257con_0001.img

fra-san · Accepted Answer · 2020-08-10T19:43:23.537

Your stable.txt file was likely created or edited on a Windows system, where the newline is represented by the carriage rturn + line feed sequence (often referred to using the abbreviation CR LF or the escape sequence \r\n).

For instance, assuming this sample file:

printf '%s\r\n' P_76 P_201 >stable.txt

After the first line is read by your script, the globbing expression *"$P"*.img matches nothing (unless your file names actually contain carriage return characters) and, if no nullglob (or equivalent) option is in effect, the value of f is *P_76\r*.img. When mv - "$f" "./stable/$f" is echoed, the two CR characters cause the subsequent text to be inserted at the beginning of the line, overwriting what was already there.

You can check your files for CR LF newline sequences with

$ cat -v stable.txt
P_76^M
P_201^M

or

$ od -An -c stable.txt
   P   _   7   6  \r  \n   P   _   2   0   1  \r  \n

or

$ file stable.txt
stable.txt: ASCII text, with CRLF line terminators

And you can convert them to the Unix, LF-terminated format with (among other ways):

$ dos2unix stable.txt

Move specific files to another folder based on partial pattern in .txt file

1 Answers1

See also: