2

I have been working on a bash script for a few days and got stuck with filenames including single and double quotes. Give I want to iterate over the following files in a directory:

'file1.txt'
'file2.txt'
"file3.txt"
file"3.txt
file'4.txt
very 'bad file.txt
also "very bad.txt

I know that this is bad filenaming – but you know, not all people are programmers and know how to proberly name files. So I try to cope with this edge cases.

I would like to loop through the files and use them in some if statements like this for example:

dir="my/dir/"
files="$(ls -N "$dir"*)"

shopt -s extglob

for f in $files; do if [[ "$files" = "(.png|.jpg|.jpeg|.JPG|.JPEG|*.webp)" ]]; then for $e in "$files"; do cp "$e" "$dir""somefilename.pdf" else cp "$f" "$dir""somefilename.pdf" fi done

Now the problem arises, that bash is intepreting the double and single quotes in the filenames and splitting up the files – errors follow…

So my question is: How to cope with these cases in bash?

benjamin10
  • 125
  • 4
  • Sorry forgot that – updated the code… – benjamin10 Dec 19 '23 at 23:15
  • 1
    Good update. In the future, you should include code so readers can easily recreate your test data, i.e. mkdir ./myTestDir ; cd ./myTestDir ; touch \'file1.txt\' ; touch \"file3.txt\" etc. – shellter Dec 20 '23 at 00:29
  • You could just use find instead and do -exec things to the files, and/or have the filenames separated by null characters for xargs to process – Xen2050 Dec 20 '23 at 08:49

1 Answers1

4

Don't parse the output of ls. That way lies madness and all sorts of strife. If you are trying to do things to or with all files in a given directory, let the shell take care of the filenames for you:

workdir="/path/to/directory"
for file in "${workdir}"/*; do
    case "$file" in
        *.png|*.jpg|*.jpeg|*.JPG|*.JPEG|*.webp)
            cp -- "$file" "$dir"/"somefilename.pdf"
            ;;
    esac
done

Note that I have taken the predicate directly from your question, which has the dubious logic of repeatedly overwriting somefile.pdf with each matching file, which is probably not what you want. Addressing that is beyond the scope of the question of iterating over files, the short(er) answer for which is to simply use for file in path/to/place/*; do stuff_to "$file"; done.

DopeGhoti
  • 76,081
  • Sorry I meant cp "$e" "$dir""somename.pdf" – benjamin10 Dec 19 '23 at 23:47
  • Thanks for the answer! The thing why I want to put the files into a list is, that I want the script to use only the files within this moment. The directory is an upload directory, so it might be the case, that some files are uploaded while the script is running and then it might be the case, that the script takes files which are not fully uploaded to the directory. – That is why I came up with the idea to put the files into the variable or an array and work thourgh that files… – benjamin10 Dec 19 '23 at 23:56
  • 1
    In the example solution given, the list of files in directory is captured at run time at the instantiation of the for loop; any files added while the script is running beyond that point will not be taken into account. – DopeGhoti Dec 20 '23 at 00:06
  • 2
    @benjamin10 that's how for f in * works too, but if you wanted to reassure yourself, just build an array using globs: files=( my/dir/* ) and then use that for f in "${files[@]}" – muru Dec 20 '23 at 00:09
  • Okay thanks – very nice to know! So for the sake of excluding infinished upload files I could just add some sort of sleep 10 just to get sure that all the files which are captured have finished uplaoding? – But anyway thanks! I think the answer is very well solves my issue – the other issue seems to be a new one... – benjamin10 Dec 20 '23 at 00:11
  • @muru Thanks for the answer but as far as I see in the follwing commands the filename of the array is split up due to the special characters which are in the filenames…I think I have already tried i but I couldn't solve the issue this way...I give it a shot tmrw.... – benjamin10 Dec 20 '23 at 00:17
  • 1
    @benjamin10 no, used correctly like in my comment, arrays made using globs handle any valid paths correctly. – muru Dec 20 '23 at 00:34
  • You do though have to quote the reference to the variable. So if using @muru's example, be sure to refer to "$f" (or, better, "${f}") rather than just $f. – DopeGhoti Dec 20 '23 at 01:01
  • 1
    FYI, using the shell's * to list files omits those beginning with a dot . (aka the "hidden" files). Unless the shell option dotglob is enabled – Xen2050 Dec 20 '23 at 08:39
  • He there! Thanks a lot for your help – marked the question as answered because I testes your answers and it worked out finally! Besides on question for understanding what is going on in the background: @DopeGhoti : You said that "${f}" is better than "$f" – What is the difference here? Why is it better to use an array instead of the variable? – benjamin10 Dec 20 '23 at 09:01
  • @DopeGhoti Just for completeness: Your answer in short than is: Do not use filenames in variables and arrays in combination with for loops. Let the shell handle things in the for loop itself. Am I right? – benjamin10 Dec 20 '23 at 09:56
  • Using filenames in variables is fine (that's what for f in * does)- it's trying to manually parse the output of a list of filenames that will end in tears. The more you hang out on U&L, the more often you will see the refrain of "don't parse ls", and for good reason (: – DopeGhoti Dec 20 '23 at 23:07
  • 1
    Not quite but close to tears I was I tried in despair to format the output of ls so that it handles the single and double quotes...Thanks anyway! – benjamin10 Dec 21 '23 at 09:05