Simple loop of files - 'file name too long'

Question

I'm trying to do that most basic of things: perform one or more commands on all files within a particular folder.

In this instance, it's converting eps files to PDF using macOS's pstopdf command.

#!/bin/zsh
FILES=$(find "/Users/Ben/Pictures/Stock Illustrations" -type f)
for f in "$FILES"
do
   echo "$f"
   pstopdf "$f"
done

echo "$f" produces a correct list of all the files; but I then get a second list of files -- seemingly from pstopdf itself** -- starting with File name too long: /Users/Ben/Pictures/Stock Illustrations/Flock wallpaper.eps, but the rest of the files are listed correctly.

However, the pstopdf command doesn't create any PDF files.

** I've tried commenting out the echo and the pstopdf command, so I know that each produces a list of filenames.

If I run pstopdf <file.eps> in the Terminal, I get no output to the CLI (e.g. no filename listed), but the file is processed and a PDF file created.

I dare say I could use xargs in the find command, though I prefer the more structured approach of a loop with arguments, not least because it gives the option of multiple commands and other logic, and it's easier to read.

There is a similar question here: I get message "File name too long" when running for..in and touch

But I don't understand how the answer applies. It says "or touch them one by one" (which is what I want), but if I do something like:

FILES=/Users/Ben/Pictures/Stock\ Illustrations/*

I just get "No such file or directory".

Answer 1

FILES=$(find "/Users/Ben/Pictures/Stock Illustrations" -type f)
for f in "$FILES"

As in the linked post, $FILES here is a single string containing all the filenames. With enough files, it's too long for a single filename. With just a few names, you'd be attempting to access a file with embedded newlines in the name.

Here, you'd be better off by having find call pstopf itself, e.g.

$ find "/Users/Ben/Pictures/Stock Illustrations" -type f -print -exec pstopdf {} \;

If you want to use a shell loop over find's output, you'd have to do something like this (in Bash, I didn't pick up the mention of zsh there) :

set -f      # disable globbing
IFS=$'\n'   # set IFS to just the newline (Bash/ksh/zsh, not POSIX)
files=$(find "/Users/Ben/Pictures/Stock Illustrations" -type f)
for f in $files; do
    echo "$f"
    pstopdf "$f"
done

Or use find ... -print0 | while IFS= read -r -d '' f; do ...; done in Bash.

Or use for f in "/Users/Ben/Pictures/Stock Illustrations"/**/*; do ... in Bash (with shopt -s globstar), ksh or zsh (with some variations in specifics between the shells.

Both shell solutions would have issues with filenames containing newlines, which I hope you don't have, but which sadly are allowed filenames.

Answer 2

FILES=$(find "/Users/Ben/Pictures/Stock Illustrations" -type f)

Is the syntax for a scalar variable assignment (in zsh or any other Korn-like shell), so you get a $FILES with only one value which is the whole output of find.

Here, you'd rather want:

files=( ${(0)"$(find "/Users/Ben/Pictures/Stock Illustrations" -type f -print0)"} )

If you wanted $files to be an array with elements being each file found by find. You need find to supply that list 0-delimited and split it on 0s with the 0 parameter expansion flag, as that's the only byte value that cannot occur in a file path.

Here though, you don't need find since zsh globs can do the same with:

files=( "/Users/Ben/Pictures/Stock Illustrations"/**/*(ND.) )

Where **/* does recursive globbing and . is the equivalent of -type f.

To loop over them, you want:

for f in $files; do
  print -r -- $f
  pstopdf $f
done

Or:

for f ($files) {
  print -r -- $f
  pstopdf $f
}

"$files" would join the elements of the array with the first character of $IFS (same as "${files[*]}" in ksh) so would not be what you want. Using $files unquoted in zsh loops over the non-empty elements of the array. "${files[@]}" to loop over all the elements though here it makes no difference as the array is guaranteed not to contain empty elements.