22

Recursively iterating through files in a directory can easily be done by:

find . -type f -exec bar {} \;

However, the above does not work for more complex things, where a lot of conditional branches, looping etc. needs to be done. I used to use this for the above:

while read line; do [...]; done < <(find . -type f)

However, it seems like this doesn't work for files containing obscure characters:

$ touch $'a\nb'
$ find . -type f
./a?b

Is there an alternative that handles such obscure characters well?

  • 1
    find ... -exec bash -c 'echo filename is in \$0: "$0"' {} \; is a better way to do it. – jw013 Jun 26 '14 at 14:08
  • You can solve this, and keep your original design, by changing your read line to IFS= read -r line. The only character that will break it then is a newline. – phemmer Jun 26 '14 at 14:43
  • 1
    @Patrick, but filenames can contain newlines. That's why -d $'\0' is preferable. – godlygeek Jun 26 '14 at 15:14

4 Answers4

10

Yet another use for safe find:

while IFS= read -r -d '' -u 9
do
    [Do something with "$REPLY"]
done 9< <( find . -type f -exec printf '%s\0' {} + )

(This works with any POSIX find, but the shell part requires bash. With *BSD and GNU find, you can use -print0 instead of -exec printf '%s\0' {} +, it will be slightly faster.)

This makes it possible to use standard input within the loop, and it works with any path.

l0b0
  • 51,350
  • 2
    Because I had to look it up: "read ... If no names are supplied, the line read is assigned to the variable REPLY."

    So do echo "Filename is '$REPLY'"

     – Andrew Oct 07 '19 at 17:45
9

Doing this is as simple as:

find -exec sh -c 'inline script "$0"' {} \;

Or...

find -exec executable_script {} \;
mikeserv
  • 58,310
  • The first example gave a whole bunch of inline: not found errors for me, but this does what I expect find -exec sh -c 'echo inline script "$0"' {} \;. – Liam May 24 '21 at 16:05
6

The simplest (yet safe) approach is to use shell globbing:

$ for f in *; do printf ":%s:\n" "$f"; done 
:a b:
:c
d:
:-e:
:e  f:
h:

To make the above recurse into subdirectories (in bash), you can use the globstar option; also set dotglob to match files whose name begins with .:

$ shopt -s globstar dotglob
$ for f in **/*; do printf ":%s:\n" "$f"; done 
:a b:
:c
d:
:-e:
:e  f:
:foo:
:foo/file1:
:foo/file two:
h:

Beware that up to bash 4.2, **/ recurses into symbolic links to directories. Since bash 4.3, **/ recurses only into directories, like find.

Another common solution is to use find -print0 with xargs -0:

$ touch -- 'a b' $'c\nd' $'e\tf' $'g\rh' '-e'
$ find . -type f -print0 | xargs -0 -I{} printf ":%s:\n" {}
h:/g
:./e    f:
:./a b:
:./-e:
:./c
d:

Note that the h:/g is actually correct since the file name contains a \r.

terdon
  • 242,166
5

It's a bit difficult to do your read loop portably, but for bash in particular you can try something like this.

Relevant portion:

while IFS= read -d $'\0' -r file ; do
        printf 'File found: %s\n' "$file"
done < <(find . -iname 'foo*' -print0)

That instructs find to print its output delimited by NUL characters (0x00), and read to fetch NUL-delimited lines (-d $'\0') without handling backslashes as escapes for other characters (-r) and not do any word splitting on the lines (IFS=). Since 0x00 is a byte that can't occur in filenames or paths in Unix, this should handle all of your weird filename problems.

Community
  • 1
godlygeek
  • 8,053