Use Find to list files and include counter value in output

Question

For performance reasons, I want to use find to list a large number of files, but also include a counter on each line. The following is what I have so far:

local root_dir="."
local ouptut_file="/tmp/foo.txt"
File_Number=99              ## Initial value
echo "" > "${ouptut_file}"  ## Initialize file to empty
find "$root_dir"   

    -type f        

    -depth 2       

    -name '*.xxx'  

    -print0        

| sort -z          

| xargs -0  printf "DoSomething %5d '%s'\n" $[File_Number++] -- 

> "${ouptut_file}"

This outputs

DoSomething    99 '--'
DoSomething     0 'dirA/dirB/file1.xxx'
DoSomething     0 'dirA/dirB/file2.xxx'
DoSomething     0 'dirA/dirB/file3.xxx'

what I want is

DoSomething     100 'dirA/dirB/file1.xxx'
DoSomething     101 'dirA/dirB/file2.xxx'
DoSomething     102 'dirA/dirB/file3.xxx'

How do I include the counter and also eliminate the first spurious line in the output.

System: Am using bash on macOS Ventura 13.0.1

Answer 1

A couple of notes:

1. $[...] is a really old form of arithmetic expansion. These days the bash documentation doesn't even mention it. Use $((...)) instead.
2. xargs will start a new process for each invocation of the command, so any changes to variables in one won't affect other invocations.
3. For printf(1), -- is just like any another argument. What's happening is that xargs runs printf "DoSomething %5d '%s'\n" $[File_Number++] dirA/dirB/file1.xxx dirA/dirB/file2.xxx ..., and printf loops over the arguments to fit the format string requirements. So every alternate filename is actually being treated as a number, resulting in the 0 in the output.

You might want to use something like:

find "$root_dir"   \
    -type f        \
    -depth 2       \
    -name '*.xxx'  \
    -print0        \
| sort -z          \
| while read -r -d '' file; do
     printf "DoSomething %5d '%s'\n" $((File_Number++)) "$file"
  done > "${ouptut_file}"

Also note that if you're actually using this to print commands for running in a shell, you should use %q instead of '%s' for proper quoting. From help printf:

In addition to the standard printf(1) formats, %b means to expand backslash escape sequences in the corresponding argument, and %q means to quote the argument in a way that can be reused as shell input.

Answer 2

I'd use perl:

find "$root_dir"   \
    -type f        \
    -depth 2       \
    -name '*.xxx'  \
    -print0        \
| sort -z          \
| perl -l -0ne 'printf "something %05d %s\n", $., $_' > "$output_file"

If you need to quote the files, putting ' on either side may not be enough depending on what file names may contain and what language those quotes are intended for. For instance, in several languages including some shells, the \ and newline characters are still special within quotes and would need to be escaped. In any case, the ' character cannot be embedded asis in a single quoted string.

If the quoting is for a Bourne-like shell, you can do quoting with a:

sub shquote {return "'" . ($_[0] =~ s/'/'\\''/gr).  "'"}

function which is the safest form of quoting in those shells¹. So here:

perl -l -0ne '
  sub shquote {return "'\''" . ($_[0] =~ '"s/'/'\\\\''/gr"').  "'\''"}
  printf "something %05d %s\n", $., shquote($_)'

Or if you struggle to keep track of all those quotes:

perl -l -0nse '
  sub shquote {return $q . ($_[0] =~ s/$q/$q\\$q$q/gr) . $q}
  printf "something %05d %s\n", $., shquote($_)' -- -q="'"

Also note that outputting to files with fixed name in world writable areas is bad practice from a security standpoint. Someone could create a symlink with the same name to some sensitive file you have write access to for instance. Creating files in /tmp should only be done with randomly generated file names and with O_NOFOLLOW like mktemp does, or in freshly made private directories there (bailing out if it can't be created, like for instance if it already existed).

---

^{¹ Safer than the String::ShellQuote CPAN module in any case which has a few issues.}