Transforming a list of files separated by new lines into a variable

Question

Goal: I'm trying to find all instances of *.clj or *.cljs files recursively within a directory, store them in a string variable (separated by new lines) and then transform them.

So if the following clj(s) files are found within my directory dir1:

/dir1/dir2/hello1.clj
/dir1/dir2/hello2.clj
/dir1/dir2/hello3.cljs
/dir1/dir2/hello4.clj
/dir1/dir2/hello5.cljs

And my transformation is, let's say, merely to return the basename of each of these strings:

/dir1/dir2/hello1.clj -> hello1.clj
/dir1/dir2/hello2.clj -> hello2.clj
/dir1/dir2/hello3.clj -> hello3.clj
/dir1/dir2/hello4.clj -> hello4.clj
/dir1/dir2/hello5.clj -> hello5.clj

Then how can I write a function f such that

$ VAR=$(f dir1)

satisfies

$ echo "$VAR"
  hello1.clj
  hello2.clj
  hello3.clj
  hello4.clj
  hello5.clj

?

Attempt:

I know that I can generate the .clj and .cljs files of a directory via

FOUND_FILES=$(find "dir1" -type f -regex ".*\.\(clj\|cljs\)")

and that I can use the basename command to get the basename of a file. How to do the rest?

You can safely assume that there no files with whitespace characters (especially spaces) in their names? — phk, Mar 13 '16 at 12:41

Chris Down · Answer 1 · 2016-03-13T15:17:09.840

3

You can do this with globs and parameter expansion. You don't need to use find if you have a version of bash that has globstar to enable ** syntax (bash4+).

# Enable `**`, and expand globs to 0 elements if unmatched
shopt -s globstar nullglob
# Put all files matching *.clj or *.cljs into ${files[@]} recursively
files=(dir1/**/*.clj{,s})
# Print all files delimited by newlines, with all leading directories stripped
printf '%s\n' "${files[@]##*/}"

To apply some arbitrary transformation, replace the last line with:

for file in "${files[@]}"; do
    some-arbitrary-transformation <<< "$file"
done

edited Mar 13 '16 at 15:17

answered Mar 13 '16 at 12:20

Chris Down

125,559
25
270
266

In my actual use case, I'm not trying to get a file's basename, but rather have my own transformation t in mind. How could I apply t to each line of files as you've got it here? – George Mar 13 '16 at 12:28
@George See my edit. – Chris Down Mar 13 '16 at 15:17

score 1 · Answer 2 · answered Mar 13 '16 at 12:30

1

shopt -s globstar nullglob
var="$(echo **/*.clj **/*.cljs | xargs -n1 basename)"
echo "$var"

answered Mar 13 '16 at 12:30

Cyrus

12,309

This works, but doesn't work if basename is a custom function defined above. How can it made to work for custom functions defined within the same shell script? – George Mar 13 '16 at 12:39
Add full path to basename, e.g. /usr/bin/basename – Cyrus Mar 13 '16 at 12:43
That assumes that the custom function is contained within a separate shell script. Is there a way to do use a custom function defined within the same shell script (i.e. custom_function is defined a few lines above)? – George Mar 13 '16 at 12:48
or use GNU find: var=$(find "dir1" -type f -regex ".*\.$clj\|cljs$$" -printf "%f\n") – Cyrus Mar 13 '16 at 13:26
This breaks on whitespace due to failure to use a delimiter that's not permitted in POSIX filenames. – Chris Down Mar 13 '16 at 15:16

Transforming a list of files separated by new lines into a variable

2 Answers2