How can I store a string literal to a variable with no substitution or interpretation?

Question

To flatten a directory structure, I can do this:

find . -type f -exec sh -c 'mv "{}" "./`basename "{}"`"'  \;

I want to store the following in my profile as $FLATTEN

-exec sh -c 'mv "{}" "./`basename "{}"`"'  \;

so that later I can just execute find . $FLATTEN

I'm having trouble storing the variable because it gets interpreted too early. I want it to be stored as a string literal and interpreted only in usage on the shell, not when sourced.

I'm trying to put a command in a variable, but the complex cases always fail! — glenn jackman, Sep 18 '17 at 01:25

Stéphane Chazelas · Accepted Answer · 2017-09-18T08:30:51.057

If using GNU mv, you should rather do:

find . -type f -exec mv -t . {} +

With other mvs:

find . -type f -exec sh -c 'exec mv "$@" .' sh {} +

You should never embed {} in the sh code. That's a command injection vulnerability as the names of the files are interpreted as shell code (try with a file called `reboot` for instance).

Good point for quoting the command substitution, but because you used the archaic form (`...` as opposed to $(...)), you'd need to escape the inner double quotes or it won't work in sh implementations based on the Bourne shell or AT&T ksh (where "`basename "foo bar"`" would actually be treated as "`basename " (with an unmatched ` which is accepted in those shells) concatenated with foo and then bar"`").

Also, when you do:

mv foo/bar bar

If bar actually existed and was a directory, that would actually be a mv foo/bar bar/bar. mv -t . foo/bar or mv foo/bar . don't have that issue.

Now, to store those several arguments (-exec, sh, -c, exec mv "$@" ., sh, {}, +) into a variable, you'd need an array variable. Shells supporting arrays are (t)csh, ksh, bash, zsh, rc, es, yash, fish.

And to be able to use that variable as just $FLATTEN (as opposed to "${FLATTEN[@]}" in ksh/bash/yash or $FLATTEN:q in (t)csh), you'd need a shell with a sane array implementation: rc, es or fish. Also zsh here as it happens none of those arguments is empty.

In rc/es/zsh:

FLATTEN=(-exec sh -c 'exec mv "$@" .' sh '{}' +)

In fish:

set FLATTEN -exec sh -c 'exec mv "$@" .' sh '{}' +

Then you can use:

find . -type f $FLATTEN

@haventchecked The second sh is what goes into the inline script's $0 (what gives a name to that inline script) — Stéphane Chazelas, Sep 18 '17 at 07:14
thank you, that was enough to find another source https://unix.stackexchange.com/a/226898/118708
I really like your answer and appreciate you pointing out the defects in my approach. I didn't know what I didn't know about it yet. — haventchecked, Sep 18 '17 at 07:42
@haventchecked, you're welcome. See edit for another issue in your code I forgot to mention in the first place. — Stéphane Chazelas, Sep 18 '17 at 08:31

Kusalananda · Answer 2 · 2017-09-17T20:34:20.773

It would be better to use a shell function for this, and to get the -exec right (don't put {} in a subshell):

flatten () {
     ( cd "${1:-.}" && 
       find . -type f -exec sh -c 'for n; do test -e "${n##*/}" || mv "$n" "${n##*/}"; done' sh {} + )
}

This also doesn't need to call the external utility basename and will not try to overwrite already existing things.

You would use this by just typing flatten and it will act on the current directory. Giving it a directory name will make it act on that (copying all files under that directory to the top of that directory).

score 3 · Answer 3 · edited Sep 18 '17 at 07:39

3

How about a function?

flatten(){
  find "$@" -type f -exec sh -c 'mv -- "$0" "${0##*/}"' {} \;
}

Usage:

> flatten .

If you use zsh, there's the -g option of alias for that. It lets you define an alias which is inserted globally, not just at the command name.

edited Sep 18 '17 at 07:39

Stéphane Chazelas

544,893

answered Sep 17 '17 at 19:37

dessert

1,687

Looping over find’s output is bad practice. – Scott - Слава Україні Sep 17 '17 at 20:14
@Scott You're right, I reverted it – thanks for the feedback! – dessert Sep 17 '17 at 20:19
This will copy all files in the given directory (or under) to the current directory. This may be unexpected. I dunno. – Kusalananda Sep 17 '17 at 20:33
@Kusalananda I wondered that myself, but OP clearly wants it this way. Is it a bug? Is it a feature? – dessert Sep 17 '17 at 20:36
@dessert Note that the OP is defaulting to the current directory, always. To move all files below path/dir into path/dir you would have to cd there first and then move to the current directory, alternatively pass the path/dir as an argument to the sh -c shell and use it in the to augment the destination path. – Kusalananda Sep 17 '17 at 20:40

Hauke Laging · Answer 4 · 2017-09-17T19:44:36.340

3

A function is probably the best way. Otherwise you need an array or eval:

find_array=(-exec sh -c 'mv "{}" "./`basename "{}"`"'  \;)
find . "${find_array[@]}"

or

FLATTEN="-exec sh -c 'mv \"{}\" \"./`basename \"{}\"`\"'  \;"
eval find . $FLATTEN

edited Sep 17 '17 at 19:44

answered Sep 17 '17 at 19:40

Hauke Laging

90,279

How can I store a string literal to a variable with no substitution or interpretation?

4 Answers4