Escaping double quotes for variables in bash and qmake

Question

I have wrote following meta-project file for qmake that is designed to build all .pro files in the following subdirectories (for historical reasons and because of other toolchains, the file names do not match folder names). Which essentially boils down to something like this (throwing aside project-specific stuff)

THIS_FILE=make-all.pro
TEMPLATE=subdirs
FIND= "find -name \'*.pro\' -printf \'%p\\n\'"
AWK= "awk \'{$1=substr($1,3); print}\'"
RMTHIS= "awk \'!/$$THIS_FILE/\'"
SUBDIRS= $$system($$FIND | $$AWK | $$RMTHIS )

I need to get a list of folders that contain .pro files within a Bash script, so I decided to copy the method

#!/bin/bash
FIND="find -name '*.pro' -printf '%h\\n"
AWK="awk '{\$1=substr(\$1,3);printf}'"
SUBDIRS=$($FIND | $AWK)

Apparently this doesn't work, awk was spewing error invalid char '' in the expression. Trying to execute same lines in Bash directly had shown that awk actually works only if double quotes are used

find -name '*.pro' | awk "{\$1=substr(\$1,3);printf}"

Replacing the line in question with

AWK='awk "{$1=substr($1,3);printf}" '

gave no working result, the output of script is empty, unlike the output of manually entered command. Apparently

 find -name '*.pro'

In script finds only files in current folder, while its counterpart in command line of bash find it in subfolders. What is wrong and why qmake works differently as well?

@wildcard not relevant. in both cases resulting string is used by toolchain. in first case it's only mechanism possible in qmake — Swift, May 03 '17 at 16:13

7z. · Answer 1 · 2017-05-03T20:04:55.147

1

If all you need to do is find directories that directly contain a file with a .pro extension, you can use xargs, and optionally uniq, if you want to only list a directory once, even if it contains multiple pro files:

find -type f -name "*.pro" | xargs -I{} dirname {} | uniq

If you need absolute paths rather than relative ones:

find -type f -name "*.pro" | xargs -I{} dirname {} | xargs readlink -f | uniq

As you requested, a solution using awk:

find -type f -name "*.pro" | awk -F/ 'BEGIN { OFS="/" } { $1=""; $NF=""; print $0 }' | cut -c 2- | uniq

Which gives you exactly what you need; relative paths without the prepended ./.

edited May 03 '17 at 20:04

answered May 03 '17 at 14:24

7z.

86

1

See Why is looping over find's output bad practice? – Wildcard May 03 '17 at 16:11
i need relative paths without starting ./ that's what awk is for. good point at foolprofing output though . what i dont get why same commandrequires different quotes in different contexts, what's the rule? – Swift May 03 '17 at 16:21
i'm not able to get output from find still, it finds only file in current directory, while run from script – Swift May 03 '17 at 16:31
sad that I cant do more than one plus, a great awk hint. – Swift May 04 '17 at 13:05

score 1 · Accepted Answer · answered May 03 '17 at 23:56

I'm not familiar with qmake syntax, but from your sample it uses quotes and variables in very different ways than shells. So you can't just use the same code.

http://unix.stackexchange.com/questions/131766/why-does-my-shell-script-choke-on-whitespace-or-other-special-characters covers what you need to know, so here I'll just summarize what's relevant for your question.

In a nutshell, you cannot simply stuff a shell command into a string. Shells such as bash do not parse strings recursively. They parse the source code and build commands as lists of strings: a simple command consists of a command name (path to an executable, function name, etc.) and its arguments.

When you use an assignment like AWK="awk '{\$1=substr(\$1,3);printf}'", this sets the variable AWK to a string; when you use the variable as $AWK outside double quotes, this turns the value of the variable into a list of strings by parsing it at whitespace but it does not parse it as shell code, so characters like ' end up being literally in the argument. This is rarely desirable, which is why the general advice on using variables in the shell is to put double quotes around variable expansions unless you know that you need to leave them off and you understand what this entails. (Note that my answer here does not tell the whole story.)

In bash, you can stuff a simple command into an array.

#!/bin/bash
FIND=(find -name '*.pro' -printf '%h\\n')
AWK=(awk '{$1=substr($1,3);print}')
SUBDIRS=$("${FIND[@]}" | "${AWK[@]}")

But usually the best way to store a command is to define a function. This is not limited to a simple command: this way you can have redirections, variable assignments, conditionals, etc.

#!/bin/bash
find_pro_files () {
  find -name '*.pro' -printf '%h\\n'
}
truncate_names () {
  awk '{$1=substr($1,3);print}'
}
SUBDIRS=$(find_pro_files | truncate_names)

I'm not sure what you're trying to do with this script (especially given that you change the find and awk code between code snippets), but there's probably a better way to do it. You can loop over *.pro files in subdirectories of the current directory with

for pro in */*.pro; do
  subdir="${pro%/*}"
  basename="${pro##*/}"
  …
done

If you want to traverse subdirectories recursively, in bash, you can use **/*.pro instead of */*.pro, but beware that this also recurses into symbolic links to directories.

Great, thanks. as for what i needed is that have to have output in form of single string containing all relevant folders once (qmake required filenames instead, that's why %h instead of %p), so for loop+globbing approach wasn't helpful. I don't know why i didn't thought of functions, probably coding in cmd shell and bash shell was affecting perception- even while code used a couple of functions, e.g. for ctrl-C trapping. — Swift, May 04 '17 at 12:57

Escaping double quotes for variables in bash and qmake

2 Answers2