Bash string eval unclear

Question

Consider the following in a bash script:

STOP_SEARCH_STR="'""for $TASKNAME""'"
echo $STOP_SEARCH_STR               # prints 'for Our Special Task_4'
echo "$STOP_SEARCH_STR"             # prints 'for Our Special Task_4'
echo "grep -F "$STOP_SEARCH_STR" "$TEMP"/draft" # prints grep -F 'for Our Special Task_4' /tmp/draft
echo "$(grep -F "$STOP_SEARCH_STR" "$TEMP"/draft)"  # prints nothing!
echo $("grep -F "$STOP_SEARCH_STR" "$TEMP"/draft")  # prints nothing!

And the following content in the $TEMP/draft text file:

2019-11-21 08:13:58,825 Task started: 'Our Special Task_4' of type 'Our - Special Task' 
2019-11-21 08:14:10,509 Task ended: 'Success' for Our Special Task_4 -- Elapsed time: 11.0 seconds

If I manually type the command grep -F 'for Our Special Task_4' /tmp/draft, I receive the second line in the draft text file:

2019-11-21 08:14:10,509 Task ended: 'Success' for Our Special Task_4 -- Elapsed time: 11.0 seconds

But the the last 2 commands above (inside a bash script) print nothing!

Any idea why?

@Jesse_b I am pretty sure you are right, but at the moment I have no clue how to fix this. Can you elaborate on your comment? — datsb, Nov 23 '19 at 18:34
@datsb Remove the single-quotes from STOP_SEARCH_STR. They are being treated as part of the string to search for. — Gordon Davisson, Nov 23 '19 at 18:36
@GordonDavisson I did that. It makes matters worse: The command fails even when typed manually. It doesn't like the spaces in for Our Special Task_4. — datsb, Nov 23 '19 at 18:50
That last command ought to give you an error! There is no command named literally "grep -F". — Kusalananda, Nov 23 '19 at 19:20
@Kusalananda You are correct. See my comment to the answer below. — datsb, Nov 23 '19 at 20:20

ilkkachu · Accepted Answer · 2019-11-23T20:41:50.340

3

Quotes aren't special after a variable is expanded.

Assuming TASKNAME contains Our Special Task_4, then STOP_SEARCH_STR is set to 'for Our Special Task_4', including those quotes. That's what your echo shows.

When you run grep -F "$STOP_SEARCH_STR" "$TEMP"/draft, you're giving grep the string 'for Our Special Task_4' to look for. That string doesn't exist in the file, so no lines match, and grep prints nothing.

Remove the quotes, you're not looking for them:

task="Our Special Task_4"
str="for $task"
grep -F "$str" "$TEMP/draft"

Note: you do need the double quotes around the variable expansion "$str" or "$STOP_SEARCH_STR" to prevent word-splitting, see Why does my shell script choke on whitespace or other special characters?

When you manually type the command grep -F 'for Our Special Task_4' /tmp/draft, the quotes are part of the shell syntax and effectively remove the special meaning of the spaces within them. This is different from quotes inside a variable which act like ordinary characters.

As an aside: 1) echo "$(somecmd)" is usually redundant, you could just run somecmd directly. 2) Here: echo $("grep -F "$STOP_SEARCH_STR" "$TEMP"/draft"), because of how the quotes surround the whitespace, you're trying to run a command called grep -F 'for. You should probably get an error for that:

$ echo $("grep -F "$STOP_SEARCH_STR" "$TEMP"/draft")
bash: grep -F 'for: command not found

edited Nov 23 '19 at 20:41

answered Nov 23 '19 at 19:09

ilkkachu

138,973

I was too quick to celebrate: I removed the quotes but now the command effectively becomes grep -F for Our Special Task_4 /tmp/draft which finds all lines with the word for in them but complains "No such file or directory" for the rest of the words in the search string (Our, Special, Task_4). How do I add quotes around the "string with spaces to be searched for" so that work as if the command is typed manually? – datsb Nov 23 '19 at 20:19
@datsb, what's the exact command you used? – ilkkachu Nov 23 '19 at 20:20
The exact command is just as I originally posted, except that I removed the quotes. It's basically the same as grep -F "$str" "$TEMP/draft" that you suggested. I am now totally confused, although you explanation is excellent. Note that the challenge here is the spaces in the search string. – datsb Nov 23 '19 at 20:28
@datsb, yes, I know. As long as you have the double-quotes around the variable expansion, it should work fine. They're the quotes that are necessary. (I should have been explicit about that from the start.) – ilkkachu Nov 23 '19 at 20:41
@datsb "Always double quote any expansion" is a simple rule of thumb that you could follow, and which will be correct more often that not. – Kusalananda Nov 23 '19 at 20:45
see also: https://mywiki.wooledge.org/WordSplitting – ilkkachu Nov 23 '19 at 20:45

score 1 · Answer 2 · answered Nov 23 '19 at 23:39

You should change this line:

STOP_SEARCH_STR="'""for $TASKNAME""'"

to this line (yes, no single quotes needed, they are not useful):

STOP_SEARCH_STR="for $TASKNAME"

And use this line in the script (yes, quoted as shown):

grep -F "$STOP_SEARCH_STR" "$TEMP"/draft

If you want/need to add an echo (which is superfluos and may even cause problems) use this line (yes, quote all the expansions (variables and subshell)):

echo "$(grep -F "$STOP_SEARCH_STR" "$TEMP"/draft")"

Thanks. @ilkkachu already helped me resolve my problem. But your answer further clarifies the issue and may help others in the future. — datsb, Nov 24 '19 at 08:17

Bash string eval unclear

2 Answers2