exec terminate command by the space in single quotes

Question

I have a simple bash script goes like

#!/bin/bash
set -e
SPARK_CONF="--master ... \
...
--conf spark.driver.extraJavaOptions='-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps' \
--conf ...."
SPARK_CMD="spark-submit $SPARK_CONF"
echo $SPARK_CMD
exec $SPARK_CMD

I'll get an error

Error: Unrecognized option: -XX:+PrintGCDetails

I can see that I print the whole command before exec it, so I copied the printed command, and paste in my shell then I found it can be executed successfully.

I tried to replace the single quote with \" but still the same error.

Also, if I replace exec with eval, it will also work. I googled around the difference between them, but didn't found why exec will terminate inside the quotes...

I just want to understand why exec fail in this case.

Update: According to https://unix.stackexchange.com/a/444949/524235, I changed my related part to

set -e -x
SPARK_CONF="--master ... 

...
--conf spark.driver.extraJavaOptions='-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps' 

--conf ...."
SPARK_CMD=($SPARK_HOME/bin/spark-submit $SPARK_CONF)
"${SPARK_CMD[@]}"

It's still failing with the same error message mentioned above, but I can see the executed command becomes:

.../bin/spark-submit ...
--conf 'spark.driver.extraJavaOptions='\''-verbose:gc' -XX:+PrintGCDetails '-XX:+PrintGCDateStamps'\'''
...

I can see there're some extra single quotes and backslash added into the command, how can I progress now?

This may help: How can we run a command stored in a variable? — steeldriver, May 01 '22 at 02:23
@steeldriver Thanks a lot for the useful link! I am trying with the solution menioned there, but I am still failing with the same error, I've updated my description, could you take another look? Respect, man. — Alain ux, May 01 '22 at 02:54
The shell parses quotes and escapes before expanding variables, so putting quotes in a variable doesn't do anything useful; if you need to store a command or argument list, use an array instead. This applies to SPARK_CONF as well as SPARK_CMD. Or better yet, don't store them at all; just execute the command directly. (Or maybe use a function, depending on the situation.) See BashFAQ #50: "I'm trying to put a command in a variable, but the complex cases always fail!" — Gordon Davisson, May 01 '22 at 05:45
"Update: According to [this], I changed my related part to..." The parameters should go into the array. You've still got them as a string — Chris Davies, May 01 '22 at 08:35

exec terminate command by the space in single quotes

0 Answers0