1

In bash, given these files:

$ cat noquotes.txt
s/a/b/g
$ cat quotes.txt
"s/a/b/g"

Why does

$ echo "aaa" | sed -e $(cat noquotes.txt)
bbb

succeed, but

$ echo "aaa" | sed -e $(cat quotes.txt)
sed: 1: ""s/a/b/g"
": invalid command code "

fails?

If if run set -x first, then I see that

$ echo "aaa" | sed -e $(cat noquotes.txt)
+ echo aaa
++ cat noquotes.txt
+ sed -e s/a/b/g
bbb

and

$ echo "aaa" | sed -e $(cat quotes.txt)
+ echo aaa
++ cat quotes.txt
+ sed -e '"s/a/b/g"'
sed: 1: ""s/a/b/g"
": invalid command code "

So it seems like there are extra quotes being inserted around the contents of noquotes.txt, but not around quotes.txt.

mark
  • 165
  • 1
  • 6
  • @don_crissti let me add another input file, spaces.txt with content -e "s/a /b/g". Now when I try echo "a a" | sed $(cat spaces.txt) I get a similar error. Would spaces.txt be better written as -e s/a /b/g? This seems contrary to the command-line usage of sed where I need to add quotes around the sed command like this: echo "a a" | sed -e "s/a /b/g" – mark May 03 '16 at 21:11
  • @don_crissti thanks that's helpful. One more example that would help clarify: I can run echo "a b " | sed -e "s/a /b/g" -e "s/b /c/g" => bc. What should the contents of multi.txt be if I want to write echo "a b " | sed "$(cat multi.txt)" and similarly produce output bc? – mark May 03 '16 at 21:34
  • @don_crissti Interesting, I didn't know about ; in sed. You're right, this is an XY question. In reality I want to pass multiple -x module arguments to the browserify command, where each module is specified in a newline-separated file, and each module is contained in double-quotes within the file. The modules may be file paths – mark May 03 '16 at 22:22
  • @don_crissti I'll give it a shot http://stackoverflow.com/questions/37016419/pass-multiple-externals-to-browserify-command-line-from-file . In general I'm interested in learning about bash, so I wanted to inquire about a general-purpose way of passing command line arguments from a file into a bash command using command substitution. I chose sed as an example hoping it would be familiar to the unix.stackexchange community – mark May 04 '16 at 00:23
  • Well, I'm not familiar with that browserify thingie... It looks like you're trying to build up a command line based on the contents of some files so Etan's suggestion there isn't bad though I'm not sure why you have posted the question there - stackoverflow is a black hole - questions are forgotten in the next 3 minutes. Wait a couple of days and if you don't get any reply delete it from there and re-post it here on Unix&Linux (don't cross-post as it will be closed). – don_crissti May 04 '16 at 10:55

2 Answers2

3

There is a step in the processing of the command line that is called the quote removal step. This is usually the last thing that happens before the command is executed and it removes the outer sets of quotes that you used to quote strings in the command.

For a command such as

sed -e "s/a/b/g"

this step make sure that the sed command is given the string s/a/b/g as its last argument, and not "s/a/b/g".

When you are reading the string "s/a/b/g" from a file using cat in a command substitution, the quotes will not be removed, as they are not part of the original command (they are part of the data read from a file).

This means that sed will receive the literal string "s/a/b/g" as the expression to run, and it therefore complains that it does not understand the initial " (it was expecting a sed command).

As for the output from set -x tracing, treat it as nothing more than debugging output. The bash shell will add quotes around strings that contain certain characters in the trace, so it's not indicative of what the quoting of any single argument actually was or how it was interpreted by the invoked utility.

As an aside, to run sed with an editing script on file, use the -f option:

sed -f noquotes.txt <<<"aaa"

Also, the y command in sed could be used to transliterate between one set of single characters to another more efficiently than a substitution with s would. Your s/a/b/g substitution is better written as y/a/b/.

If this is the only operation that you need to perform, then it may be even more efficient to use the simpler tr utility:

tr a b <<<"aaa"
Kusalananda
  • 333,661
2

So it seems like there are extra quotes being inserted around the contents of noquotes.txt, but not around quotes.txt.

This is just Bash being helpful. You'll notice it will put quotes around things whenever there's a symbol that would otherwise be interpreted by Bash. It acts just like you would, using quotes only when necessary. Try to put any of the following into quotes.txt:

s/a$/a/g
s/"hi"/"bye"/g

The list goes on and on.

Now back to your original question. First of all, your sed prints out a weird message. From set -x it's obvious that the input is "s/a/b/g", not ""s/a/b/g". I wouldn't trust this message. Here's what GNU sed 4.2.1 prints out for "s/a/b/g":

$ sed -e '"s/a/b/g"'
sed: -e expression #1, char 1: unknown command: `"'

Pretty clear: since the first char is a command name and there's no command named ", sed has no idea what to do.

Alexander Batischev
  • 2,393
  • It's just a matter of formatting the error message. Just like for non-gnu users unknown command: \"'` may also be a weird message that they wouldn't trust: a backtick followed by a double-quote and single quote ? – don_crissti May 03 '16 at 20:52
  • At least with GNU the quotes are balanced. Here, there's a quote at the beginning but not at the end. – Alexander Batischev May 04 '16 at 09:25