7

I started using sed to replace text in files, to make it easier to mass change the contents of files, and I have each value that needs to be set as a variable with BASH and the replacement as a function that calls all of the variables to replace the existing text. Example~

sed -i -e "s/$answer/$FORME $answer/g" ~$answer.txt

$answer is the variable input with the command in terminal. $FORME is a variable that is defined depending on the input of $answer, however $FORME contains "/" and I have been browsing online and can't find a way to replace using variables that contain special characters. When I try to use the command on $answer's that create $FORME with a special character displays this error:

sed: -e expression #1, char 15: unknown option to `s'

I'm assuming that means that the special character in $FORME is registered as part of the actual command, not a string to replace. Please help!

Thanks in advance!

Brandon K
  • 71

2 Answers2

9

You can use any character as a s command delimiter, so do, for example,

sed -i -e "s|$answer|$FORME $answer|g" ~$answer.txt
enzotib
  • 51,661
6

Stunningly, this is asked and answered all over Stack Exchange, but the two generally useful answers to this very reasonable question are needles in the haystack. They are:

1. Escape just 4 possible special characters in your variable

As pointed out here (despite the question falling victim to the SE aversion to general questions), there are just 4 characters considered special in the replacement part of the sed s command: \, &, newline and the delimiter (usually / but can actually be anything).

So to use an arbitrary variable as a replacement in sed, escape any occurrences of those four characters first, like so:

REPL='Arbitrary variable potentially with special characters like / and & and \ and 
newline\
newline2'
REPL=$(sed -e 's/[&\\/]/\\&/g; s/$/\\/' -e '$s/\\$//' <<<"$REPL")
hexdump -C <<< "$REPL"

For example:

REPL='/&\ 
newline'
REPL=$(sed -e 's/[&\\/]/\\&/g; s/$/\\/' -e '$s/\\$//' <<<"$REPL")
sed "s/x/$REPL/" <<< "STARTxEND"

produces:

START/&\ 
newlineEND

as expected.

2. Use an equivalent tool that interprets the replacement variable verbatim

perl (ref) has this capability because variables (provided they are part of the environment by exporting them or setting them in the invocation line) are first class citizens. Note \Q is not needed in the replacement string (but is useful in the regexp string).

export REPL='/&\ 
newline'
perl -pe 's/x/$ENV{REPL}/' <<< "STARTxEND"

produces:

START/&\ 
newlineEND

awk, despite ref, does not.

Heath Raftery
  • 1,402