2

I would like to do the following

  1. Search file content with pattern (^[ \t]*services:[ \t]*)
  2. If found, replace the pattern by a String (in variable)
  3. Else, append the String (in variable) at end of file

I got stuck on step 2 and step 2 of the script will replace services: in file to ${serviceLoopBack}. What is the problem to my sed command?

Test script

#!/bin/bash
set -e

destfile=config.yml


serviceLoopBack="services:|NL|- name: loopback-service|NL||FS||FS|url: http://127.0.0.1:12345|NL||FS||FS|routes:|NL||FS||FS|- name: loopback-test-route"
serviceLoopBackWithFrontSpace="|NL||NL|$serviceLoopBack"


if grep -E "^[ \t]services:[ \t]" $destfile
then
    # service tag found, replace target string
    sed -i -e 's/^[ \t]services:[ \t]/${serviceLoopBack}/g' $destfile
else
    # service tag not found, append at end of file
    echo $serviceLoopBackWithFrontSpace >> $destfile
fi


replace separator 1 to new line


sed  -i -e 's/|NL|/\n/g' $destfile


replace separator 2 to space


sed  -i -e 's/|FS|/ /g' $destfile
AdminBee
  • 22,803
hk6279
  • 123
  • 1
  • 4

2 Answers2

3

The problem is that in your second sed call, you have enclosed the program in single quotes (' ... '). While this is recommended practice, single quotes prevent the shell from expanding shell variables such as ${serviceLoopBack}, so that text will remain verbatim instead of being replaced by the shell variable's content.

One possibility is to use double-quotes instead of single quotes, as in

sed -i -e "s,^[ \t]*services:[ \t]*,${serviceLoopBack}," "$destfile"

Alternatively, assemble the sed expression in a shell variable using printf:

printf -v prog 's,^[ \t]*services:[ \t]*,%s,' "$serviceLoopBack"
sed -i -e "$prog" "$destfile"

Note that I am using , as separator instead of /, since your replacement text contains / also and this would otherwise confuse sed. Also, I omitted the g option since I assume that there will be only one match per line.

You may want to look at the quoting in your shell script anyway, see e.g. this answer on the reasons.

However, if sed is not a strict requirement, almost the entire shell script can be replaced with an awk program instead (which is faster):

awk -v slb="services:|NL|- name: loopback-service|NL||FS||FS|url: http://127.0.0.1:12345|NL||FS||FS|routes:|NL||FS||FS|- name: loopback-test-route" \
 '/^[ \t]*services:/{$0=slb;f=1}
  {gsub(/\|NL\|/,"\n"); gsub(/\|FS\|/," ")} 1; END{if (!f) {print "\n\n" slb}}' "$destfile"
  • This will import the "service loop back" specification in the awk variable slb.
  • It will then check if any line matches the services: pattern, and if so, replace the entire line with the string stored in slb (which I infer is what you actually want to achieve). At the same time, set a flag f to 1.
  • Using gsub(), replace all occurences of |NL| with a new-line, and all occurences of |FS| with a space.
  • The seemingly "stray" 1 instructs awk to print the current line, including all modifications made so far.
  • At the end, if f is still unset, append the "service loop back" line with the leading pattern (however, already converted to \n\n).

Note that awk will not edit the file in-place, unless you have an awk implementation that understands the -i inplace option. You may have to redirect the output to a temporary file and then replace the $destfile with the temporary file afterwards.

All in all, however, since you are using a structured language (YAML), it might be worthwile using a dedicated parser such as yq instead of line-oriented text processing tools.

AdminBee
  • 22,803
2

The problem with your sed command is that variables are not expanded inside single quotes, so you need to enclose the sed script in double quotes instead. To illustrate:

$ var="foo"
$ echo bar | sed 's/bar/$var/'
$var

$ echo bar | sed "s/bar/$var/"
foo

Your next problem will be that your replacement string contains / so you need to use a different delimiter. Something like:

sed -i -e "s#^[ \t]*services:[ \t]*#${serviceLoopBack}#g" "$destfile"

And is there any point to the g? Do you expect more than one match per line?

Here's what should be a working version of your script with a few improvements (quoting, using grep -qm1 to remove output and stop at the first match):

#!/bin/bash
set -e

destfile=config.yml


serviceLoopBack="services:|NL|- name: loopback-service|NL||FS||FS|url: http://127.0.0.1:12345|NL||FS||FS|routes:|NL||FS||FS|- name: loopback-test-route"
serviceLoopBackWithFrontSpace="|NL||NL|$serviceLoopBack"


if grep -Eqm 1 "^[ \t]services:[ \t]" "$destfile"
then
    # service tag found, replace target string
    sed -i -e "s#^[ \t]services:[ \t]#${serviceLoopBack}#g" "$destfile"
else
    # service tag not found, append at end of file
    printf '%s\n' "$serviceLoopBackWithFrontSpace" >> "$destfile"
fi


replace separator 1 with newline and separator 2 with space


sed -i -e 's/|NL|/\n/g' -e 's/|FS|/ /g'  "$destfile"
terdon
  • 242,166