Using sed command arguments to be compatible with both GNU & BSD Unix (in-place editing)

Question

I have a shell script that I currently use for some build related stuff for a mobile application.

Due to the the subtle differences between BSD & GNU one of build scripts originally written on a Mac (BSD)

environment=$1

if [[ -z $environment ]]; then 
  environment="beta"
fi
if ! [[ $environment =~ (live|beta) ]]; then
  echo "Invalid environment: $environment"
  exit 1
fi

mobile_app_api_url="https://api"$environment".mysite.com"

cp app/index.html.mob MobileApp/www/index.html

sed -i'' "s#MOBILE_APP_API_URL#\"$mobile_app_api_url\"#g" MobileApp/www/index.html

The sed command has been written on BSD (Mac) but as builds may take place on both Mac or Ubuntu (GNU) I need to modify this to work with on both flavours, what is the best approach for this?

Use ed, it can edit the files in-place and it's fully portable. — don_crissti, Mar 03 '17 at 14:12
Use perl -pi -e '...'. Both BSD and GNU sed derived their -i from perl's one. perl will be available on most GNU and BSD systems and most other systems. — Stéphane Chazelas, Mar 03 '17 at 15:09
-i'' is actually -i (-i concatenated with the empty string) so would only work with GNU sed. You'd need -i '' for BSDs. — Stéphane Chazelas, Mar 03 '17 at 15:10
@StéphaneChazelas There are Unices out there without Perl in the base system. NetBSD is one (but it may well be alone in that group). — Kusalananda, Mar 03 '17 at 15:21
@Kusalananda, true. In my experience, one is more likely to find perl than bash though. For instance here, many of the BSDs would not have bash or any shell that supports those [[...]] or =~ (which can easily be replaced by a standard and more legible sh case construct). — Stéphane Chazelas, Mar 03 '17 at 15:43

Kusalananda · Accepted Answer · 2017-03-03T16:00:11.487

2

Do this to circumvent the problematic portability issues with the -i flag of sed:

sed 'sed-editing-commands' thefile >tmpfile && mv tmpfile thefile

I.e., write to a temporary file, and then replace the input file with the temporary file if the sed command didn't fail.

This is portable to all implementations of sed that I know of.

To create a temporary filename safely, use mktemp. Although this isn't a standard utility, it is available on all Unices that I have access to (OpenBSD, NetBSD, Solaris, macOS, Linux):

tmpfile=$(mktemp)
sed 'sed-editing-commands' thefile >"$tmpfile" && mv "$tmpfile" thefile

edited Mar 03 '17 at 16:00

answered Mar 03 '17 at 14:09

Kusalananda

333,661

However, that doesn't make any attempt to preserve the ownership, permissions and other metadata of the original file like GNU or BSD seds would. – Stéphane Chazelas Mar 03 '17 at 15:17
@StéphaneChazelas Good point. To do that, one would have to cp thefile "$tmpfile" first, I think. Does that sound about right? – Kusalananda Mar 03 '17 at 15:19
Some good related reading in Scott's answer here: http://unix.stackexchange.com/a/186126/117549 – Jeff Schaller Mar 03 '17 at 15:28

score 0 · Answer 2 · answered Mar 03 '17 at 14:32

0

You need to realize that the $mobile_app_api_url should not contain the chars that are meaningful to sed on the RHS of s///. Specifically, speaking, you need to make sure that this variable has the following properly escaped away: viz., & # \ newline before they are plugged in.

answered Mar 03 '17 at 14:32

It looks as if that is already taken care of. The URL will not cause problems. The issue is with the non-portable -i flag for sed (it work very differently with different implementations of sed). – Kusalananda Mar 03 '17 at 14:44
From the OP's code shown there is nowhere any escaping the characters that I have mentioned above in my post. Just try placing an & in $1 to see what I mean. – Mar 03 '17 at 14:50
1

The URL in $mobile_app_api_url will be either https://apibeta.mysite.com or https://apilive.mysite.com, no other values will be allowed. The escaping of characters in the substitution is furthermore an issue in all implementations of sed on all Unices. This question is about portability. – Kusalananda Mar 03 '17 at 14:55
1

@Kusalananda, any value of $environment that contains either live or beta would be allowed (though it doesn't look like it was the intention). – Stéphane Chazelas Mar 03 '17 at 15:53
@StéphaneChazelas Right you are. Any URL with & # \ newline in the domain part would be a bit broken anyway though. – Kusalananda Mar 03 '17 at 15:56

Stéphane Chazelas · Answer 3 · 2017-03-03T15:52:28.820

Instead of:

cp app/index.html.mob MobileApp/www/index.html
sed -i'' "s#MOBILE_APP_API_URL#\"$mobile_app_api_url\"#g" MobileApp/www/index.html

Just do:

sed "s#MOBILE_APP_API_URL#\"$mobile_app_api_url\"#g" \
  < app/index.html.mob >  MobileApp/www/index.html

You can also remove your dependency on ksh93/bash by changing the top part to:

environment=$1

case $environment in
  "") environment=beta;;
  live|beta) ;;
  *)
    printf >&2 'Invalid environment: "%s"\n' "$environment"
    exit 1;;
esac

Using sed command arguments to be compatible with both GNU & BSD Unix (in-place editing)

3 Answers3