Premade sed string not interpreting string properly

Question

I have code that is meant to find line numbers in a file where 2 different files are equivalent, and then remove all but those line numbers from one of the files. To do so, I made a pre-made sed string I want to run in sed, but I am getting the error:

sed: -e expression #1, char 3: unknown command: `;'

My premade sed string looks like so

echo ${_sedstr}
1;2;7;11;12;13;15;17;22!d

My intention was to then use this variable in a sed command to remove all lines except those in the string within the file ${_edit}

sed -i.bak -e ${_sedstr} ${_edit}

I've tried all combinations of placing single / double quotes around ${_sedstr} and escaping the quotes with a backslash '\', but I still seem to have an issue with the code running as intended.

An example of what I am trying to achieve would be say ${_edit} looked like this:

hello
my
name
is
John
Doe

With the expression sed -i.bak -e '1;2;4!d' ${_edit}, I would expect the file to output to

hello
my
is

Deleting all lines besides 1, 2, and 4.

Your sed syntax is wrong - try sed -e '1;2;7;11;12;13;15;17;22!d' "${_edit}" and you'll see that this fails too. It might be useful for you to explain in your question what it is you're trying to achieve with this command. With an example. — Chris Davies, Dec 02 '19 at 13:24

Stéphane Chazelas · Accepted Answer · 2023-06-23T09:14:26.893

It's not the correct sed syntax.

It should be sed '1d;2d;7d' not sed '1;2;7d'

To delete all but 1, 2, 7, use sed -e 1b -e 2b -e 7b -e d (you can't use ; to separate b commands portably/POSIXly, but you could use newline).

So:

sed_script='1d;2d;7d'
sed -i.back -e "$sed_script" -- "$_edit"

Or:

sed_script='
  1b
  2b
  7b
  d
'
sed -i.back -e "$sed_script" -- "$_edit"

Parameter expansions should be quoted. See When is double-quoting necessary?. And don't forget the -- option delimiter. It's cmd -- "$arg", you don't want cmd "$option_or_arg" here.

For a large number of lines to keep/delete, with GNU awk, you could use:

LINES_TO_KEEP='1;2;7;...' gawk -i /usr/share/awk/inplace.awk -e '
  BEGIN {
    split(ENVIRON["LINES_TO_KEEP"], a, ";")
    for (i in a) keep[a[i]]
  }
  FNR in keep' -E /dev/null "$_edit"

(that one doesn't work for a file called - though; you'd have to use _edit=./- instead of _edit=-).

Do not use -i inplace as gawk tries to load the inplace extension (as inplace or inplace.awk) from the current working directory first, where someone could have planted malware. The path of the inplace extension supplied with gawk may vary with the system, see the output of gawk 'BEGIN{print ENVIRON["AWKPATH"]}'

This comment I believe addressed my misunderstanding the best, but honorable mention bu5hman, because I did not care about the ordering so it was a very simple one-liner. — TheBoi, Dec 03 '19 at 16:57

score 0 · Answer 2 · answered Dec 02 '19 at 19:29

A little off piste from sed but since that has been answered already...if you are not worried about the order, only pairing lines which are the same, you could ...

join <(sort file1) <(sort file2) > matchedlines

Doesn't require a knowledge of the line numbers and doesn't need an array passing into awk.

Premade sed string not interpreting string properly

2 Answers2