1

I need to remove first N lines matching regex but keeping the last one.

28test
32test
something else
6test
something else
something else
4test
entirelysomethingelse

I'd want something like this.

something else
something else
something else
4test
entirelysomethingelse

Tried using sed, it seems like it only works with one line and multiple strings.

I used the regex: ^(.*test)$

AdminBee
  • 22,803
UnixNoob
  • 11

2 Answers2

3

A simple way to only keep the last matching line is to print the input in reverse, select the first matching line only and then print the output in reverse, in a pipeline. Assuming tac from GNU Coreutils is available to you:

tac input_file | awk '!/test$/ || !seen++' | tac >output_file

In-place editing (as you requested in a comment) is generally obtained by wrapping commands in a script or a function that takes care of overwriting the file(s) given as argument(s) with the processed output.

tmpdir=$(mktemp -d)
cp input_file "$tmpdir/file"
tac "$tmpdir/file" | awk '!/test$/ || !seen++' | tac >input_file
rm -r "$tmpdir"

If your shell supports the pipefail option (I could successfully test bash, ksh93, mksh, zsh using setopt PIPE_FAIL, busybox ash, yash), this can be made safer with set -e and set -o pipefail: errors (including those occurring anywhere in the pipeline) will make the execution stop before the temporary file is removed.

On platforms that support it, assuming you don't care losing your data in case something goes wrong, you may also use:

{ rm file; tac | awk '!/test$/ || !seen++' | tac >file; } <file

Note that this will change the inode of file (as would do the in-place editing option provided by many common tools).

If, instead, you want to remove the first n matching lines, assuming here n = 2:

awk '!/test$/ || ++seen > 2' input_file >output_file

In this case, in-place editing is conveniently allowed by GNU awk's ability to import additional libraries (and, specifically, the "inplace" one shipped with gawk itself):

awk -i inplace '...' file

More on this can be found in this other answer on U&L.

fra-san
  • 10,205
  • 2
  • 22
  • 43
  • it seems like it works but how would I save the file? (tac test.log | awk '!/^.*test$/ || !seen++' | tac) > test.log doesn't work – UnixNoob Oct 13 '20 at 09:22
  • @UnixNoob Edited. In general, the processed output is saved to a new file (>, (standard output redirection) is a common way to do that) and later used to overwrite the input file where appropriate. Even tools that edit files in place (e.g. GNU sed with the -i option) often do this under the hood. – fra-san Oct 13 '20 at 09:30
  • @fra-san how would I go on about making it in-place modifications? I don't want to use a temp file. – UnixNoob Oct 13 '20 at 09:31
  • @UnixNoob Please, add that as a requirement in your question to make it visible to other users - that may provide alternative answers. Especially considering that the usually given solutions don't assume in-place editing. – fra-san Oct 13 '20 at 10:10
0

As for the option - "Deleting the first N lines of pattern matches"
On the sed, you can organize a counter in hold space (additional buffer).

sed -r '/test$/!b;x;s/$/-/;/-{4}/!{x;d};x' file

/test$/!b - Unconditional jump to the end if there is no pattern matching.
x - Exchange the contents of the pattern and hold spaces.
s/$/-/ - For each matching add one character to the end of the counter, in my case a hyphen.
/-{4}/!{x;d} - If the counter contains less than 4 characters, then delete the line in patten space.

nezabudka
  • 2,428
  • 6
  • 15