24

My question is related to the sed-specific solution given in this answer for this question of reverse grepping. The sed/grep solution that I am unable to decipher is the following one: 

sed '1!G;h;$!d' file

Can someone please decipher this command?

I know from VI(M) knowledge that G denotes the last line of the file and that in sed a bang(!) followed by an address work a bit like grep -v that is to say that it will not match that line. But as a whole the inline sed script above is beyond me.

Rui F Ribeiro
  • 56,709
  • 26
  • 150
  • 232
Geek
  • 6,688
  • 3
    ...very slowly... – mikeserv Oct 01 '15 at 01:06
  • 1
    As mikeserv hints. One should note that this tricky sed recipe is an extremely inefficient way (O(n^2/2) complexity) to reverse lines in a file. It would be prohibitively slow for files with a large number of lines. For a much more efficient line-order reversal alternative see tac from GNU coreutils. – arielf Oct 05 '15 at 21:52
  • and sed -n 'G;h;$p' does the same as noted here – ryenus Jun 03 '22 at 14:31

1 Answers1

37

This reverses the file line by line.

sed '1!G;h;$!d' file

First, sed has a hold space and a pattern space. We have to distinguish between them before concentrating on that specific command.

When sed reads a new line, it is loaded into the pattern space. Therefore, that space is overwritten every time a new line is processed. On the other hand, the hold space is consistent over the whole processing and values can be stored there for later usage.

To the command:

There are 3 commands in this statement: 1!G, h and $!d

  • 1!G means that the G command is executed on every line except the first one (the ! negates the 1). G means to append what is in the hold space into the pattern space.

  • h applies to every line. It copies the pattern space to the hold space (and overwrites it).

  • $!d applies to every line except the last one ($ represents the last line, ! negates it). d is the command to delete the line (pattern space).

  1. Now, when the first line is read, sed executes the h command. The first line is copied into the hold space. Then it is deleted, since it matches the $! condition. sed continues with the second line.
  2. The second line matches the condition 1! (it's not the first line), and so the hold space (which has the first line) is appended to the pattern space (which has the second line). After that, in the pattern space, there is now the second line followed by the first line, delimited by a newline. Now, the h command applies (as in every line); all that is in the pattern space is copied to the hold space. The third statement ($!d) applies: The line is deleted from the pattern space.
  3. Step 2 is now done with all lines. We skip to the last line.
  4. In the last line ($) nearly all of Step 2 is done, but not the delete part (d). sed, when invoked without -n, prints the pattern space automatically at the end of the processing for each input line. So, when not deleted, the pattern space is printed. It contains now all lines in reversed order.
Scott - Слава Україні
  • 10,519
chaos
  • 48,171
  • so the h command on the last line is sort of no-op. Does it clear itself after the end of the script? – Geek Sep 30 '15 at 14:35
  • 1
    @Geek No, the h command copies the pattern space to the hold space, which persist until sed ends. After the end of the script everything is cleared, because the binary exited. – chaos Sep 30 '15 at 14:38
  • 2
    Can we think of the hold space like registers for vim? Are they also numbered? Or there is only one of them? – Geek Sep 30 '15 at 14:41
  • 2
    @Geek In sed there is only one hold space. It's like a variable which can contain something. – chaos Sep 30 '15 at 14:44
  • In step 1, why do we have to delete the line from the pattern space at the end? Doesn't this happen as soon as we get to the next line? – user1717828 Sep 30 '15 at 19:29
  • 1
    @user1717828 If we wouldn't the first line would be printed, when processed. Since sed is not invoked with -n we have to delete every line except the last. At the last line sed appends everything from the hold space to the pattern space. And because the d command will not be executed, the line is printed (this line contains now the whole file reversed). – chaos Sep 30 '15 at 19:54
  • 2
    (1) The OP’s sub-question/observation (in the comment) is, “so the h command on the last line is sort-of a no-op.” (with added emphasis, and slightly paraphrased).  He’s right; when sed is processing the last* line of input, the G reads the hold space (in order to append it to the pattern space), and then the h copies the pattern space to the hold space, which is never referenced again*.  We could just as well say sed 'G;$!h;$!d' or sed 'G;$!{h;d}'.  (2) We could avoid using d by saying sed -n 'G;h;$p'. – Scott - Слава Україні Oct 01 '15 at 07:14