delete a line if same line exists in the previous line

Question

I want to remove lines which exist already in the previous line from the command line in UNIX. I have the following data in a file.

 <xref id="gi_525506931_ref_NP_001266519.1__brain_aromatase"/>
                    <entry ac="IPR002401" desc="Cytochrome P450, E-class, group I" name="Cyt_P450_E_grp-I" type="FAMILY">
                    <entry ac="IPR001128" desc="Cytochrome P450" name="Cyt_P450" type="FAMILY">
                    <entry ac="IPR001128" desc="Cytochrome P450" name="Cyt_P450" type="FAMILY">
                    <entry ac="IPR001128" desc="Cytochrome P450" name="Cyt_P450" type="FAMILY">

So, it should look like this:

 <xref id="gi_525506931_ref_NP_001266519.1__brain_aromatase"/>
                    <entry ac="IPR002401" desc="Cytochrome P450, E-class, group I" name="Cyt_P450_E_grp-I" type="FAMILY">
                    <entry ac="IPR001128" desc="Cytochrome P450" name="Cyt_P450" type="FAMILY">

Welcome to U&L. We are (often) a pretty forgiving bunch, if you make mistakes we'll tell you or correct them. You can review the changes that I made to your post, just click on the link above my (or any follow up editor's) avatar. — Anthon, Nov 05 '14 at 05:12
Hi Anthon, I am not able to accept two answers although cuonglm's answer also gave me required result. Also, I am not able to upvote the answers. — Prakki Rama, Nov 05 '14 at 06:57
Upvoting comes with 15 reputations, wait for it ..., there you go, should have it — Anthon, Nov 05 '14 at 06:58

score 9 · Accepted Answer · edited Apr 13 '17 at 12:36

9

If you can guarantee that the identical lines will be consecutive, you can use

uniq your_file

and cuonglm's answer will work even if they're not.

edited Apr 13 '17 at 12:36

Community

1

answered Nov 05 '14 at 05:08

Joseph R.

39,549

Yes. Thats what I wanted. – Prakki Rama Nov 05 '14 at 05:10
1

A line is normally consecutive with the previous line (as in the title of the question) ;-) – Anthon Nov 05 '14 at 05:15
@Anthon You're right. But, hey, completeness never hurts :) (Incompleteness, on the other hand,...) – Joseph R. Nov 05 '14 at 05:21

score 4 · Answer 2 · edited Apr 13 '17 at 12:36

4

With awk:

awk '!a[$0]++' file

An explanation for this idiom you can read here.

edited Apr 13 '17 at 12:36

Community

1

answered Nov 05 '14 at 05:09

cuonglm

153,898

2

This also removes doubles that are the same with entries that are not the previous line, the title states "same as the previous line" – Anthon Nov 05 '14 at 05:17
2

...cool! Maybe adding a bit of explanation? ;-) – Rmano Nov 05 '14 at 10:03
I second that emotion, @Rmano. Currently indistinguishable from magic :p – Svish Nov 05 '14 at 13:37
@Rmano and Svish: I add link to explanation. – cuonglm Nov 05 '14 at 15:22

delete a line if same line exists in the previous line

2 Answers2