0

I've read How do I delete the first n lines of an ascii file using shell commands?, it is helpful. However I've a file something as below (please consider 2 columns as 2 different files):

1 4
1 4
1 4
1 4
1 4
3 5
3 5
3 5
3 5
3 5
7 5
7 5
7 5
7 5
7 5

I need to delete line 2-5 then 7-10 and so on (Valid outputs are 1, 3, 7 and 4, 5, 5).

I know number of lines for which pattern (number) repeats. However I don't know if everytime it would follow same pattern for e.g. here its 1, 3, 7 next file can have 4, 6, 1 or 4, 5, 5 so I need to make it number of lines based than grepping the values.

Can anyone give me a pointer how would I delete lines repetitively?

Community
  • 1
wisemonkey
  • 103

3 Answers3

3

It sounds like you're looking for uniq.

Or, from the comments:

sed '2,5d;7,10d;12,$d'
Keith Thompson
  • 22,340
Samuel Edwin Ward
  • 864
  • Thanks however uniq fails in case of 4, 5, 5 (all repeated 5 times each). This is why I was looking for number of lines based solution – wisemonkey Jan 18 '13 at 21:25
  • @wisemonkey, I guess I don't properly understand the question. Maybe you should add more examples or try describing it another way. – Samuel Edwin Ward Jan 18 '13 at 21:26
  • My bad, please have a look at modified question. Hopefully second sequence makes it easier (for now I'm going to use uniq and then add extra entry if required) – wisemonkey Jan 18 '13 at 21:34
  • 1
    @wisemonkey, is sed '2,5d;7,10d;12,$d' what you're looking for? – Samuel Edwin Ward Jan 18 '13 at 21:46
  • exactly but I didn't know I can cascade them like that, awesome thanks a lot :D – wisemonkey Jan 18 '13 at 21:48
  • Note that you can use sed to look for the lines if you kwon the pattern, or even use something like grep pattern file to select just what you are looking for. More flexible tools in that line include Perl and Python (some programming required for funkier stuff). – vonbrand Jan 21 '13 at 18:02
  • @vonbrand: I generally use Perl but I'm not looking for any pattern, its basically oversampled output of my design block I want to remove unnecessary samples. – wisemonkey Jan 21 '13 at 19:17
  • I wanted to know if I can make sed anymore generic? for e.g. I don't know length of file, and I want to change number of samples to be deleted from 4 to 7 (I know I probably need to read a tutorial about sed however I want to know if I can do it and then link to tutorial if possible) – wisemonkey Jan 21 '13 at 19:19
  • For a more generic sed expression using branching try sed -n ':a;N;N;N;N;P;d;ba' file. See http://www.catonmat.net/blog/sed-one-liners-explained-part-two/ for example – steeldriver May 15 '14 at 12:16
0

If what you are really asking is output every nth line of a file:

cat file | perl -ne '$. % 5 or print'

Possibly:

cat file | perl -ne '($.-1) % 5 or print'
Ole Tange
  • 35,514
-1

use following command 

cat filename | sort | uniq > filename2
newbie17
  • 101
  • How does sort and uniq help in this case? Your solution would fail on the given example itself. Since the output is expected to be 4,5,5, however, uniq would eliminate the second 5. – darnir May 15 '14 at 11:07
  • @darnir Code tested on the given file and the output is as expected i.e. 1 4 3 5 7 5 – newbie17 May 19 '14 at 08:21