Extract all line from a file that contains two strings in any position

Question

I have a .txt file with contents similar to this:

100 150 180 200 300 400
100 200 250 350 380 400
100 160 170 400 450 500
100 120 140 160 180 200
100 120 140 160 180 300

I want to grab all the lines, starting from a specific column (like 2, 3 or any other) that contains '100' and '200' in any postion and then output it to another separate txt file. How can I do that? In the example above, the correct print should be:

100 150 180 200 300 400
100 200 250 350 380 400
100 120 140 160 180 200

I have tried using sublime's "Find All" feature and then use the right arrow to the end of the line to highlight them, but unfortunately some lines are much longer than the others so it doesn't work.

None of your sample lines has 100 and 200 in fields from 3 on, so ir's impossible to get at your desired output. — RudiC, Dec 07 '18 at 21:30

Kusalananda · Answer 1 · 2018-12-07T18:11:17.840

2

$ grep 100 <file | grep 200 >newfile
$ cat newfile
100 150 180 200 300 400
100 200 250 350 380 400
100 120 140 160 180 200

The first grep extracts all lines from the original file that contains the string 100. The second grep extracts all lines from that result that contains the string 200.

Note that this would also extract lines that contained strings like 1100 and 1200 since these contains the wanted strings as substrings. To avoid that, use grep with its -w option (if available).

To test only from a specific column onwards, use a short awk program:

$ awk -v col=1 '{ delete c; for (i=col; i<=NF; ++i) ++c[$i]; if (c[100] > 0 && c[200] > 0) print }' <file >newfile
$ cat newfile
100 150 180 200 300 400
100 200 250 350 380 400
100 120 140 160 180 200

This awk program takes a the value of the col variable from the command line (here the value is 1). It then goes through each input line from column col onwards, counting how many times each value occurs. If the values 100 and 200 occurs more than zero times, the line is printed.

The program, with nicer layout:

{
    delete c

    for (i=col; i<=NF; ++i)
        ++c[$i]

    if (c[100] > 0 && c[200] > 0)
        print
}

This program also lends itself to extracting lines with specific number of matches of certain items.

edited Dec 07 '18 at 18:11

answered Dec 07 '18 at 16:22

Kusalananda

333,661

Thanks, @Kusalananda. Is there a way to start the search from an specific column? – Hector Dec 07 '18 at 17:55
@Hector Please update the question to make it complete. I'm avoiding answering followup questions in comments, and if you keep changing the question, it runs the risk of being closed as "unclear". – Kusalananda Dec 07 '18 at 17:57
It's done. Sorry for that, Kusalananda. – Hector Dec 07 '18 at 18:04
@Hector See updated answer. – Kusalananda Dec 07 '18 at 18:14
Thanks so much, @Kusalananda! Your answer is absolutely complete! – Hector Dec 07 '18 at 18:20
Hey, @Jusalananda. I was trying your code for number with 3 or more digits and the command couldn't find them properly. Is there anything I have to change? – Hector Dec 08 '18 at 12:04
@Hector What do you mean by "three or more digits"? – Kusalananda Dec 08 '18 at 12:08
Ops, I mean less than 3 digits, like 2 3 4 or 02 10 and 27. – Hector Dec 08 '18 at 12:14
@Hector I still don't know what you are trying to do with numbers that have less than three digits. – Kusalananda Dec 08 '18 at 12:15
Something like
- 13 15 17 40 53 57
- 05 17 33 39 42 49
- 05 14 33 36 43 44
- 17 19 28 45 48 56
- 07 14 15 29 38 50
– Hector Dec 08 '18 at 12:18
Now it makes sense, @Kusalananda – Hector Dec 08 '18 at 13:21
@Hector You could just adjust the numbers 100 and 200 in the code. I'm not sure what you want to de. – Kusalananda Dec 08 '18 at 13:31
@Kusalanda, yes, I've tried it but it doesn't work. The only way (very nasty way btw) was it: awk -f ' ' '{ if((($3 == 02) || ($4 == 02) || ($5 == 02) || ($6 == 02) || ($7 == 02) || ($8 == 02)) && (($3 == 10) || ($4 == 10) || ($5 == 10) || ($6 == 10) || ($7 == 10) || ($8 == 10))) { print } }' file – Hector Dec 08 '18 at 13:40
@Hector If you want to make a difference between 2 and 02 then use "02" in the test: c["02"] > 0. – Kusalananda Dec 08 '18 at 13:50
Amazing, that worked exactly how I need! Thank you so much! – Hector Dec 08 '18 at 23:19

score 0 · Answer 2 · answered Dec 07 '18 at 21:32

0

If you're sure that there won't be false positives, try also

awk '{TMP = $0; sub ($1 FS $2, "")} /100/ && /200/ {print TMP} ' file

answered Dec 07 '18 at 21:32

RudiC

8,969

Extract all line from a file that contains two strings in any position

2 Answers2