Here is a solution with awk
if you want a customized output format
NR%2==1 {ip=$0; next}
NR%2==0 {a[ip"\n"$0]++}
END {
for(i in a)
printf "%s %d times\n", i, a[i]
}
the script can be executed as
awk -f main.awk file
Explanation
First, we use NR%2==1
to match for odd number lines since odd number modulo 2 equals 1, if any line matches this condition then we save the whole line $0
into a variable called ip
. We can use next
to skip any further processing and go straight to the next iteration.
Second, we use NR%2==0
to match even number lines, if a line matches then we create an index labeled as ip"\n"$0
in an array a
and increment the count value of that specific index. For example, an equivalent expansion would be like
a["144.252.36.69 afrloop=32235330165603"] += 1
I ignored the new line \n
in this example just for simplicity
Finally at END
, after each line has been processed, we use a for
loop to print out the value of each element inside array a
which in our case is the count number for each unique index
Fun Benchmark
Test file generation (1 million records)
awk '
BEGIN{for(i=1;i<10000000;i++)
printf "%d\nafrLoop=%d\n", int(rand()*100), int(rand()*10)}
' > test
$ head test
23
afrLoop=2
84
afrLoop=1
58
@n.caillou paste solution
$ time paste - - < test | sort | uniq -c > /dev/null
real 0m11.250s
user 0m11.352s
sys 0m0.272s
awk solution
$ time awk -f main.awk test > /dev/null
real 0m5.673s
user 0m5.636s
sys 0m0.036s