How to compare a list of hashes (line by line) against another list of hashes?

Question

(Using Bash) I am trying to compare a list of hashes against another list of hashes; am thinking a nested for loop or a while loop for each line and then an if statement. Compare the first line to the contents/each line of the second file and so on and so forth.

for a in 'cat file1.sh'   
    do
    echo $a
           for b in 'cat file2.sh'
               do
               echo $b
                   if [ "$a" == "$b" ]
                   then
                   echo $a $b
                   fi


done
done

I realize I am missing a key comparison here. My attempt is to comparing the first line of the first file to each line/hash of the 2nd file; and then once it finds a match, append it to a new file and go to the second line of the first file and repeat the process until all the match comparisons have been attempted and the matching results are appended to a new file.

Does it have to be done with bash alone? You could do this much more easily with other tools, such as grep or comm. See https://unix.stackexchange.com/questions/155309/outputting-common-lines-from-2-files-and-uncommon-lines-from-both-the-files-in-o — JigglyNaga, Jan 26 '17 at 09:43

score 5 · Accepted Answer · edited Apr 13 '17 at 12:37

$ join <( sort hashes1 ) <( sort hashes2 )

This will return all lines in hashes1 and hashes2 that are identical.

To get the ones that are not identical:

$ join -v 1 <( sort hashes1 ) <( sort hashes2 )

$ join -v 2 <( sort hashes1 ) <( sort hashes2 )

The first command will display all hashes from the first file that are not in the second, while the second command will do the opposite.

For further information, see the manual for join.

Your script suffers from three main issues:

'cat file1.txt' is just a text string. If you wanted the contents of file1.txt you would use $( <file1.txt ).
It's not a good idea to slurp the complete data of a file and then loop over it with for. For bigger files, this is just a waste of memory. Instead:
```
while IFS= read -r line; do
  ...
done <file1.txt
```
For information about IFS= read -r, see Understand "IFS= read -r line"?
You should also quote your variables.

thank you for your kind answer and explanation! will definitely do some self-research! — Mindy, Jan 26 '17 at 10:01

How to compare a list of hashes (line by line) against another list of hashes?

1 Answers1