73

I am using diff -r a b to recursively compare directories a and b. It often happens though that there are some broken links (the same broken links in both a and b directories and pointing to the same, non-existing targets).

diff then outputs error messages for those cases and exits with a non-zero exit code, however I would like it to stay silent, and exit with 0 as the directories are the same in my book.

How can I do that?

ctrl-alt-delor
  • 27,993
Marcus Junius Brutus
  • 4,587
  • 11
  • 44
  • 65

3 Answers3

47

For version 3.3 or later of diff, you should use the --no-dereference option, as described in Pete Harlan's answer.

Unfortunately, older versions of diff don't support ignoring symlinks:

Some files are neither directories nor regular files: they are unusual files like symbolic links, device special files, named pipes, and sockets. Currently, diff treats symbolic links like regular files; it treats other special files like regular files if they are specified at the top level, but simply reports their presence when comparing directories. This means that patch cannot represent changes to such files. For example, if you change which file a symbolic link points to, diff outputs the difference between the two files, instead of the change to the symbolic link.

diff should optionally report changes to special files specially, and patch should be extended to understand these extensions.

If all you want is to verify an rsync (and presumably fix what's missing), then you could just run the rsync command a second time. If you don't want to do that, then check-summing the directory may be sufficient.

If you really want to do this with diff, then you can use find to skip the symlinks, and run diff on each file individually. Pass your directories a and b in as arguments:

#!/bin/bash
# Skip files in $1 which are symlinks
for f in `find $1/* ! -type l`
do
    # Suppress details of differences
    diff -rq $f $2/${f##*/}
done

or as a one-liner:

for f in `find a/* ! -type l`;do diff -rq $f b/${f##*/};done

This will identify files that differ in content, or files which are in a but not in b.

Note that:

  • since we are skipping symlinks entirely, this won't notice if symlink names are not present in b. If you required that, you would need a second find pass to identify all the symlinks and then explicitly check for their existence in b.
  • Extra files in b will not be identified, since the list is constructed from the contents of a. This probably isn't a problem for your rsync scenario.
Community
  • 1
ire_and_curses
  • 12,362
  • The proposed script does not work recursively for any directories present in directory 'a' (the paths created for 'b' using b/${f##*} are not correct). – Marcus Junius Brutus Aug 28 '14 at 11:37
  • @MarcusJuniusBrutus - Yes, you're right. I think the solution is to remove a #, e.g. for f infind a/* ! -type l;do echo $f b/${f#*/};done. I don't have time to test this right now though. Let me know if that works. – ire_and_curses Aug 28 '14 at 20:01
  • It is better however it still messes up the filepaths in many cases. The script (with a # removed) appears to need to be invoked from a directory directly over 'a' to work. – Marcus Junius Brutus Aug 28 '14 at 21:08
  • This answer becomes obsolete when using GNU diff 3.3 (see postings below) – Bernd Gloss Aug 15 '15 at 15:18
  • The script above has several problems, due to first finding all file names and feeding them to an expanded command line. (1) It will only work with small collections of files since it. (2) Any file name with special character (even a space) won't be processed. (3) Always use $(xxx) instead of backticks. Backticks symmetry makes them less readable and prevents nesting. Regarding 1 and 2 see http://stackoverflow.com/questions/11366184/find-while-loop-with-files-containing-spaces-in-bash – Stéphane Gourichon Oct 08 '16 at 06:59
24

Since version 3.3 GNU diff supports not dereferencing symlinks, but then compares the paths they point to.

Install GNU diffutils >= 3.3 and use the --no-dereference option; there is no short option for that.

Diagnostic will be silent if equal or:

Symbolic links /tmp/noderef/a/symlink and /tmp/noderef/b/symlink differ

don_crissti
  • 82,805
Philippe De Muyter
  • 381
  • Now if only it would show the content changes, as though the symlink were a regular file... :-/ – lindes Nov 12 '15 at 19:46
10

You can use a newer version of diff

The diff in GNU diffutils 3.3 includes a --no-dereference option that allows you to compare the symlinks themselves rather than their targets. It reports if they differ, is quiet if they agree and doesn't care whether they're broken.

I don't know when the option was added; it isn't present in 2.8.1.

don_crissti
  • 82,805
Pete Harlan
  • 101