Delete all files and folders not in a list

Question

Given a text file containing paths that I want to keep like:

/mnt/cache/vfs/cf/A/file
/mnt/cache/vfs/cf/B/file2

And I want to clear everything else under /mnt/cache/vfs/cf unless it's in my text file

So /mnt/cache/vfs/cf/Z/file3 is deleted, etc

The text file is huge, and has filenames with spaces and possible accents or other special characters

Answer 1

I would list the files in the filesystem, remove those that exist in your set of files to be kept, and delete the remainder.

Here I've used NULL-terminated filenames throughout so that there is no confusion with xargs between a filename containing spaces and its space separated parts:

find /mnt/cache/vfs/cf -type f -print0 |
    LC_ALL=C sort -z |
    LC_ALL=C comm -z -23 - <(LC_ALL=C sort list-of-files-to-keep.list | tr '\n' '\0') |
    xargs -0 printf '%s\n' {}

Replace printf '%s\n' with rm -- when you are ready to perform the deletions.

The comm command takes two sorted files and compares them line by line. The first column of output is entries only in the first file, the second is entries only in the second file, and the third is entries in both files. The -1, -2, and -3 qualifiers inhibit output of the corresponding column, so our comm -23 will output lines that are present only in the first file (-, i.e stdin).

I've forced the locale to C so that sort and comm work in a consistent manner with each other (comm requires sorted input), and also so that every line is sorted in deterministic manner (some locales sort sets of characters the same, so the characters in such a set may be ordered in an inconsistent way).

Answer 2

What I would do:

shopt -s extglob
cd /mnt/cache/vfs/cf
{   printf 'rm !('; awk -F'/mnt/cache/vfs/cf' '{print $2}' file |
    paste -sd '|'
} | sed 's/$/)/'

When you are happy with the output, you can pipe the whole snippet in bash:

shopt -s extglob
cd /mnt/cache/vfs/cf
{   printf 'rm !('
    awk -F'/mnt/cache/vfs/cf' '{print $2}' file |
    paste -sd '|'
} | sed 's/$/)/' | 
    bash

See http://mywiki.wooledge.org/glob#extglob
and https://www.gnu.org/software/bash/manual/html_node/Pattern-Matching.html

Answer 3

I would use a simple one-liner like this:

for FILE in $(ls /mnt/cache/vfs/cf); do grep "$FILE" keep_files.txt || rm "/mnt/cache/vfs/cf/$FILE"; done

---

But I would, depending on how many files you want to delete, recommend to always move them into a temporary directory, to make sure you don't remove a file you need by accident :)