0

We have a Kafka production machine on Red Hat Enterprise Linux.

  1. How can we remove all the files under /var/kafka/kafka-logs that end with .index?

  2. How can we move all the files that end with .index to another folder, /var/tmp/INDEX_BACKUP?

Example contents under /var/kafka/kafka-logs:

./hd3gd.ewhd.pri.processed-98/00000000000000000011.index
./hd3gd.ewhd.pri.processed-99/00000000000000000000.index
./hd3gd.ewhd.suspected_relations-0/00000000000000000000.index
./hd3gd.ewhd.suspected_relations-1/00000000000000000000.index
./hd3gd.ewhd.suspected_relations-2/00000000000000000000.index
./hd3gd.ewhd.suspected_relations-3/00000000000000000000.index
./hd3gd.ewhd.suspected_relations-4/00000000000000000000.index
./hd3gd.ewhd.suspected_relations-5/00000000000000000000.index
./frfwjnwe.fwefew.heartbeat-0/00000000000000000000.index
./frfwjnwe.fwefew.heartbeat-1/00000000000000000000.index
./frfwjnwe.fwefew.heartbeat-1/00000000000000017239.index
./frfwjnwe.fwefew.heartbeat-2/00000000000000000000.index
./frfwjnwe.fwefew.heartbeat-2/00000000000000017238.index
Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
yael
  • 13,106

4 Answers4

5

  • to remove all files ending with .index under /var/kafka/kafka-logs, using GNU find or compatible:

    find /var/kafka/kafka-logs -name \*.index -delete

    POSIXly:

    find /var/kafka/kafka-logs -name \*.index -exec rm -f {} +

  • to move them to another folder, with GNU mv:

    find /var/kafka/kafka-logs -name \*.index -exec mv -t /var/tmp/INDEX_BACKUP {} +

    POSIXly:

    find /var/kafka/kafka-logs -name \*.index -exec sh -c '
  exec mv "$@" /var/tmp/INDEX_BACKUP/' sh {} +
Stéphane Chazelas
  • 544,893
Stephen Kitt
  • 434,908
0

Ad.1.:

find /var/kafka/kafka-logs -type f -name "*.index" -exec rm -v {} \;

Answer for the second question you can find here: How to integrate mv command after find command?.

mariaczi
  • 559
0

Such command will work faster (comparing with find / -exec) due to larger arguments to rm command at once. This not requires GNU find.

Q1.

find /var/kafka/kafka-logs -type f -name "*.index" | xargs rm

Q2.

find /var/kafka/kafka-logs -name "*.index" | xargs -I {} mv {} /var/tmp/INDEX_BACKUP

Also note, that GNU find does not support ACLs or Extended Attributes, so you definitely do not want to use GNU find on Solaris.

Sasha Golikov
  • 251
  • It will not work faster that @StephenKitt's -delete where the deletion is done internally by find and would be a lot faster than invoking a separate rm utility in a child process. It's not faster than -exec rm -f {} + either which like xargs runs as few rm invocations as necessary. But it has a few problems like: it breaks with file names containing blanks, newlines, quotes..., If rm issues a prompt, it will read the response from the output of find. – Stéphane Chazelas May 14 '18 at 12:41
  • Dear @Stéphane, not all systems have GNU find. For example Solaris 10. – Sasha Golikov May 14 '18 at 12:45
  • -exec rm -f {} + is standard, added to SysV in the mid 80s. In POSIX since 1992. Actually, GNU find was one of the last ones to support it. Solaris definitely supports it. -delete is a GNU extension (though also supported by a few other implementations now). – Stéphane Chazelas May 14 '18 at 12:55
  • Solaris11 supports -delete only in its GNU extension (/usr/gnu/bin/find). Solaris10 has not it preinstalled, admin needs install it. – Sasha Golikov May 14 '18 at 13:01
  • And please note that GNU find does not support ACLs or Extended Attributes, so you definitely do not want to use GNU find on Solaris. – schily May 23 '18 at 12:17
0

Using bash and assuming that there are not thousands of these files:

  1. rm /var/kafka/kafka-logs/**.index

  2. mv /var/kafka/kafka-logs/**.index /var/tmp/INDEX_BACKUP

    This would not check whether the names on the destination already exist, so it may overwrite files there (with your example files, you would only end up with a single copy of the 00000000000000000000.index file, for example).

Both of these commands assumes that the globstar shell option has been set in bash using

shopt -s globstar

This enables the ** glob pattern, which will match like * but across / in pathnames.

Community
  • 1
Kusalananda
  • 333,661