Find files and delete x from total of the list

Question

I want to delete all log files in some directory, but not the latest 3.

I've done:

DATA_PATH=$(gadmin config get System.DataRoot)
ZK_PATH=${DATA_PATH}/zk/version-2
log_count=$(ls -ltrh ${ZK_PATH} | grep log | wc -l)
limit_files=expr $log_count - 3
echo There is ${log_count} files found in ${ZK_PATH}, ${limit_files} will be deleted, here the list:
ls -ltrh ${ZK_PATH} | grep log | head -${limit_files}
while true; do
    read -p "Are you sure to delete these files? " yn
    case $yn in
        [Yy1]* ) echo execute to delete the files; break;;
        [Nn0]* ) exit;;
        * ) echo "Please answer y or n.";;
    esac
done

How can I delete 9 of 12 files listed?

I thought about printing the list to the files first and then delete it one by one using looping, but I am sure there is code to do it only with one line.

I try to use find ... -delete, -exec, and xargs rm, but I cannot use it properly.

How can I do it?

Re: log_count=…: Why not to parse ls output (and what to do instead); Grepping for log is also a bit strange; you could just have done something like ${ZK_PATH}/*log* and gotten a list of the files, which is properly countable. — Marcus Müller, Mar 20 '23 at 08:17
i try to use find and delete all the log files found, but i want to delete the files but no with the latest 3. like i found 12 log files, so i want to delete 9 files of it — Yudha Restu, Mar 20 '23 at 08:25
is the find command already sort the files found by time modified? — Yudha Restu, Mar 20 '23 at 08:36
You haven't written a find command, so we can't tell you whether you told it to do that. But also, find is not a useful tool here. As Stéphane's answer shows, this can be done through automated filename generation alone (see my first comment!). On its own, find can't, but you can decorate with time, zero-terminate, sort, de-decorate the things. Again, complicated and not necessary, as Stéphane's answer shows. — Marcus Müller, Mar 20 '23 at 08:46

Stéphane Chazelas · Answer 1 · 2023-03-20T08:47:17.593

6

Use zsh instead of bash where this kind of thing can be done much more easily and reliably:

#! /bin/zsh -
data_path=$(gadmin config get System.DataRoot) || exit
zk_path=$data_path/zk/version-2
keep=3
files=( $zk_path/log(N.om) )
if (( $#files > keep )); then
  printf >&2 '%s\n' "$#files files found in $zk_path, $(( $#files - keep )) will be deleted, here is the list from newest to oldest:"
  shift keep files
  printf >&2 ' - "%s"\n' $files:t
read -q '?OK to delete? ' && rm -f -- $files
fi

Of special interest is the $zk_path/*log*(N.om) glob; it expands to all file names matching the pattern *log* in $zk_path, and the parentheses (…) specify that if no files are found, that's no error (Nullglob), that we're only looking for plain files and not directories, symbolic links, devices… (.), and that we want to have the files sorted in ascending order of age (based on modification time, like ls -t does).

edited Mar 20 '23 at 08:47

answered Mar 20 '23 at 08:38

Stéphane Chazelas

544,893

That's a very elegant approach there with the N-shift. Like it a lot! – Marcus Müller Mar 20 '23 at 08:45
Thanks, and for the edit @Marcus – Stéphane Chazelas Mar 20 '23 at 08:47
I felt a bit bad making that edit, it's your answer after all :) – Marcus Müller Mar 20 '23 at 08:49
Thank for the answer, i've tryed find . -name "log.*" | head -5 | xargs rm -r as a test and its looks work as far, but i still doubt. I just now about zsh, i'll learn about it, thanks for the answer and the knowladge – Yudha Restu Mar 20 '23 at 08:54
@YudhaRestu that is not sorted by date at all! Instead, it's just ordered in the order that find is presented with the files by the file system – which isn't guaranteed to follow any specific ordering. So, your find solution (in which the find is frankly a bit superfluous – echo log.* would have done the same) is complicated but wrong. Also very fragile! I sketched how you could do this correctly with find in my previous comment. Your solution really isn't. – Marcus Müller Mar 20 '23 at 09:16
@MarcusMüller. Don't. I've answered hundreds of questions with zsh glob qualifiers. Describing them every time is quite tedious. Any help is appreciated. – Stéphane Chazelas Mar 20 '23 at 09:19
@YudhaRestu see also Why is looping over find's output bad practice? for the good and bad ways to process find's results. – Stéphane Chazelas Mar 20 '23 at 09:20
1

@MarcusMüller, strictly speaking, some find implementations do or can do some ordering of the files based on inode number as an optimisation so the inode table can be read sequentially when find needs to get information from there like the file's type. In any case, the order will still appear random. – Stéphane Chazelas Mar 20 '23 at 09:26

score 6 · Answer 2 · edited Mar 20 '23 at 19:45

In a similar situation I used the following approach (probably, this is not the most robust solution, but it still may be useful in common situation...)

LOGDIR=my/logs    ##
ls -dpt -- "$LOGDIR"/log* | # get log files sorted by date
  grep -v '/$'            | # but remove directories/ from the list
  tail -n +4              | # remove also the newest 3
  vidir -

In vidir, if you agree, delete all the lines (example: use dG vim command), leave, and they get deleted.

vidir from moreutils is a projection-editor for directories (the best tool to see, rename, delete, directory contents)

Thank you for the edit @StéphnaneChazelas. – JJoao Mar 21 '23 at 08:07 — JJoao, Mar 21 '23 at 08:07

score 0 · Accepted Answer · answered Mar 21 '23 at 01:57

0

This gets you a list of all but the youngest limit_files files, which you can then remove.

FileList=`ls -1rt ${ZK_PATH}/*log* | head -${limit_files}`
rm $FileList

It won't work with files that contain spaces or other special characters, but the log files in my log directories only ever have alphanumeric characters, dash and period, so works perfectly.

answered Mar 21 '23 at 01:57

RonJohn

1,148

Whether it fails of files with spaces depends on what you have in $IFS. $IFS also needs to contain the newline character as you rely on split+glob to split $FileList. None of the files must be directories, nor start with a dash. – Stéphane Chazelas Mar 21 '23 at 06:36

Find files and delete x from total of the list

3 Answers3