How can I find the latest log for multiple jobs in AIX and delete the latest one

Question

I am new to unix. I am trying to find the latest log of multiple jobs and delete the latest one.

Can I write a shell script for it? I have tried with basic commands, but there it is selecting a time range and can delete the log, but it may also delete useful logs .

---

Sharing what I found:

touch -t 201903281325.00 start           
touch -t 201903281331.00 stop                 
find . -newer start \! -newer stop -type f \( -name "**" -o -name "**" \) -exec rm -f {} \;

This deletes all the files in that time limit

---

What I want to do :

Let the names of jobs are A,B,C, etc, and these run 5 times a day -- say A_1, A_2, A_3, B_1, B_2, B_3, etc.

Say today's run logs generated

Order_created_20190611_1.log
Order_created_20190611_2.log 
Order_created_20190611_3.log
Order_zip_rec_20190611_1.log
Order_zip_rec_20190611_2.log

What I have to delete automatically:

Order_created_20190611_3.log
Order_zip_rec_20190611_2.log

I want to automatically find Order_created_20190611_3.log and Order_zip_rec_20190611_2.log and delete them.

Example of job names :

• Order_created -- job name
• 20190611 - Date of execution
• _2 ( mentioned at last) - is the number of time it has run. _2 means it is running 2nd time for today's run.

Answer 1

For simple filenames such as these, where you can guarantee that they will not contain strange characters such as newline, it is safe and convenient to parse the output of ls. (In the general case this is not recommended.)

As described in your question let's assume we're working with files matching these pattern, where the * will of course match "anything":

Order_created_20190611_*.log
Order_zip_rec_20190611_*.log

Additionally, for a files that is based on today's date you can find the "most recent" like this:

# Today's date in YYYYMMDD format
today="$(date +%Y%m%d)"

# Get "ls" to give us the files ordered by date modified (newest first). Pick off the first
newest="$(ls -t "Order_created_${today}_"*".log" | head -n1)"

# Show what we would delete (remove "echo" to action)
echo rm "$newest"

Using this principle you can roll this up into a loop across multiple filenames:

today="$(date +%Y%m%d)"
for prefix in Order_created Order_zip_rec
do
    newest="$(ls -t "${prefix}_${today}_"*".log" 2>/dev/null | head -n1)"
    [[ -f "$newest" ]] && echo rm "$newest"
done

Things to note

1. You shouldn't generally assume you can parse the output of ls. I have done so in this case because your file names are well defined (and there are no GNU extensions for sort or find)
2. I have discarded error output from ls in the case that there are no matching files
3. Similarly, I attempt the rm only if there is a file to delete

Answer 2

To delete the "last" (highest-numbered "run") log file for each set of jobs and dates, I would suggest a ksh93-based solution that can keep track of the highest job runs seen for each combination as well as a list of files to delete:

#!/bin/ksh93

typeset -A highestjobruns=()
typeset -A deletions=()

for logfile in *_????????_*.log
do
  base=${logfile%.log}
  run=${base##*_}
  jobdate=${base%_?}
  if [ -z "${highestjobruns[$jobdate]}" ]
  then
        # no job runs found yet, this is it; nothing to delete yet
        highestjobruns[$jobdate]=$run
  else
        # if this file has a higher run number, it's the new target for deletion
        if [ "$run" -gt "${highestjobruns[$jobdate]}" ]
        then
                highestjobruns[$jobdate]=$run
                deletions[$jobdate]=$logfile
        fi
  fi
done
printf 'Would rm: %s\n' "${deletions[@]}"
#rm -- "${deletions[@]}"

The basic idea is to loop over every potential log file (adjust the wildcard to taste; I' requiring "anything", underscore, six digits, underscore, "anything", ending in .log), pick apart the run number and the combination of job and date, then ask some questions:

1. Have we recorded any runs for this combination of job and date? If not, record this one and continue on. This means we won't delete a singleton job run.
2. If we have seen previous runs of this combination of job and date, is the run we're looking at higher than the highest one we've seen so far? If so, record the new run number and populate the current file as one we might delete. Subsequent logs of the same job and date may overwrite this entry, so we'll only every delete one log file per job name and date.

At the end, I print sample rm commands for the files that we would delete; comment that out if you like, and uncomment the actual rm command itself to do the removals.