Find result not up to date

Question

My problem : The results returned by the find command do not seem "up to date".

I have a bash script in which :

1. i get the list of gif files in a directory
2. i use convert (from imagemagick) on them to make png
3. i get the list of png files in the same directory (which may contain the previously generated ones if there were gif files here)
4. i optimize them with optipng

If i let this script as is, when i have a gif, the corresponding png is not found. If i force the script to pause with a read from keyboard, then the new png is found.

My interpretation :

• either the convert immediately returns, so the find png executes before the gif to png file is created. In this case, how to force the next command to wait for it ?
• or the find command uses some system functions which needs time to update. In this case, how to force update on dir info ?

Thanks

Answer 1

Following your four steps (ignoring the "get the list..." steps as they are not needed):

#!/bin/bash

topdir=some/directory/path

find "$topdir" -type f -name '*.gif' -exec sh -c '
    for gifpath do
        convert "$gifpath" "${gifpath%.gif}.png"
    done' sh {} +

find "$topdir" -type f -name '*.png' -exec optipng {} \;

This would first find all regular files whose names end in .gif in or under $topdir, and would convert these to PNG images.

It then finds all the PNG images in the same way and runs optipng on each (note that I know nothing about this command so I'm just guessing here). The second find would also find PNG images that possibly existed before running the script.

If you only want to run optipng on the newly created PNG images, use a single find:

find "$topdir" -type f -name '*.gif' -exec sh -c '
    for gifpath do
        pngpath=${gifpath%.gif}.png
        convert "$gifpath" "$pngpath"
        optipng "$pngpath"
    done' sh {} +

Related: Understanding the -exec option of `find`

Or, using ** in bash to match recursively under $topdir:

#!/bin/bash

topdir=some/directory/path

shopt -s globstar nullglob dotglob

for gifpath in "$topdir"/**/*.gif; do
    if [ -f "$gifpath" ]; then
        pngpath=${gifpath%.gif}.png
        convert "$gifpath" "$pngpath"
        optipng "$pngpath"
    fi
done

In each and every piece of code above, the optipng command would not run until the convert command finished. If still this gives you issues of the same nature as you describe (the PNG images are not available after convert), then I would look for error messages that describe what went wrong.

Answer 2

Someone recommended a sync command, which would have made the system flush all caches, and then avoid unfinished disk business.

But this strange bug disappeared. My guess : syntax quirk on my part, not detected. Nevertheless, thanks for your help, and here what it looks like now :

#!/bin/bash

clear

mapfile -t tab < <(jq -r '.[] | .' img_folders_to_optimize.json)
printf '%s\n' "${tab[@]}" | while read -r folder; do
  echo "dossier à traiter : $folder"
  cd "$folder" || exit

  # GIF
  for filepath in *.gif; do
    echo "GIF : $filepath"
    nb_frames=$(identify -format %n "$filepath")
    if [ "$nb_frames" != 1 ]; then
      echo "animated"
    else
      echo "convert to png"
      filepath_png=${filepath/\.gif/.png}
      convert "$filepath" "$filepath_png" &>/dev/null
      echo "converted"
    fi
  done

  # PNG
  for filepath in *.png; do
    echo "PNG : $filepath"
    echo "optimize"
    optipng -o7 -strip all "$filepath" &>/dev/null
    echo "convert lossless to webp"
    filepath_webp=${filepath/\.png/.webp}
    cwebp -lossless "$filepath" -o "$filepath_webp" &>/dev/null
  done

  # JPG
  for filepath in *.jpg; do
    echo "JPG : $filepath"
    echo "optimize"
    mogrify -quality 70 "$filepath"
    echo "convert lossy to webp"
    filepath_webp=${filepath/\.jpg/.webp}
    cwebp -q 50 -m 4 -mt "$filepath" -o "$filepath_webp"
  done

  echo "dossier traité"
done