How do I write a retry logic in script to keep retrying to run it upto 5 times?

Question

I want to write logic in shell script which will retry it to run again after 15 sec upto 5 times based on "status code=FAIL" if it fails due to some issue.

Answer 1

for i in 1 2 3 4 5; do command && break || sleep 15; done

Replace "command" with your command. This is assuming that "status code=FAIL" means any non-zero return code.

---

Variations:

Using the {..} syntax. Works in most shells, but not BusyBox sh:

for i in {1..5}; do command && break || sleep 15; done

Using seq and passing along the exit code of the failed command:

for i in $(seq 1 5); do command && s=0 && break || s=$? && sleep 15; done; (exit $s)

Same as above, but skipping sleep 15 after the final fail. Since it's better to only define the maximum number of loops once, this is achieved by sleeping at the start of the loop if i > 1:

for i in $(seq 1 5); do [ $i -gt 1 ] && sleep 15; command && s=0 && break || s=$?; done; (exit $s)

Answer 2

This script uses a counter n to limit the attempts at the command to five. If the command is successful, break ends the loop.

n=0
until [ "$n" -ge 5 ]
do
   command && break  # substitute your command here
   n=$((n+1)) 
   sleep 15
done

Answer 3

function fail {
  echo $1 >&2
  exit 1
}

function retry {
  local n=1
  local max=5
  local delay=15
  while true; do
    "$@" && break || {
      if [[ $n -lt $max ]]; then
        ((n++))
        echo "Command failed. Attempt $n/$max:"
        sleep $delay;
      else
        fail "The command has failed after $n attempts."
      fi
    }
  done
}

Example:

retry ping invalidserver

produces this output:

ping: unknown host invalidserver
Command failed. Attempt 2/5:
ping: unknown host invalidserver
Command failed. Attempt 3/5:
ping: unknown host invalidserver
Command failed. Attempt 4/5:
ping: unknown host invalidserver
Command failed. Attempt 5/5:
ping: unknown host invalidserver
The command 'ping invalidserver' failed after 5 attempts

For a real-world, working example with complex commands, see this script.

Answer 4

GNU Parallel has --retries:

parallel --retries 5 --delay 15s ::: ./do_thing.sh

Example:

parallel -t --retries 5 --delay 0.1s 'echo {};exit {}' ::: {0..10}

Answer 5

Here is my favorite one line alias / script

    alias retry='while [ $? -ne 0 ] ; do fc -s ; done'

Then you can do stuff like:

     $ ps -ef | grep "Next Process"
     $ retry

and it will keep running the prior command until it finds "Next Process"

Answer 6

Here is function for retry

function retry()
{
        local n=0
        local try=$1
        local cmd="${@: 2}"
        [[ $# -le 1 ]] && {
        echo "Usage $0 <retry_number> <Command>"; }

        until [[ $n -ge $try ]]
        do
                $cmd && break || {
                        echo "Command Fail.."
                        ((n++))
                        echo "retry $n ::"
                        sleep 1;
                        }

        done
}

retry $*

Output :

[test@Nagios ~]$ ./retry.sh 3 ping -c1 localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.207 ms

--- localhost ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.207/0.207/0.207/0.000 ms

[test@Nagios ~]$ ./retry.sh 3 ping -c1 localhostlasjflasd
ping: unknown host localhostlasjflasd
Command Fail..
retry 1 ::
ping: unknown host localhostlasjflasd
Command Fail..
retry 2 ::
ping: unknown host localhostlasjflasd
Command Fail..
retry 3 ::

Answer 7

Having a need to do this multiple times, the scripting was getting out of hand, so I created a dedicated tool for this called retry.

retry --until=success --times=5 --delay=15 command ...

If you need multiple commands, you can use sh -c, e.g.

retry -- sh -c 'date && false'

Retry is available here: https://github.com/minfrin/retry

Answer 8

I use this script that makes the retries of a given command, the benefit of this script is that if fails all retries it will preserve the exit code.

#!/usr/bin/env bash

if [ $# -ne 3 ]; then
    echo 'usage: retry <num retries> <wait retry secs> "<command>"'
    exit 1
fi

retries=$1
wait_retry=$2
command=$3

for i in `seq 1 $retries`; do
    echo "$command"
    $command
    ret_value=$?
    [ $ret_value -eq 0 ] && break
    echo "> failed with $ret_value, waiting to retry..."
    sleep $wait_retry
done

exit $ret_value

Probably it can get simplier

Answer 9

You can use the loop command, available here, like so:

$ loop './do_thing.sh' --every 15s --until-success --num 5 

Which will do your thing every 15 seconds until it succeeds, for a maximum of five times.

Answer 10

See below Example :

n=0
while :
do
        nc -vzw1 localhost 3859
        [[ $? = 0 ]] && break || ((n++))
        (( n >= 5 )) && break

done

I'm trying to connect port 3389 on localhost, it will retry until 5 times fail , if success then it will break the loop.

$? it's exist status of command if it zero means command successfully run , if other than zero means command fai

Seems little bit complicated, may be someone do it better than this.

Answer 11

slight mods for one-liner that has an increasing delay in seconds for the next execution

 DELAYS=(0 1 3 5); (for i in 1 2 3 4; do sleep ${DELAYS[$i]}; <COMMAND> && break || [ $i -lt 4 ] && echo "retry in ${DELAYS[$i+1]}s"; done)

example (git push used for the error)

✗  DELAYS=(0 1 3 5); (for i in 1 2 3 4; do sleep ${DELAYS[$i]}; git pish && break || [ $i -lt 4 ] && echo "retry in ${DELAYS[$i+1]}s"; done)
git: 'pish' is not a git command. See 'git --help'.
The most similar command is
    push
retry in 1s
git: 'pish' is not a git command. See 'git --help'.
The most similar command is
    push
retry in 3s
git: 'pish' is not a git command. See 'git --help'.
The most similar command is
    push
retry in 5s
git: 'pish' is not a git command. See 'git --help'.
The most similar command is
    push

replace git pish with proper git push

✗  DELAYS=(0 1 3 5); (for i in 1 2 3 4; do sleep ${DELAYS[$i]}; git push && break || [ $i -lt 4 ] && echo "retry in ${DELAYS[$i+1]}s"; done)
Everything up-to-date

Answer 12

Here's a recursive retry function for functional programming purists:

retry() {
  cmd=$1
  try=${2:-15}       # 15 by default
  sleep_time=${3:-3} # 3 seconds by default

  # Show help if a command to retry is not specified.
  [ -z "$1" ] && echo 'Usage: retry cmd [try=15 sleep_time=3]' && return 1

  # The unsuccessful recursion termination condition (if no retries left)
  [ $try -lt 1 ] && echo 'All retries failed.' && return 1

  # The successful recursion termination condition (if the function succeeded)
  $cmd && return 0

  echo "Execution of '$cmd' failed."

  # Inform that all is not lost if at least one more retry is available.
  # $attempts include current try, so tries left is $attempts-1.
  if [ $((try-1)) -gt 0 ]; then
    echo "There are still $((try-1)) retrie(s) left."
    echo "Waiting for $sleep_time seconds..." && sleep $sleep_time
  fi

  # Recurse
  retry $cmd $((try-1)) $sleep_time
}

Pass it a command (or a function name) and optionally a number of retries and a sleep duration between retries, like so:

retry some_command_or_fn 5 15 # 5 tries, sleep 15 seconds between each

Answer 13

Answering this question as existing answers fail to,

1. Doesn't throw Error Code.
2. By doing exit errCode, Bash doesn't honor certain traps such as trap somefunc ERR

COMMAND="SOMECOMMAND"
TOTAL_RETRIES=3

retrycount=0
until [ $retrycount -ge $((TOTAL_RETRIES-1)) ]
do
   $COMMAND && break
   retrycount=$((retrycount+1))
   sleep 1
done

if [ $retrycount -eq $((TOTAL_RETRIES-1)) ]
then
    $COMMAND
fi

Answer 14

    # Retries a given command given number of times and outputs to given variable
    # $1 : Command to be passed : handles both simple, piped commands
    # $2 : Final output of the command(if successfull)
    # $3 : Number of retrial attempts[Default 5]
    function retry_function() {
        echo "Command to be executed : $1"
        echo "Final output variable : $2"
        echo "Total trials [Default:5] : $3"
        counter=${3:-5}
        local _my_output_=$2 #make sure passed variable is not same as this
        i=1
        while [ $i -le $counter ]; do
            local my_result=$(eval "$1")
            # this tests if output variable is populated and accordingly retries,
            # Not possible to provide error status/logs(STDIN,STDERR)-owing to subshell execution of command
            # if error logs are needed, execute the same code, outside function in same shell
            if test -z "$my_result"
            then
                echo "Trial[$i/$counter]: Execution failed"
            else
                echo "Trial[$i/$counter]: Successfull execution"
                eval $_my_output_="'$my_result'"
                break
            fi
            let i+=1
        done
    }

    retry_function "ping -c 4 google.com | grep \"min/avg/max\" | awk -F\"/\" '{print \$5}'" avg_rtt_time
    echo $avg_rtt_time

 - To pass in a lengthy command, pass a method echoing the content. Take care of method expansion accordingly in a subshell at appropriate place.
 - Wait time can be added too - just before the increment!
 - For a complex command, youll have to take care of stringifying it(Good luck)

Answer 15

This is an old question but I found myself returning to this often. My use case was to have a one liner that can retry a command up to n times that can be used on with kubernetes pod (of course if it will work for bash script).

TRY=6; until [ $TRY -eq 0 ] || <your command over here> ; do echo $TRY; echo "<output message>"; TRY=$(expr $TRY - 1); sleep 15; done;

The one liner is a bit hard to get your head around, but it can be very helpful.