0

I'm currently working on a disaster recovery setup in Azure we have a cassandra cluster running on one region. The plan is if the region goes down, we will utilize Azure Site Recovery to failover all machines to the secondary region.

Each region has its own IP set, lets say region A is 10.1.0.x region B is 10.2.0.x

As this means that I need to change the cassandra.yaml file, I created a simple file that runs on startup with crontab that changes the following entries

- seeds: 10.1.0.101,10.1.0.102,10.1.0.103
rpc_address: 10.1.0.101
listen_address: 10.1.0.101

into the following on the secondary site

- seeds: 10.2.0.101,10.2.0.102,10.2.0.103
rpc_address: 10.2.0.101
listen_address: 10.2.0.101

The seed is a fixed list, so I made it like this

# get node IP address
IPADDR=$(ip addr show eth0 | grep 'inet ' | cut -d' ' -f6 | awk '{ print $1}' | cut -d'/' -f1)

breakdown IP address into IP blocks of sec1.sec2.sec3.sec4


slice="$IPADDR"
count=1
while [ "$count" -le 4 ]
do
        declare sec"$count"="${slice%%.}"
        slice="${slice#.}"
        count=$((count+1))
done


check the sec2 block of the IP and define the IP seed list


if [[ $sec2 == 2 ]]
then


the below seed list is only for use in ASR testing - need to reconfigure this when running failover/failback in actual production


SEED="10.1.0.101,10.1.0.102,10.1.0.103"



else
    SEED="10.2.0.101,10.2.0.102,10.2.0.103"
fi


#change the seed list, rpc_address and listen_address of each cassandra nodes
sed -i "0,/^\s* - seeds:.*/s// \         - seeds: $SEED/" /etc/cassandra/conf/cassandra.yaml

Now this part works fine, when I failover the VMs, the seed list is properly changed.

now for the rpc_address and listen_address, I did this

LOCALIPADDR=$(ip addr show eth0 | grep 'inet ' | cut -d' ' -f6 | awk '{ print $1}' | cut -d'/' -f1)
sed -i "0,/^rpc_address:.*/s//rpc_address: $LOCALIPADDR/" /etc/cassandra/conf/cassandra.yaml
sed -i "0,/^listen_address:.*/s//listen_address: $LOCALIPADDR/" /etc/cassandra/conf/cassandra.yaml

This works fine if run the script on my shell. The both the rpc_address and listen_address gets updated according to the ip address. But when running in crontab at startup, all I got was

rpc_address: 
listen_address:

without any ip address listed in the file.

The crontab looks like this

@reboot /bin/sh /etc/cassandra/conf/asr-address-conf.sh

Any idea why the script not working on crontab?

Bayu Riyadi
  • 1
  • Have you checked the usual causes, such as your PATH not being the same when run under cron? Please [edit] your post to include the specific error message you get when running under cron. Also, since your shell uses bash-isms, you may want to run it in cron using bash. – Jim L. Nov 16 '21 at 22:07
  • @JimL. Thanks for the inputs. I've modified the crontab to /bin/sh then tried adding PATH variable but no luck. Added direct paths to each commands (i.e. /bin/sed /bin/cut etc.) still didn't work. The fact that there's really nothing under /var/log/cron or journalctl doesn't help in troubleshooting this. what confuses me is the first part of the script works. IPADDR and LOCALIPADDR uses the same set of commands. I tried to send the value of LOCALIPADDR to /tmp/script.log to see what I got. Its just empty – Bayu Riyadi Nov 20 '21 at 02:10
  • Look at your script! You're using bash syntax. Don't use /bin/sh, use /bin/bash. And rather than trying to dump specific variables to /tmp/script.log you might instead just do a dump of the entire output of set. That will show you EVERYTHING, without you having to guess which variable might be absent or incorrect, and will also include the PATH, etc. etc. Also, and I can't stress this enough, CAPTURE THE STDERR from your script, so you can see the error messages it's creating. [Edit] your post to include those error messages so we can see what's going wrong. – Jim L. Nov 20 '21 at 07:28

0 Answers0