Run command automatically when files are copied into a directory

Question

I have two folders called: A and B, in different paths on the same computer. When I add any new file(s) into folder A, I want to copy it to folder B automatically.

My folders:

/auto/std1/nat1/A
/auto/std2/nat2/B

What I currently do to copy the files:

cp -r A B

But I want this process to run automatically in the background for every new file and folder in A into B.

Added question/problem

While copying files I would like specific actions to be performed on certain files types, example: when I have a zip file in folder A, I would like it to unzip this file in folder B automatically.

This is on a CentOS 7` system.

Answer 1

Per your bonus question, add the following line below the rsync command in the shell script I provided below. I wrote this in the comment but I'll officially add it to my answer here:

    find /auto/std2/nat2/B -name '*.zip' -exec sh -c 'unzip -d `dirname {}` {}' ';'

This will handle unzipping all the zip files that are copied via rsync from folder /auto/std2/nat2/A to /auto/std2/nat2/B

---

If you have rsync installed why not just cron it and have rsync manage the file mirroring?

Create script myrsyncscript.sh

Don't forget to make it executable: chmod 700 myrsyncscript.sh

#!/bin/sh

LOCKFILE=/tmp/.hiddenrsync.lock

if [ -e $LOCKFILE ]
        then
        echo "Lockfile exists, process currently running."
        echo "If no processes exist, remove $LOCKFILE to clear."
        echo "Exiting..."
#        mailx -s "Rsync Lock - Lock File found" myemail@domain.com <<+
#Lockfile exists, process currently running.
#If no processes exist, remove $LOCKFILE to clear.
#+
        exit
fi

touch $LOCKFILE
timestamp=`date +%Y-%m-%d::%H:%M:%s`
echo "Process started at: $timestamp" >> $LOCKFILE

## Run Rsync if no Lockfile
rsync -a --no-compress /auto/std1/nat1/A /auto/std2/nat2/B


echo "Task Finished, removing lock file now at `date +%Y-%m-%d::%H:%M:%s`"
rm $LOCKFILE

Options breakdown:

-a is for archive, which preserves ownership, permissions etc.
--no-compress as there's no lack of bandwidth between local devices

Additional options you might consider man rsync:

--ignore-existing

skip updating files that exist on receiver

--update

This forces rsync to skip any files which exist on the destination and have a modified time that is newer than the source file. (If an existing destination file has a modification time equal to the source file’s, it will be updated if the sizes are different.) Note that this does not affect the copying of symlinks or other special files. Also, a difference of file format between the sender and receiver is always considered to be important enough for an update, no matter what date is on the objects. In other words, if the source has a directory where the destination has a file, the transfer would occur regardless of the timestamps.

This option is a transfer rule, not an exclude, so it doesn’t affect the data that goes into the file-lists, and thus it doesn’t affect deletions. It just limits the files that the receiver requests to be transferred.

Add it to cron like so, and set the frequency to whatever you feel most comfortable with:

Open cron with crontab -e and add the below:

### Every 5 minutes
*/5 * * * * /path/to/my/script/myrsyncscript.sh > /path/to/my/logfile 2>&1 

# * * * * *  command to execute
 # │ │ │ │ │
 # │ │ │ │ │
 # │ │ │ │ └───── day of week (0 - 6) (0 to 6 are Sunday to Saturday, or use names; 7 is Sunday, the same as 0)
 # │ │ │ └────────── month (1 - 12)
 # │ │ └─────────────── day of month (1 - 31)
 # │ └──────────────────── hour (0 - 23)
 # └───────────────────────── min (0 - 59)

Answer 2

Implementing @Izkata's suggestion using inotifywait with paced event response to keep the rsyncs down to at most 1 every 5 minutes while still responding quickly to initial changes:

#!/bin/sh
# usage: whateveryouwanttotcallthis "$directorytowatch" rsync args here

cd "$1" || { echo "${0##*/}: can't cd to $1"; exit 1; }
shift
rsync -nq "$@" || { echo "rsync doesn't like your $# arguments $@"; exit 1; }

notbefore=0
watchlock=.lock.inotifywait
rsynclock=.lock.rsync-pending

mkdir $watchlock ||
       { echo "${0##*/}: already running, rmdir '$PWD/$watchlock' if kill -9 took it out"
         exit 1;
       }
trap "rmdir '$watchlock'" 0 1 2 3 15

inotifywait -m -e close-write . |    # maybe also add move
        while read; do
                mkdir $rsynclock 2>&- || continue
                schedule=$(( notbefore <= (now=`date +%s`+2) ? (now+2)
                                                           : (notbefore) ))
                notbefore=$((schedule+300))
                ( ( trap "rmdir '$rsynclock'" 0 1 2 3 15
                    sleep $(( schedule-now ))
                  )
                  # substitute your payload here
                  rsync --exclude='.lock.*' "$@" \
                          || echo ssia | mail -s "${0##*/} '$PWD' rsync failed" opswatch
                ) &
        done

The two-second delay both helps with batching up small bursts and allows time for the renames some things do when a write is complete. Maybe 15 seconds would be better.

Can't test, on windows atm, I hope and believe it's at least pretty darn close.

Answer 3

You could use what DevNull suggested which rsyncs periodically. Personally I would use inotify. It is a nifty tool that you can give a folder to watch. It sets up watches and notifies you whenever a filesystem change occurs. You could then trigger an rsync based on the trigger from inotify.

For the specific case at the end you talk about, you can use the trigger from inotify to see what change has happened and then write a simple bash script to check if it is a zip file that has been added to folder A and if it is, then you can just unzip it to folder B (or whatever else you would like to do) instead of just copying the zip.