9

I'd like to use rsync to...

  • delete files on the receiving side that were also deleted on the sending side
  • not delete other files that are in the rsynced directory on the receiving side

For example, let's say I have a directory local-src:

BEFORE: local-src locally contains...

a.txt
b.txt
c.txt

my remote directory that I'd like to sync to contents of local-src to is called remote-src.

BEFORE: remote-src remotely contains...

a.txt
b.txt
c.txt
d.txt
README.md

Let's say that I delete some files in local-src:

AFTER LOCAL DELETE: local-src locally contains...

c.txt

How can I use rsync in a way to ensure that the files deleted at the source are also deleted at the destination, but without deleting other files at the destination. For example, I'd like to have the following at the destination:

AFTER LOCAL DELETE: remote-src remotely contains...

c.txt
d.txt
README.md

That is, a.txt and b.txt are remotely deleted as well, but d.txt and README.txt are left alone.

Is there any way to achieve this with rsync?

EDIT: The verdict seems to be that this might be impossible with rsync. I've been asked why I need this, so to illustrate my use-case:

Let's say I have a web server. On that web server, I've got a bunch of directories, let's say that I have a directory A and a public_html directory that my site is served from. Let's say that I have some automated process that produces files in directory A. I'd like to rsync (or sync using some other tool) the files generated or updated in A to the public_html directory, without deleting other arbitrary files that might be within public_html. I certainly don't want rsync to to accidentally delete my website.

If rsync isn't the tool for this job, does someone else know how I can do this?

Heather Miller
  • 191
  • 2
    After re-reading your question, I don't think this is possible with rsync as there is no way to know which files where already in the remote folder. You might need to find out another tool. – Spack May 22 '13 at 21:02
  • rsync won't let you do this, but if you scp the whole directory everytime you delete files you can keep them in sync, not a solution just a suggestion. – Aadi Droid Jul 03 '13 at 05:40
  • 1
    I'm guessing you've already thought of this, but couldn't you just put these files in a subdirectory (or somewhere else) and reference them from public_html? That way you have one directory that is easily and explicitly synced, without affecting files in other parts of the webserver's filesystem. – MattJenko Mar 01 '14 at 04:06

4 Answers4

2

What you want to do is reasonable, but using rsync to do it on its own is not. So the answer is no.

The reason is simple: rsync keeps no history of what was in each directory and has no way of knowing what needs to be deleted and what not. Not without additional support.

You should ask yourself why you like to do this with rsync and make that more clear. There are other programs that use librsync1.so that are more intelligent.

With the relaxed constraints that you don't need rsync per se, you can take a look at rdiff-backup: 
mkdir a
touch a/xx
touch a/yy
rdiff-backup a b
ls b

This shows xx and yy are in b.

touch b/zz
rm a/xx
rdiff-backup a b

This shows xx and zz are in b. rdiff-backup also keeps a directory rdiff-backup-data in b so you can rollback any changes, you should purge this on a regular basis using the rdiff-backup commands. (The example is with local files to show extra data in the target does not get deleted, but rdiff-backup works over a network as well).

Another alternative is to setup some distributed revision control system (mercurial, bazaar, git). With mercurial e.g. you can have a script (I use a Makefile for that), that pushes all the changes to the server and then does an update of the checked out files there, ignore any additional files that are on the remote server (but have not been put under revision control).

On the server you would do:

hg init
hg add file_list_excluding_that_should_not_should_be_deleted_if_not_on_client
hg commit -m "initial setup"

On the client:

hg clone ssh://username@server/dir_to_repository

Now if you remove a file on the client and do:

hg commit -m "removed file"
ssh username@server "cd dir_to_repository; hg update --clean"

Your removed file is removed on the server, but any other data (not added to the repository) does not get deleted.

Anthon
  • 79,293
  • I can accept that rsync won't do this. But I don't agree that this would be impossible with rsync- if rsync knows on the sending side which files have been deleted, why can't it send that information to the receiving side in the diff? After comparing for freshness, I don't see why the receiving side can't just delete the files that were indicated to be deleted in the diff, without deleting everything else in the directory. Deleting every other (undeleted at the source) innocent file in the directory seems unreasonable to me. – Heather Miller May 23 '13 at 05:36
  • In any case, the reason I need this is as follows. I have a directory, let's call it A, where some process is automated, and files are automatically generated there. I've got a web server, and I'd like the files within A to be rsynced to the public_html directory of the webserver, of course without deleting everything else in the public_html folder of the web server. If anyone has any ideas for achieving this with another tool, it'd be more than welcome. I'll update my question to reflect this. – Heather Miller May 23 '13 at 05:40
  • To clarify my first comment above- I should've said "I don't agree that something like this should be impossible with a tool like rsync". Intuitively, it just seems like this can't be too hard to achieve (unless I'm missing something). – Heather Miller May 23 '13 at 05:49
  • Hmm, OK. I think I see now- how can rsync know when something was deleted in the local-src directory without having some process watching that directory for changes. Maybe this would be tough. – Heather Miller May 23 '13 at 06:01
  • @HeatherMiller Like I wrote, your request is reasonable, but rsync is not the tool. Please realise that sync in rsync comes from synchronising and that is not exactly what you want to do. In the development of rsync the focus has been on efficient (minimizing) the transfer of data. Other tools like rdiff-backup (and possible cvsup) have used its techniques for that but build upon it with additional features. – Anthon May 23 '13 at 06:25
  • @HeatherMiller I updated my answer with two options based on the relaxed constraints. – Anthon May 23 '13 at 06:47
1

I don't think this is possible without explicitly excluding the files on the receiving side as part of the rsync command. See the man page section for rsync: "PER-DIRECTORY RULES AND DELETE".

Without a delete option, per-directory rules are only relevant on the sending side, so you can feel free to exclude the merge files themselves without affecting the transfer. To make this easy, the ’e’ modifier adds this exclude for you, as seen in these two equivalent commands:

          rsync -av --filter=': .excl' --exclude=.excl host:src/dir /dest
          rsync -av --filter=':e .excl' host:src/dir /dest

However, if you want to do a delete on the receiving side AND you want some files to be excluded from being deleted, you’ll need to be sure that the receiving side knows what files to exclude. The easiest way is to include the per-directory merge files in the transfer and use --delete-after, because this ensures that the receiving side gets all the same exclude rules as the sending side before it tries to delete anything:

          rsync -avF --delete-after host:src/dir /dest

However, if the merge files are not a part of the transfer, you’ll need to either specify some global exclude rules (i.e. specified on the command line), or you’ll need to maintain your own per-directory merge files on the receiving side. An example of the first is this (assume that the remote .rules files exclude themselves):

   rsync -av --filter=’: .rules’ --filter=’. /my/extra.rules’
      --delete host:src/dir /dest

In the above example the extra.rules file can affect both sides of the transfer, but (on the sending side) the rules are subservient to the rules merged from the .rules files because they were specified after the per-directory merge rule.

In one final example, the remote side is excluding the .rsync-filter files from the transfer, but we want to use our own .rsync-filter files to control what gets deleted on the receiving side. To do this we must specifically exclude the per-directory merge files (so that they don’t get deleted) and then put rules into the local files to control what else should not get deleted. Like one of these commands:

       rsync -av --filter=':e /.rsync-filter' --delete \
           host:src/dir /dest
       rsync -avFF --delete host:src/dir /dest
slm
  • 369,824
0

If I correctly understood, the --exclude might be what you're looking for:

$ ls src dst
dst:
a.txt  b.txt  c.txt  d.txt  README.md

src:
c.txt
$ rsync --update --delete --recursive --exclude="d.txt" --exclude="README.md" src/ dst
$ ls src dst
dst:
c.txt  d.txt  README.md

src:
c.txt
Spack
  • 1,997
  • Well, no. I don't want to have to manually list all of the files that I'd like to exclude. I'd just like rsync only to delete the files that I've deleted at the source- I shouldn't have to know at the source what other possible files exist in the same directory at the destination. – Heather Miller May 22 '13 at 20:57
0

I have an answer for that. I think it works. And it works for me . First you should have to rsync remote files to local files. Then local side contains all the files.

sudo rsync -r -a -v --delete /root@xx.xx.xx.xx:/remote_dir/ /local_dir/

now in local side

a.txt
b.txt
c.txt
d.txt
README.md

Then you can delete the files or do whatever you want.(in local side). In your question ,you delete these files.

deleted files

a.txt
b.txt

After that you can rsync local files to the remote side.Then both sides have same files.

sudo rsync -r -a -v --delete /local_dir/ root@xx.xx.xx.xx:/remote_dir/

it gives 

c.txt
d.txt
README.md

files in remote side and local side.(By using --delete, it deletes other files in remote side that are not match with local side).

Thusitha Sumanadasa
  • 247