1

My question is similar to this one but the answers don't address how I would like to solve it in my case:

Background: I have two computers generating experimental results that are filed in directories based on timestamps. E.g.:

190525184212  190525184433  190525185428  190525194813 ...

I am moving the results to one host computer using rsync (with --remove-source-files) but since it is possible for two experiments to start during the same second it is possible that this will overwrite results already on the host computer.

What I would like is it append to the directory names when they are identical. Something like this:

190525184212  190525184433  190525184433_1  190525185428  190525194813 ...

(The alternative would be for each machine to add its own unique id to the directory names to avoid conflicts but that means hashing the hostname or something and I am not expecting many conflicts so the solution above would be more appealing to me)

I'm guessing I have to write some kind of bash script (like this one but it was down-voted)? But was hoping for a ready-made solution.

thanks.

Footnote: The sub-directories are not all in one main directory as implied above. There are more than one sub-directory so solutions have to be recursive.

asktyagi
  • 675
Bill
  • 129
  • Not getting any feedback, I'll assume this is not a good idea and do the merge operation in Python. – Bill May 26 '19 at 19:47
  • the title of your post is convoluted ... a simpler title may attract more attention .... something like how do I merge identically named files? – jsotola May 26 '19 at 20:31
  • 1
    @jsotola They don't want to merge the files though, they want to rename them if they already exist. – Sparhawk May 26 '19 at 23:56
  • rsync with "no clobber" sounds (to me) like one should use "cp" with appropriate command options, something like: cp --backup=existing -t – Theophrastus May 27 '19 at 00:13
  • The new title is misleading. Maybe "How do I merge two sets of files without over-writing duplicates?" would be better. – Bill May 27 '19 at 00:29
  • @Bill That sounds better to me. You should edit your question with the new title. – Sparhawk May 27 '19 at 01:42
  • @Theophrastus I'm not sure that cp works here, since it has to be recursive. – Sparhawk May 27 '19 at 01:45
  • @Sparhawk, you are missing the point of my comment ... the title does not have to, and usually cannot, describe everything in detail .... however, a cryptic title may keep some people from actually reading the post – jsotola May 27 '19 at 02:12
  • @jsotola Agreed, but here I think the new title directly conflicts with the question. They do not want to merge the files at all, but do the opposite, i.e. keep them separate. – Sparhawk May 27 '19 at 03:17
  • A (shell) scripted solution would usually mean you have no password prompt for login - or be very insecure in storing you PW as string. – FelixJN May 27 '19 at 14:26
  • 1
    If you are looking to distinguish 'same name' from 'same file' then why not just script a find, compare file names and rename any files with the same names but different MD5 or SHA256 hashes? – bu5hman May 29 '19 at 17:56
  • most simple solution for your problem (not your question) would be to append hostnames to output file names in the prog which generates them and later merge them… – Jaleks May 29 '19 at 20:07

0 Answers0