3

Problem: I'm looking for a way to rename or copy a file without overwriting the destination file, if it exists, and then check the success of the move or copy operation. I'm seeking a method that will work with the BSD versions of mv/cp installed on MacOS/Unix, and also the GNU coreutils versions I have on Linux.

Solution attempt: In all versions of mv/cp, I can prevent overwriting the destination file with the -n flag:

mv -n file1 file2
cp -n file1 file2

Similar questions suggest testing the success of mv and cp using the exit status, which is 0 if successful and >0 if an error occurred. However, for both versions of mv/cp, the exit code is 0 when the destination file already exists and the -n flag is used.

The only other option I can think of is to also use the -v flag, and look at the output of the command: 

mv -nv file1 file2
cp -nv file1 file2

However, the GNU and BSD versions of mv/cp behave differently when the -nv flags are used and file2 already exists: the GNU versions of mv/cp return nothing, whereas the BSD versions return file2 not overwritten.

Our previous method was to check whether the destination file exists first, then do the mv/cp operation. Believe it or not, this caused problems because the destination file would sometimes get created by another process between the time that the check was performed and the mv/cp operation was executed.

Is there a way to accomplish this task that works with both BSD and GNU versions of mv/cp?

Alternatively, is there are way to do this using Python 2? I couldn't find a way to do this using os.rename()

srcerer
  • 131

3 Answers3

4

If you have bash - https://stackoverflow.com/questions/13828544/atomic-create-file-if-not-exists-from-bash-script

set -o noclobber
{ > file ; } &> /dev/null

This command creates a file named file if there's no existent file named file. If there's a file named file, then do nothing (but return a non-zero return code).

I.e. create an empty file first using this technique. If that succeeds, you can then overwrite the empty file.

Similarly for python. Use os.open() to create an empty file, making sure to include O_EXCL in the flags. ("For a description of the flag and mode values, see the C run-time documentation." See POSIX standard / Linux man page).

The bash technique is using O_EXCL behind the scenes. There is also RENAME_NOREPLACE, but it is a relatively recent addition in Linux, and I do not think it is present on OS X.

sourcejedi
  • 50,249
  • 2
    I guess you meant do something like os.open("file", os.O_CREAT|os.O_EXCL) ? – srcerer Jan 25 '19 at 20:20
  • I have edited and linked to reference documentation for os.open(). I am being lazy here and just pointing you towards O_EXCL, didn't feel like checking through all the implications (in two different languages). If you want to accept (and/or write) an answer that provides more detail, that's fine. – sourcejedi Jan 25 '19 at 21:17
  • @srcerer the question as written is still slightly ambiguous to me, because I'm not certain that bash is not as conveniently available for whichever OS X setups you are targeting... Sorry I don't know what exactly to suggest to make the question unambiguous to me, but it's why I originally posted as a comment, saying if bash was available. If you had only needed Linux instructions I think there would be nothing to stop me upvoting; I like thinking about the general topic. – sourcejedi Jan 25 '19 at 21:42
  • @srcerer it doesn't explicitly say which shell you're using to script mv / cp. General Linux, I could assume bash is available. I was aiming for a little collaboration... actually looks like OS X provides bash, it's general BSD that you can't rely on it being there, although looking at the versions I am not sure whether it is a good plan on OS X... https://unix.stackexchange.com/questions/82244/bash-in-linux-v-s-mac-os – sourcejedi Jan 25 '19 at 22:18
2

If the files are on the same filesystem, then you can create a hardlink.

ln SRC DEST

If that succeeds, you can then remove the source file.

rm SRC
None
  • 167
0

FreeBSD does indeed return failure if cp -n is asked to overwrite a file:

$ rm -f foo.*
$ date > foo.1
$ date > foo.2
$ # this should fail
$ cp -n foo.1 foo.2 || echo fail
fail
$ rm foo.2
$ # this should succeed
$ cp -n foo.1 foo.2 || echo fail
$ exit

You are correct when you state that FreeBSD's mv returns success even when the destination exists:

$ rm -f foo.*
$ date > foo.1
$ date > foo.2
$ # this should fail
$ mv -n foo.1 foo.2 || echo fail
$ exit

One workaround is to && the mv result code with [ ! -f src-file ], as in:

$ rm -f foo.*
$ date > foo.1
$ date > foo.2
$ # this should fail
$ ( mv -n foo.1 foo.2 && [ ! -f foo.1 ] ) || echo fail
fail
$ rm foo.2
$ # this should succeed
$ ( mv -n foo.1 foo.2 && [ ! -f foo.1 ] ) || echo fail
$ exit

Under GNU, neither utility performs the way you would like. The same workaround for mv works for me on Ubuntu, so that leaves GNU cp as as the one remaining problem case.

Just as an aside, I hear your comment about the race condition with testing for the destination file before calling cp, but it strikes me that the same race condition would be present even if cp did the right thing. The window of opportunity might be smaller, but my intuition is that it would still be there. IANAE, however.

Since the workaround for mv works on both platforms, perhaps this workaround will suffice:

$ rm -f foo.*
$ date > foo.1
$ date > foo.2
$ # this should fail
$ ( cp -n foo.1 TEMPFILE && mv -n TEMPFILE foo.2 && [ ! -f TEMPFILE ] ) || echo fail
fail
$ rm -f TEMPFILE
$ rm foo.2
$ # this should succeed
$ ( cp -n foo.1 TEMPFILE && mv -n TEMPFILE foo.2 && [ ! -f TEMPFILE ] ) || echo fail
$ rm -f TEMPFILE
$ exit
Jim L.
  • 7,997
  • 1
  • 13
  • 27