9

The following script sometimes gives me things like can't remove a/b as it's not empty

ssh -T user@host <<EOF
  cd somewhere
  rm -rf a/b    
EOF

But I log in to the server and execute rm -rf a/b then I never had the problem.

There are other processes that are generating files to a/b, is that related?

Anyway, how to make a script that will ensure "a/b" will be deleted?

Cheng
  • 6,641
  • 1
    That's funny, rm -rf should always succeed, short of permission issues. Ask the script to print debugging output if the removal fails.ls -laR a/b should suffice. – Faheem Mitha May 04 '11 at 16:41

3 Answers3

16

There are other processes that are generating files to a/b, is that related?

Very possibly. It might be that rm -rf first removes all files, then removes all directories. Under the hood, the rmdir system call for removing a directory will fail if the directory is not empty. You might have a race condition going on, where rm -rf checks that the directory is empty, and it is; another process then creates a new file; finally rm -rf calls rmdir, but another process creates a file beforehand.

Faheem's advice is good: put an ls into your error condition

ssh -T user@host <<EOF
  cd somewhere
  rm -rf a/b || ( ls -a a/b && exit 1 )
EOF
jmtd
  • 9,305
  • Very interesting. – boehj May 05 '11 at 04:01
  • The && should be ; or else ls failure will prevent the script from exiting. – R.. GitHub STOP HELPING ICE May 07 '11 at 02:06
  • Yep. ls shows that there are new files in that dir. I'm going to give the mv solution a try. – Cheng May 07 '11 at 13:08
  • @R, that is not true. If ls fails, the subshell will return ls's failure code. the '&& exit 1' is necessary to ensure that if ls succeeds, the sub-shell still returns an error code. If you substitute it for '||', you will mask the true error code for ls. – jmtd May 09 '11 at 13:44
9

rmdir(2) will fail if the directory is not empty. If another process is creating files while rm(1) is removing them, it will not know to delete them and consequently when it comes time for rm(1) to try to delete what it believes should be an empty directory, it will fail with the error you've posted.

One way to delete the directory in the face of concurrent file creations in the directory is to rename it:

mv a a~
rm -rf a~

It's possible that this may not work if the processes creating the files in a/b are not doing so by path (open(2) vs. openat(2)).

I am assuming that the process(es) that creates files in a/b will recreate that directory if it does not exist, or will handle failure gracefully if it does not exist. Since you are already trying to delete the directory from under other processes, that seems like a safe assumption.

camh
  • 39,069
  • thanks, glusterfs is healing directory and cannot heal it normally by some bug in glusterfs, so mv a directory to diferent name and rm is fixing it. – jmp Apr 16 '19 at 09:09
0

In case of very high race condition: next command removes outdated files and then removes empty directories:

find /somedir -type f -atime +3 -print0 | xargs -0 --no-run-if-empty rm -f; find /somedir -mindepth 1 -type d -empty -not -name "*.empty" -print0 | xargs -0 --no-run-if-empty -I{} mv {} {}.empty; sleep 30; find /somedir -type d -name '*.empty' -print0 | xargs -0 --no-run-if-empty rm -rf

Or step by step:

find /somedir -type f -atime +3 -print0 | xargs -0 --no-run-if-empty rm -f;
find /somedir -mindepth 1 -type d -empty -not -name "*.empty" -print0 | xargs -0 --no-run-if-empty -I{} mv {} {}.empty;
sleep 30;
find /somedir -type d -name '*.empty' -print0 | xargs -0 --no-run-if-empty rm -rf;

sleep 30 is necessary to all processes to finish writing into the removing direcory.

WARNING: you can lose your new data (in my case it is cache so I don't care).

MingalevME
  • 101