Rename command seemed to remove a bunch of files. Can I recover them?

Question

Using Ubuntu 22.04

I had a folder with a bunch of files I wanted to change the file extension so I used:

rename 's/\.zip$/\.cbz/' *.zip

The command completed without throwing any errors, but I found about 2/3 of my files were missing after. I think it had to do with the names of some the files containing brackets or quotes but I know some missing files had very simple names (comic5.zip) and I see that every file who's name started with a bracket is now missing.

I tried using photorec to recover the files but it couldn't find a single one of them.

The folder size is a lot smaller now which seems to indicate they got deleted but if that were the case I would've thought photorec would be able to recover at least some of them. Just using my file explorer, the folder was ~120GB before running the command and ~40GB after. Unfortunately I didn't look at the actual number of files before the command, but I could also tell just by scrolling through that there are much fewer files.

Does anyone know what happened to the files and if they can be recovered?

$ rename -V
/usr/bin/rename using File::Rename version 1.30, File::Rename::Options version 1.10

Update:

Turns out I forgot to change some of my previous settings in photorec, I ran it again and it did find the missing files, pretty sure all files are recovered now. I think that confirms the files were in fact deleted. Still not exactly sure how it happened but I think Stéphane's explanation is the most likely.

In any case, thanks for the help.

Answer 1

Consider the command that you ran:

rename 's/\.zip$/\.cbz/' *.zip

This changes the last four characters of every file or directory^* name matching the glob pattern *.zip in the current directory from .zip to .cbz. That's all. It uses a Perl substitution expression to perform this rename.

Furthermore, at least on my system, rename will refuse to rename an item (file) where the target name already exists:

touch a.zip a.zap
rename 's/\.zip$/.zap/' *.zip
a.zip not renamed: a.zap already exists

I found about 2/3 of my files were missing after

I cannot see anything reasonable⁺ in the scenario that you have described that would cause files (or directories, etc.) to disappear. Perhaps there was another command such as mv, that does not perform sanity checks before overwrite. Check your history to see what that shows you

---

^{* Actually it'll rename any type of item, whether regular file, directory, device, pipe... but only where it matches the criteria I've described}

^{+ Files starting with a dash (-) or double dash (--) are always problematic. Here too unless you protect your file patterns by separating command arguments and file arguments with a standalone --, or by using a modified ./name format instead of plain name. For example mv -- -a dasha to rename -a to dash.}

Answer 2

$ touch {comic5,{a..d}}.zip './--e=BEGIN{unlink for <c*>}#.zip'
$ ls
 a.zip   b.zip   comic5.zip   c.zip   d.zip  '--e=BEGIN{unlink for <c*>}#.zip'

$ rename 's/\.zip$/\.cbz/' *.zip
$ ls
 a.zip   b.zip   d.zip  '--e=BEGIN{unlink for <c*>}#.zip'

Oops! All the files whose name starts with c have been removed.

For the File::Rename implementation of rename, this kind of issue was fixed in 1.99 (on my request) but Ubuntu 22.04 comes with version 1.30.

Here, you'd want instead:

rename 's/\.zip\z/\.cbz/' ./*.zip

With that ./ prefix preventing any of the zip file names from being taken as an option by rename¹.

Or, with that particular implementation of rename, pass the file list via its standard input instead of arguments:

printf '%s\0' *.zip | rename -0 's/\.zip/\.cbz/'

Which with shells where printf is builtin (most) also works around the Argument list too long limit.

Or use zsh's zmv instead which doesn't have this kind of problem, has some extra safeguards and is generally easier to use²:

autoload -Uz zmv
zmv '(*).zip' '$1.cbz'

Now, this kind of thing is unlikely to happen by chance. If that's what happened, someone would have planted a malicious similarly named file in the current directory with the knowledge that you would run such a command.

If the malicious code did unlink() or if the files were removed as the result of a rename() replacing the file, you might have some luck using some filesystem undelete/recovery approach. If it did the equivalent of what shred/wipe do or encrypted the contents before moving the files elsewhere as ransomware would do, your best option would be restoring from backup.

In the absence of malicious file names, rename should not normally lose data as it checks whether the target file exists before renaming. You might lose symlink files though if the target of those symlinks are not accessible as rename uses -e <the-target> and that would return false for a symlink that doesn't resolve like for the -e of the [ utility:

$ touch a.zip
$ ln -s /x a.bcz
$ ls -n
total 4
lrwxrwxrwx 1 1000 1000 2 Dec 22 08:39 a.bcz -> /x
-rw-r--r-- 1 1000 1000 0 Dec 22 08:39 a.zip

$ rename -v 's/\.zip$/\.bcz/' *.zip
a.zip renamed as a.bcz
$ ls -n
total 0
-rw-r--r-- 1 1000 1000 0 Dec 22 08:39 a.bcz

Oops! That (broken) a.bcz symlink is gone.

But symlinks take very little space on disk, so that couldn't explain the 80 out of 120GiB loss that you suffered from.

Like Chris, I suspect your problem was elsewhere. Your files may even still be there, only not in the directory you think they should be. Try a sudo find / -iname 'comic5.zip' -o -iname 'comic5.cbz'

---

^{¹ Also note the \z in place of $ which you generally want to use in regexps that match things other than lines as $ matches either at the end of the subject (which you want here) or before a newline character at the end of the subject (which you don't want though in this particular case, the subject ends in .zip and not newline, so that wouldn't be a problem).}

^{² even if generally slower as it calls one mv per file. Though you can run zmodload zsh/files to get a builtin mv.}