144

Recently I accidentally did rm on a set of files and it got me thinking where exactly these files end up?

That is to say, when working with a GUI, deleted files go to the Trash. What's the equivalent for rm and is there a way of undoing an rm command?

Gilles 'SO- stop being evil'
  • 829,060
boehj
  • 2,630

6 Answers6

161

Nowhere, gone, vanished. Well, more specifically, the file gets unlinked. The data is still sitting there on disk, but the link to it is removed. It used to be possible to retrieve the data, but nowadays the metadata is cleared and nothing's recoverable.

There is no Trash can for rm, nor should there be. If you need a Trash can, you should use a higher-level interface. There is a command-line utility in trash-cli on Ubuntu, but most of the time GUI file managers like Nautilus or Dolphin are used to provide a standard Trash can. The Trash can is standard itself. Files trashed in Dolphin will be visible in the Trash from Nautilus.

Files are usually moved to somewhere like ~/.local/share/Trash/files/ when trashed. The rm command on UNIX/Linux is comparable to del on DOS/Windows which also deletes and does not move files to the Recycle Bin. Another thing to realize is that moving a file across filesystems like to your USB disk from your hard disk drive is really 1) a copy of the file data followed by 2) unlinking the original file. You wouldn't want your Trash to be filled up with these extra copies.

jesse_b
  • 37,005
penguin359
  • 12,077
  • 4
    Thanks very much for the clear explanation. I don't mind using the CLI, I just need to be a little more careful when using wildcards. :) – boehj Apr 08 '11 at 04:42
  • 1
    Well, there is a Trash can for rm if you use libtrash. – cjm Apr 08 '11 at 04:45
  • 22
    I'd be cautious with using something like libtrash to change the behavior of rm. Lots of scripts use rm to clean up files and you don't want those showing up in Trash. I recommend using a dedicated command like trash from the trash-cli package. @pedro I should add that I once created a file * in my home directory. I had accidentally quoted * when I shouldn't creating it so I decided to remove it with with rm * natually. When I realized what I did I quickly killed the command, but it had already deleted a number of files in my home directory. – penguin359 Apr 08 '11 at 05:13
  • 5
    It is very rare to have a something like a trash can in the shell, so if you add it on you local machine and get used to it or even depend on it in your daily work. You can get in trouble when using some of the other 99% of the unix:s without one... – Johan Apr 08 '11 at 07:04
  • 3
    I think the take-home message here is that I need to stop CLI'ing in the wee hours and pay better attention. – boehj Apr 08 '11 at 08:06
  • 3
    Along the same lines as what @Johan said, RedHat used to (still does?) set aliases for commands like cp, and mv, to cp -i and mv -i when running as root. This changes the default behavior so those commands will always ask before overwriting existing files. Some sysadmins recommended to specifically remove those aliases so you don't end up expecting that behavior which may end up being lethal when on another system that follow default behavior. – penguin359 Apr 08 '11 at 08:13
  • 2
    Which brings up another thought. I've been used to the behavior of not having a Trash/Recycle Bin for many years now and therefore, I don't have any tendencies to rely on one. If I have any concerns about having to retrieve a file, I will make a copy or move the files out of the way first. And maybe, once or twice a year, I might have to retrieve the file from my backups. – penguin359 Apr 08 '11 at 08:17
  • As I've said elsewhere, use distributed version control for everything you can. Of course it is not practical to do this for everything, but you can at least do it for text files. – Faheem Mitha Apr 08 '11 at 09:32
  • 1
    @penguin359: I've now aliased cp to cp -i. Can't look back. Wonder how I survived w/o it in fact. It'll certainly be one of my bash aliases from now on on every install. – boehj Apr 24 '11 at 10:39
  • One query, if the file is getting only unlinked then how the space is getting reduced if we delete a file – upkar Jan 05 '17 at 04:40
  • 1
    -1 for "nor should there be." If you want a system with no safety net feel free to have one. Don't make that choice for others and demand they accept it. – Jay May 03 '20 at 16:06
  • 1
    @Jay The rm command is a low-level command from early in UNIX history decades before Linux came along. It's used in many non-user contexts and would break certain assumptions in the code that uses it if it were changed. It is similar to the del command still present in modern day Windows 10 command prompt and which also delete files without going through the recycle bin first. If you want a higher-level CLI interface with does provide a safety net, no problem, that's what trash-cli is for, aimed at the user as I mentioned in my answer. – penguin359 May 06 '20 at 00:10
  • 1
    @penguin359 There is no valid excuse for this to persist as a user accessible command defaulting to this behavior. Moving files to a trash bin is not a breaking change. I down voted for your comment which seems elitist and/or passive/aggressive. Which you then try to defend and excuse. The visitors need help not attitude. – Jay May 06 '20 at 04:54
  • While the information is correct, there's also a lot of opinion baked into this answer. -- Guys, there's a time and place for everything. -- I'm not sure that adding "--trash" flag to the rm command (or rd and del on windows) would be completely remiss. -- Some situations (especially scripted ones) might benefit from such being available to the scripter. (... and I'm also not sure it shouldn't be the default.) – BrainSlugs83 Dec 15 '20 at 20:14
  • @upkar the "free space" on your disk isn't counting the literal unused bytes, it's counting the bytes that are linked vs the total capacity. -- Hence, that space is free to be reused (regardless of the garbage data that may be contained in those sectors). – BrainSlugs83 Dec 15 '20 at 20:17
15

For ext3/ext4, you can try recovering files using tools like extundelete or ext3grep, or even go messing with the low-level structures manually (not for the faint of heart); for many filesystems, you can try to search for the not-yet-overwritten blocks by certain patterns (e.g. magicrescue can search for JPEG headers, amongst other things). Note that these are using heuristics to recover the files from the metadata left behind, so full recovery is not guaranteed - it's more of a last-chance bet (as those require that some traces of the files remain in the journal, and that the blocks weren't overwritten yet).

So, for all intents and purposes, files removed with rm are gone - you could try such necromancy as these tools offer, but don't depend on it: these are the tools to try when everything else fails. Better dig out your latest backups (you have been making backups, right? Oh well, live and learn...).

Piskvor left the building
  • 682
  • 2
    For high-value text data, you can always use any robust general-purpose tool (even emacs or perl) to look at the "raw device" for the disk that contained the removed file, and search for known strings; I've recovered Word documents for people this way; they lose the mark-up, but can recover most of the text. Obviously this is disaster recovery, not "Undo". – alexis Dec 07 '13 at 14:05
  • A proper 'manually' link: https://web.archive.org/web/20131221183925/http://carlo17.home.xs4all.nl/howto/undelete_ext3.html – sjas Jan 29 '15 at 22:56
  • I've used photorec several times with success on SD cards that'd been wiped out by mistake to recover photos. But it might work better because 1) it's an easier filesystem (FAT32 or EXFAT) 2) other processes (or worse the system itself) aren't accessing it at the same time. – dargaud Jan 25 '24 at 16:18
9

Regarding undoing the effects of rm:

Given that most filesystems only remove the reference to the data and indicate that the blocks as free, you could try to locate your data reading directly from the device. With a bit of luck the blocks containing your file(s) haven't been claimed for something else.

This assumes you have something fairly unique to look for, that you have root on the system and I'm guessing piecing together anything that spans more than one file system block (probably 4k) might end up quite laborious if the file system didn't manage to put the file(s) in contiguous blocks.

I have successfully recovered the contents of a couple of plain-text files by running strings on the device the file system was on, and using grep looking for something from those files with a large context (-C). (And shortly after that incident, the company decided to spend some resources on implementing backups)

erch
  • 5,030
Kjetil Jorgensen
  • 1,244
  • 7
  • 7
  • It's slightly more complicated e.g. by ext3 zeroing out the block pointers in the inode, but yes, looking for the files directly might work - if they're small enough or allocated in a contiguous block. This is sometimes called file carving and there are tools like magicrescue that try to find images or sounds by their distinctive patterns. – Piskvor left the building Apr 08 '11 at 20:48
7

Whenever you delete a file using rm command, the file's data is never deleted. In other words the blocks in the file system containing data is still there.

What happens is when you run the rm command, the system marks the inode belonging to that file as unused and the data blocks of that file also as unused (but not wiped out). However ext3 zeros most of the fields in the inode, when a file is deleted.

This normal marking of unused is done for the speed... Otherwise for deletion it will take some more time. That's why you might have noted deleting even large files are faster (you can recover the data if that data blocks are not overwritten).

More Info: Inode Structure, How file deletion works

jasonwryan
  • 73,126
sarath
  • 71
  • 1
    ...unless the file is explicitly marked with chattr +s ("shred") attribute. It tells the filesystem to specifically overwrite this file with zeroes on deletion. Only some filesystems will support that attribute. – telcoM Feb 04 '18 at 21:29
5

In Unix-style filesystems (including on Linux), files are not really "at" any particular place. Instead, the system uses hardlinks to point into pieces of what amounts to a big blob of data. So when you create a file, you also create its first hardlink: the one which actually resides at the place where you "saved" the file. If you make more hardlinks, then as far as the system knows, the file actually exists in several places at once.

When you "delete" a file, normally you're actually only deleting the hardlink that existed at the place you specified. This is why the system call to delete files is called unlink(). The system won't actually delete the file until there are no hardlinks left to it. But once that last hardlink is destroyed, so is the data.

So, where do files you delete go? If there are still hardlinks, they files are wherever the hardlinks you didn't delete are. If there are no hardlinks left, the files are gone.

The Spooniest
  • 361
0

Look also into ~/.snapshot if the file was recently removed.

Andy
  • 1
  • 5
    This will only work if you have some magic filesystem that provides that feature (like a NetApp) or if you're using a special version of rm. – mattdm Mar 08 '12 at 03:18
  • This only works on filesystems like BTRFS with a specific tool. – 0xLogN Mar 31 '22 at 02:36