23

Today the /tmp directory filled up on a machine at work. The problem was, it was on the root partition which wasn't very big. In order to fix this, a co-worker created a /new/tmp directory elsewhere, copied all the contents to the new directory, removed the original /tmp and made a symlink /tmp -> /new/tmp.

When he copied the files (really, this was somebody else, not me!) he didn't use -a so the owner of every file under /new/tmp was root. Furthermore, he didn't set the permissions of the /new/tmp directory so it was the default 0755. This caused no end of trouble and even tweaking mode and ownership bits failed to restore the machine to an acceptably working state. I ended up having to nuke everything in /tmp and reboot.

The /tmp directory contained various sockets and pipes and whatnot, since a bunch of people run Gnome through VNC, and I use screen which has its own pipes.

Is there a safe way to move a /tmp directory to a different volume on a running system? I'm not sure what I would have actually done to keep everything working. I'm particularly curious about what happens to pipes and sockets.

Greg Hewgill
  • 7,053
  • 2
  • 32
  • 34
  • No idea if this is "safe" but I was able to move the tmp folder in a live, running system by doing mv /tmp /tmp.bak and rsyncing /tmp to the new destination, and then symlinking /tmp to the new destination. Everything kept running smoothly, existing tmp files kept being accessed (from what I could tell) and new tmp files started being created in /tmp (which symlinks to the new volume/destination). The plan is to delete /tmp.bak after rebooting the server during a proper maintenance window. – Joshua Pinter Oct 05 '20 at 15:06

1 Answers1

24

On “client” machines, the safe way to move /tmp is to reboot. Here, by client, I mean anything that runs programs that put sockets in /tmp, in particular X servers and screen.

The new /tmp definitely needs to have the right permissions (1777), otherwise you can't hope to have a working system.

For /tmp, you pretty much can't copy any files. That's because most of the time, programs that put stuff in /tmp open the files. If you copy the file, that copies the contents, but the programs still have the old files open. You might be able to reach into them with a debugger (ptrace), but this will be a lot more complicated than rebooting, and with many programs all you'd do is crash them anyway.

If your /tmp is full and you want to switch to a new one live, you need to restart all programs that have files open there. Since that means restarting X and screen sessions, it's not much better than rebooting.

You should be able to switch for new programs but keep existing open files in place by using a union mount. (The principle is sound, but I've never tried, so there may be by-me-unexpected issues.) Here's a way to do this on Linux.

  1. Keep all existing files in /tmp except for a few manually-selected big ones.
  2. Create a /tmp.new (mode 1777).
  3. Expose /tmp on a different path: mount --bind / /.root.only. This is necessary because the next step will shadow /tmp. There may be different union mount implementations that don't require this step.
  4. Make a union mount of /.root.only/tmp and /tmp.new, mounted on /tmp. This way new files created in /tmp will be written in /tmp.new, but files in /.root.only/tmp are also visible under /tmp. One possibility is unionfs-fuse: unionfs-fuse /tmp.new:/.root.only/tmp /tmp.

If you don't want to go the union mount root (e.g. because it's not available on your platform, or because it's too much trouble), at least do not delete the old directory. Move it, so that running programs will keep using the old directory and new programs will use the new one. (Of course new programs won't be able to communicate with old programs through sockets or pipe in /tmp unless you set TMPDIR or otherwise tell them where to look.)

mv /tmp /tmp.old && mkdir /tmp
Gilles 'SO- stop being evil'
  • 829,060