2

I've been doing several chrooted environments in parallel, each in separate folder (when the script with chroot exited I sometimes run another in same folder), and for a while it seemed fine. But now I have almost empty /dev of "main" system (cannot start apps/open new windows, etc.).

I've been mounting like this:

sudo mount -t proc proc $work_path/fin_sq/proc
sudo mount -t sysfs sys $work_path/fin_sq/sys
sudo mount -t devtmpfs devtmpfs $work_path/fin_sq/dev
sudo mount -t devpts devpts $work_path/fin_sq/dev/pts

and unmounting after exiting chroot:

umount "${work_path}"/fin_sq/dev/pts
umount "${work_path}"/fin_sq/dev
umount "${work_path}"/fin_sq/proc
umount "${work_path}"/fin_sq/sys

Occasionally I've interrupted script running in chrooted environment via ctrl-c. For cases when the script run in a folder where there were remnants of previous run, the script is coded to umount and delete previous work.

Today I saw target is busy at least in one terminal window where I run script with chroot. After umount failed the script deleted all folders of chrooted system. For the future, I think I better cancel further unmounts/deletions if one of umounts fails.

However, what do you think might cause such result? How to investigate? How to get system to working condition? I'm not rebooting the system for a while hoping to learn Linux deeper. TIA

P.S. full script(s) for those interested: script running other script via chroot, searching for /dev/pts finds two lines (mount/umount) in the code.

"Naturally" lsof shows empty output and adding --force seems to make no difference. Trying mount -t devpts devpts /dev resulted in "devtmpfs already mounted on one_of_chrooted_paths/dev".

I've read umount: target is busy, but the question is for -o rbind, I mount w/out it. Again, it was working with two chrooted, I increased to 3-4 and something broke...

Alex Martian
  • 1,035

1 Answers1

0

Note: this answer is how I solved the issue, answers on technical details how it possibly had happened are welcome.

I've found with ps aux couple of sudo chroot .../work5/ processes with T and Ss status respectively.

TL:DR

Also several other processes which I've found once I run lsof for another subfolder in chroot I saw still mounted in findmnt (not "${work_path}"/fin_sq/dev/pts which I checked earlier). After those were terminated (not easily, I've posted another related question: linux: kill -9 process success only on second attempt), I was able to unmount problematic mount points.

After that I had damaged /dev, in particular /dev/null was not special character, I fixed that finding googling my own answer in https://superuser.com/a/1767579/607929:

# mknod -m 666 /dev/null c 1 3 
# mknod -m 666 /dev/ptmx c 5 2

All seems to work, particulars of how exactly issue started will be INO even harder to find. Looks like I've cleared terminal output of previous run of script for chroot before running next. Ctrl-c was almost certainly the root cause. I wonder why it interrupted before many times w/out issues but recently resulted in such "mess".

Alex Martian
  • 1,035