1

I'm currently playing with building disk images (based on Debian) for a raspberry-like computer.

My current workflow looks like:

  1. create disk image (about 16GB uncompressed; or 3GB compressed)
  2. power-down the target system
  3. eject the SDcard on the target system
  4. insert the SDcard into the build computer
  5. dd the image onto the SDcard
  6. eject the SDcard from the build computer
  7. insert the SDcard in the target system
  8. powerup the target system into the new OS version

this is tedious.

I would like to shorten the cycle to:

  1. create disk image (about 16GB uncompressed; or 3GB compressed)
  2. scp the disk-image to the target system
  3. overwrite the SDcard of the target system with the new image with dd
  4. reboot the target system into the new OS version

Now this obviously has the problem that I'm trying to overwrite a mounted partition containing the rootfs (the disk-image contains only a root partition and a boot partition). Eventually, dd (which I use to copy the image to the SDcard) will segfault, and after that the filesystem is broken beyond repair and nothing works anymore.

So my question is: is there a way to do a low-level data "restore" on the current rootfs? Something like: copy some "statically linked" versions of the relevant binaries (dd, reboot) to some safe space (a RAM disk), and run from there.

Ideally I would like to do this without physical access to the target system (provided that my disk image is not borked).

terdon
  • 242,166
umläute
  • 6,472

3 Answers3

1

If you are doing this a lot, to the same system. Then, consider using network boot. You would install once a bootloader on the target system. The target would then on every re-boot load and run the current OS from a network server.

Alternatively

Writing to the root partition is a bad idea. However you could build a new minimal root, and pivot root, to the new root.

ctrl-alt-delor
  • 27,993
1

Brian's answer is what I'd recommend: receive the image from the network as you go. There's no other way, because you clearly can't put the image on the storage device you're overwriting.

But that exact problem will also become with Brian's approach: Assuming you did the image receiving from your currently booted full debian, then you'll start overwriting the software that's currently on the file system. Uh, what happens if some hardware interrupt, some timer / cron job, or some file system maintenance tool kick in while you're downloading and extracting to raw disk? Your whole system might, or might not crash.

If it crashes, you're left with an embedded device that's not bootable in a location you potentially can't access. That makes the effort much worse.

So, what you need to do is that the receiving&unpacking software doesn't overwrite what you're currently running. Two approaches:

  1. Have two root volumes, say, A and B. Your current debian be running from A, you can safely overwrite all of B. After that's finished, you can tell your boot process to boot from B, and properly reboot your system. (Typically, you have a separate "boot" partition from which your bootloader runs, and which contains the boot configuration files for the RPi's boot hardware, which is… peculiar.)
    You'd need twice your system's size needs in storage space.
    I drafted a version how you'd do that, exactly on a RPi, in this very recent answer
  2. Have one root volume (A) that you overwrite, but don't do it while you're running from that volume. Instead, add an entry to your bootloader, and an initial RAM file system file to your boot drive, which boots from a couple-of-megabytes image which gets loaded into RAM, and does the receiving and unpacking to A from there. Or, you have another (C) small volume that contains a minimal version of your operating system entailing very little but networking setup, SSH and a decompressor.

Looking at Raspberry Pi's foremost hardware failure mode being SD cards stopping to work due to write fatigue, and looking at how low SD card prices are, I'd probably go with the first approach (A/B booting, no separate "maintenance" OS), and if the new system works flawlessly, I'd blkdiscard the now unused volume, so that the SD card's wear leveling has space to work with.

So, to conclude, assuming you're going with Method 1.:

  • get a 64 GB card if your system needs a bit less than 32 GB of space
  • Install debian on that system; but modify the default partitioning scheme a bit:
    • use "Guided - use entire disk and set up LVM" when asked to Partition Disks, and either "All files in one partition" or "Separate /home partition" on the "Partitioning Scheme" dialog, depending on whether you want the files in your home directory on the pi to survive an upgrade or not (you can change that later, as well, no big deal).
    • make sure that the sum of sizes in the overview of volumes to be created is a bit less than half the SD card size (for simplicity, 30 GB). If necessary, reduce the size of / (and /home if present) accordingly.
    • write the changes to disk, finish installation, boot RPi: should work.
    • (Of course, you can also achieve this by reducing the size of a partition of an existing installation, with some caveats, and trouble later on; I'd recommend you start from a clean LVM installation, you'll not regret that. I honestly don't know why debian doesn't default to LVM in 2023, but prefers to use a partitioning scheme that was specific to a specific brand of personal computer with a 8 bit CPU @ 5MHz with 16 kB of RAM, literally from 1983, forty years ago)

So, after your new system runs (and before you set up anything further), we'll describe the update process:

  • On the Pi:
    • Check whether there's an oldsystem logical volume (Look into /dev/mapper)
      • if so, delete it, sudo lvremove your-volume-group-name/oldsystem
    • On the Pi: Make a new volume of your uncompressed image size (assuming 30 GB here): sudo lvcreate -L 30G --name image_prep your-volume-group-name
  • On the end with the image you want to send:
    • Compress the image, using something like zstd -12 -T0 < /what/you/want/to/compress > /path/to/images/filesystem.image.zst (-12 is already quite high compression, and thus not very fast, but it saves you time on the network, so there's that; values between 1 and 18 work, and .)
    • take the checksum of your compressed image: sha256sum /path/to/images/filesystem.image.zst >> checksums
    • Get the image from the hosting computer to the receiving one; I'm lazy, I usually just set up a small python webserver (because copying through SSH will make the poor RPi's CPU a bit warm): cd /path/to/images; python -m http.server -p HTTP/1.1 8765 (if necessary, I'll open port 8765; note down this computer's IP address)
  • On the Pi:
    • get the image, uncompress it to the new volume, and simultaneous calculate the checksum: mkfifo checksumpipe; sha256sum > checksum < checksumpipe &
      curl http://ip.address.as.noted:8765/filesystem.image.zst | tee checksumpipe | zstd -d > /dev/your-volume-group-name/image_prep
    • verify that the checksum in checksum is the same as you saved in checksums on the transmitting host
    • figure out what your current root volume is called (if in doubt, cat /proc/cmdline, look for root=/dev/mapper/your--volume--group--name-???), and rename it to oldsystem: sudo lvrename your-volume-group-name ??? oldsystem
    • rename the freshly imaged volume to the former name sudo lvrename your-volume-group-name image_prep ???
    • mount that volume, bind-mount /dev/, /proc /sys, /boot into it, chroot into the mount and run a boot loader update
    • reboot

A less "debian" way would that's a lot more streamline probably would be using mender; there's actually an official guide for RPi; ignore the part about registering for "hosted mender", you're looking for "standalone" mode.

umläute
  • 6,472
Marcus Müller
  • 28,836
0

You can image over the network.

On source, dd if=/dev/hda bs=16065b | netcat < targethost-IP > 1234

On target, netcat -l -p 1234 | dd of=/dev/hdc bs=16065b

Hit enter on target first, then source. Dd will run from the memory until the job is done. Programs don't run from the storage drive. Substitute sensible partitions, and block size (bs) of your choice. If your imaging /boot and /, do /boot first.

Brian
  • 158
  • For this to safely work, you will want to run your receiving end from a ramdisk, not based on hope that nothing will cause anything that will try to read from a disappeared piexe of storage. So, generally, yes, but not without further ado. – Marcus Müller Sep 20 '23 at 07:54
  • also, pretty certain that this blocksize isn't a great idea considering it will not fit in any network packets; this probably is an unnecessary (and potentially counter-productive) use of dd; on source, do netcat target-ip 1234 < /dev/sda (hda would be for IDE drives. If you remember what these are, you're as old as me :-) ) and on target, netcat -l 1234 > /dev/sdc. But again, the problem with this is not the inadequate use of dd, but that you're making the wrong assumption that your system will not crash when you overwrite its root device while running. – Marcus Müller Sep 20 '23 at 09:42
  • this worked insofar as i have been able to write the entire image without segfaulting while doing so (i ended up using netcat from busybox-static's, which was copied onto /dev/shm to make sure it did not get overwritten while i copied. as expected by the critics, the resulting image was borked, as obviously some process also wrote the the harddisk while i was... – umläute Sep 20 '23 at 13:21
  • DD can use a different block size than a packet. DD doesn't control the networking. When doing this, you want to be in run level 1. DD is in memory, so you can overwrite the root partition. It won't stop DD. DD is a bitsteam. It has robust error checking. Cat moves bytes, and has no error checking. 'The parameters used with DD are up to the user. I provided a template.. If one is really worried about overwriting root, boot knoppix and run the command on it. – Brian Sep 22 '23 at 19:46