Generating a .img from tarball: The /dev/loop devices

Question

I'm trying to make a .img from a tarball using the following script I found:

#!/bin/bash

# Packages required
# dosfstools parted 
# Can be run on any Linux system
# loop.max_part=15 must be in kernel cmdline (cmdline.txt for rpi)
# then reboot

echo "creating image to fit on 2Gb card"
dd if=/dev/zero of=arch-rpi.img bs=1M count=1850

echo "Partitioning"
fdisk arch-rpi.img <<EOF
o
n
p
1

+100M

t
c
n
p
2


w
EOF

sleep 5

losetup -f arch-rpi.img
sleep 5

echo "Formatting vfat"
mkfs.vfat /dev/loop0p1
sleep 5
mkdir boot
echo "Mounting boot"
mount /dev/loop0p1 boot
echo "Installing"

echo "Formatting ext4"
mkfs.ext4 /dev/loop0p2
sleep 5
mkdir root
echo "Mounting root"
mount /dev/loop0p2 root

wget http://archlinuxarm.org/os/ArchLinuxARM-rpi-latest.tar.gz
echo "Installing"
bsdtar -xpf ArchLinuxARM-rpi-latest.tar.gz -C root
sync
mv root/boot/* boot
sync
umount boot root
losetup -d /dev/loop0p1
losetup -d /dev/loop0p1
losetup -d /dev/loop0
echo "All complete, image arch-rpi.img created, compressing...."
zip -9 arch-rpi.img.zip arch-rpi.img

I'm doing this on my Raspberry Pi - Raspbian Wheezy. And when it gets to the line mkfs.vfat /dev/loop0p1 It says there's no such file / directory. I've read enough on Linux to know I'm trying to mount the .img as a loop device and then use mkfs to prepare the image for the tarball, but I don't quite know why loop0p# files are not there, they are listed by fdisk -l. What do I need to to in order to get this script I found working?

Answer 1

You need to use losetup -P for creating a -Partitioned loop device, or else you need to partition the original loop device then partx -update the kernel's partition table afterward. The /dev/loop0p devices will appear after the kernel recognizes that the partition has actually been partitioned. Probably this is partly what the intent behind the sleep 5 is - but that should almost definitely be a sync - or both - instead.

Anyway, to demonstrate:

sudo sh -s <<\IN
    losetup -D                          
    fallocate "-l$((1024*1024*1024))" loop        
    printf %s\\n n '' '' '' ''  w y | gdisk loop
    sync; losetup -f loop
    lsblk /dev/loop*
IN

So the above sequence first -Detaches all current loop devices (if any) fallocates a 1GB tmp file, writes a GPT partition table to it and does a single partition on it, then syncs the filesystem and assigns it to the -first available loop device before trying to list with lsblk all available loop devices. It prints:

NAME  MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0   7:0    0   1G  0 loop 

But if I alter the losetup line to read:

losetup -fP loop

...it instead prints:

NAME      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0       7:0    0    1G  0 loop 
├─loop0p1 259:0    0 1023M  0 loop 
└─loop0p2 259:1    0 1007K  0 loop 

...because losetup first scans the backing file for a partition table. As you can see, it's not very good at it - rather than noting the extra 1M (which is almost definitely the last iteration's partition table, actually) as unallocated space it interprets it as a separate partition, but that's probably because I'm writing to a tmpfs (twice in a row) at the top of a 1GB file hole - working w/ actual data will be much more precise (and zeroing the backing file as your script does w/ dd would handle that anyway). In any case - I can freely mkfs.whatever on the real partition there and afterwards mount it. partx -u on the /dev/loop0 would achieve the same results.

Answer 2

In the loop driver, support for partitions is optional, determined by the max_part argument when the driver is loaded. The default value is 0, so the loop driver won't even look for partitions; with a nonzero value, the driver supports that many partitions. Depending on the kernel build options, the driver may either be included in the kernel, in which case you need to pass loop.max_part=… on the kernel command line at boot time, or be loaded as a module, in which case you need to pass max_part=… when the module is loaded.

On Debian wheezy, loop is a module and the max_part argument is not passed. To get partition support, unload the module and load it back with a max_part argument (you'll need to first deactivate any existing loop device with losetup -d):

if lsmod | grep -wq loop; then rmmod loop; fi
modprobe loop max_part=31

You can make this the default by adding options loop max_part=31 to /etc/modprobe.conf.

If you can't afford to unload the module (or to reboot, on a distribution where loop is built into the kernel), you can instead calculate the partition's offset manually and use the -o option to losetup. See Reading a filesystem from a whole disk image

With newer distributions, you can use losetup -P when setting up the loop device:

…
losetup -P -f arch-rpi.img
…

but the util-linux package on Debian wheezy is too old to have this option.