11

I'm building a very minimal Linux system that just consists of the kernel (v4.1-rc5) and an initramfs populated with busybox (v1.23.2). It works fine for the most part, but I observe a difference in behavior of command execution in /init whether I'm using an embedded initramfs vs. an external one.

The /init script is:

#!/bin/sh

dmesg -n 1

mount -t devtmpfs none /dev
mount -t sysfs none /sys
mount -t proc none /proc
echo "Welcome"
while true
do
    setsid cttyhack /bin/sh
done

Then I either set the CONFIG_INITRAMFS_SOURCE option in the kernel .config to the directory containing all the folders for the initramfs, or I run

find . | cpio -H newc -o | gzip > ../rootfs.cpio.gz

to build it.

When I then compile the kernel, either with or without CONFIG_INITRAMFS_SOURCE set, I end up with two variants of my system:

  1. bzImage with initramfs embedded

  2. bzImage + rootfs.cpio.gz (external initramfs)

when I now start those using qemu

qemu-system-x86_64 -enable-kvm -kernel bzImage

or

qemu-system-x86_64 -enable-kvm -kernel bzImage -initrd rootfs.cpio.gz

I get the following difference in behavior:

with version 2 (external initramfs) everything works fine, "Welcome" is displayed and I get a prompt. With version 1 however (embedded initramfs) I get the warning 

unable to open an initial console

"Welcome" is not displayed, and I get my prompt.

As far as I understand the process, those two versions of initramfs should contain the same files, since I build it (or have the kernel build it) from an identical folder.

I wonder if anyone can help me with an explanation for this behavior?

* UPDATE *

as mikeserv said in the comments, The kernel includes a minimal embedded initramfs per default. This is still present when using an external one, but gets overwritten if you embed your own. I found that contrary to the specification, this is indeed not empty, but contains a dev folder, a root folder and the /dev/console device. This device then gets used when using an external initramfs, but overwritten if you embed your own. So you have to include the /dev/console device in your initramfs source mknod -m 622 initramfs_src/dev/console c 5 1 when embedding your own.

Thanks a lot to mikeserv, frostschutz and JdeBP for helping me get my head around that!

Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
clw
  • 111
  • What are the permissions set to on /dev/console on your builtin one? I think the difference might be about who does the packing in the two cases. – mikeserv Jun 09 '15 at 17:41
  • A similar question is of course http://stackoverflow.com/questions/10437995/ . – JdeBP Jun 09 '15 at 17:57
  • @mikeserv the console device has identical permissions and ownership in both builds. – clw Jun 09 '15 at 18:39
  • @JdeBP I'm not sure if its that similar, since in both cases I boot, get a prompt and have a console device. Only that in one init executes the echo and in the other it can't. – clw Jun 09 '15 at 18:39
  • @JdeBP ok, adding the console device in /dev of the initramfs with mknod -m 622 console c 5 1 actually works in getting rid of the problem in the embedded version. That leaves me still puzzled why the behavior is different for the two cases, in that the external initramfs has its console device ready already when its supposed to execute the "echo" – clw Jun 09 '15 at 18:49
  • 1
    How could the permissions have been the same in initramfs if you didn't even have it at all? – mikeserv Jun 09 '15 at 18:54
  • @mikeserv I had the device in all the cases at the point where I get the command prompt and can check it. For some reason however, its not present for the embedded version at the point where it executes the "echo" within /init, while for the external one it is – clw Jun 09 '15 at 18:57
  • @mikeserv I think I missunderstood the question. The initramfs itself didn't have console devices in any case. /dev was populated through mount -t devtmpfs none /dev in init, which for some reason shows different results when using an embedded initramfs vs. external – clw Jun 09 '15 at 19:08
  • Well, even when you use an external initramfs the kernel still includes an internal one - it's the mount point. The 2.6 kernel build process always creates a gzipped cpio format initramfs archive and links it into the resulting kernel binary. By default, this archive is empty (consuming 134 bytes on x86). You should read that whole link. – mikeserv Jun 09 '15 at 19:08
  • @mikeserv I did read that, but thought it would have no effect since the archive is supposedly empty. So the default initramfs carries the /dev/console mountpoint that is used by the external one, but removed when I use my own embedded one? Would this have effects on populating anything else (sys, proc, other devs)? – clw Jun 09 '15 at 19:12
  • I dunno man, does it? Pull it all apart and find out. There will be differences, though. So do some cmping and diffing until you figure it out. It's only a few kb in either case - you just need to dig, dude. – mikeserv Jun 09 '15 at 19:14
  • Can you update the question with your final init file? Thanks! – sanatana Dec 03 '15 at 07:19

2 Answers2

2

Are they really identical?

The built-in one you can find in /usr/src/linux/usr/initramfs_data.cpio.gz or extract it from the bzImage as described here: https://wiki.gentoo.org/wiki/Custom_Initramfs#Salvaging

If you use that built-in one and use it as external one instead, does it work?

If it's still different, is the kernel itself identical? (compare /proc/config.gz for both)

There should be some difference. I'm not aware that the kernel cares where the initramfs came from. I'd sooner suspect qemu of using different settings when passing the -initrd parameter...

On a sidenote, your /init looks like its spawning infinite shells to me. setsid is not exec. Am I wrong?

frostschutz
  • 48,978
  • 1
    This answer seems to be all questions. – JdeBP Jun 09 '15 at 17:45
  • 1
    @JdeBP: You're not thinking fourth-dimensionally! – frostschutz Jun 09 '15 at 18:02
  • 1
    @frostschutz Thanks a lot for your reply! When I use the initramfs that the kernel builds (usr/initramfs_data.cpio.gz) as external it works fine as well! Also, when I supply the kernel that was compiled with the embedded initramfs with an external one, the warning appears, even though the external should overwrite the embedded (https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt). So its probably also not qemu -initrd but something within the kernel itself. I changed nothing other then CONFIG_INITRAMFS_SOURCE though.. – clw Jun 09 '15 at 18:28
  • @frostschutz Answering your On a sidenote, your /init looks like its spawning infinite shells to me. setsid is not exec. Am I wrong?: The loop mimics getty or similar tools, since the call the sh blocks until that shell exits. – stefanjunker Jan 29 '18 at 12:03
  • @stefanjunker and that would be fine, except setsid doesn't block at all... – frostschutz Jan 29 '18 at 12:17
2

You may also be interested in how Buildroot 2018.02 deals with this.

Whenever you use initramfs (BR2_TARGET_ROOTFS_INITRAMFS=y) or initrd (BR2_TARGET_ROOTFS_CPIO=n), it adds the following /init to your rootfs https://github.com/buildroot/buildroot/blob/2018.02/fs/cpio/init

#!/bin/sh
# devtmpfs does not get automounted for initramfs
/bin/mount -t devtmpfs devtmpfs /dev
exec 0</dev/console
exec 1>/dev/console
exec 2>/dev/console
exec /sbin/init "$@"

The copy is done by https://github.com/buildroot/buildroot/blob/2018.02/fs/cpio/cpio.mk :

# devtmpfs does not get automounted when initramfs is used.
# Add a pre-init script to mount it before running init
define ROOTFS_CPIO_ADD_INIT
    if [ ! -e $(TARGET_DIR)/init ]; then \
        $(INSTALL) -m 0755 fs/cpio/init $(TARGET_DIR)/init; \
    fi
endef

It is also useful to know that the init path is /init for initramfs, unlike /sbin/init otherwise: What can make passing init=/path/to/program to the kernel not start program as init?

Ciro Santilli OurBigBook.com
  • 18,092
  • 4
  • 117
  • 102