A virtual file containing the concatenation of other files

Question

Is there a way of creating a filesystem object akin to this:

mknod files p
cat file1 file2 ... fileN > files

but such that it can be seeked in, as if it were a regular file?

Answer 1

On Linux-based operating systems, that can be done with network block devices or device mapper devices. The file you obtain is a block device.

With nbd:

ln -s /path/to/first-file file.0
...
ln -s /path/to/last-file file.19
nbd-server -C /dev/null -m 127.0.0.1:12345 file

sudo nbd-client localhost 12345 /dev/nbd0

(the concatenation is /dev/nbd0).

With device mapper (file sizes have to be multiple of 512):

sudo losetup /dev/loop0 file1
sudo losetup /dev/loop1 file2
s0=$(sudo blockdev --getsize /dev/loop0)
s1=$(sudo blockdev --getsize /dev/loop1)
printf '%s\n' "0 $s0 linear /dev/loop0 0" "$s0 $s1 linear /dev/loop1 0" |
  sudo dmsetup create mybundle

(the concatenation is /dev/mapper/mybundle).

Answer 2

wrote a fuse driver today, if someone is interested in the fuse solution (the device mapper as well as the nbd-solution above will create block devices not regular files - which will break, if you want to use the resulting output directly with video editing software or other tools which are not prepared to read from a block device directly)

https://github.com/schlaile/concatfs

Answer 3

You basically answered it in the first sentence of the question: yes, it can be done. However, you'd have to write custom file system driver. If it is supposed to be a file system object, it has to be handled by the kernel on some level (that includes FUSE). The driver would have to provide backend for the standard file system syscalls API (stat(), open() etc., including seeking). You can't do it entirely in user space (at least not with a monolithic kernel - but even with a microkernel you'd still need to provide a filesystem driver, albeit running as a regular userspace process).