0

I created a .gz archive of a disk image (Ubuntu 18.04) of one of my virtual servers with the following command:

dd if=/dev/sdb | gzip -1 - | pv | ssh user@ip dd of=/home/user/sdb.gz

While unpacking the archive I ran into the following error: gzip: sdb.gz: unexpected end of file

As I tried to run the .img-file on my own unraid machine as a VM, I ended up in grub. So I knew that there was probably something wrong with the image. Using ls the most important files seem to be there.

The server had a 15GB disk, but I didn't realize a part was missing since it was compressed anyway. The resulting image file is only 4.3GB.

Since then my virtual server has been deleted by my VPS-provider. Now I've realized, that the archive is not complete as it ran into a disk size limit on the receiving machine.

As the basic file structure is available and I'd naively believe that the first 4GB contained all of the VM I'd like to recover the system. The data is not to important imho, but it seemed like something useful to learn and worth a try. My suspection is that something is wrong with the partition information and that the system can't find the correct boot path for some reason.

I tried the steps under Booting from Grub in this tutorial and this question.

While executing linux /boot/vmlinuz-4.15.0-135-generic root=/dev/sda1, I receive the error: error: attempt to read or write outside of hd1.

The correct file system for me seems to be (hd1,1).

Therefore I used set root=(hd1,1) and set prefix=(hd1,1)/boot/grub beforehand.

Running the following commands

insmod linux
insmod normal
normal

does seem to do nothing at all.

What steps do I need to go through to get the system booting again? Can I use a ubuntu iso to repair it?

Additional Information (edit):

As said before the original VPS had a size of 15GB, the .img file i pulled out of the broken .gz archive has a size of around 4.3GB. Do I need/ am I able to increase this size with gparted or similar tools? (I'm running it on my unraid machine with above 4TBs of space, but I suspect that's not what @Bodo meant.)

Also, pulling out individual files has limited value for me. Everything really important has been backed up a long time ago, the point of this image was to be able to spin up the exact (or as close as I can get to) environment my software was running in on the rented virtual server.

Gparted (Edit 2):

I just looked into the gparted route: I received this error message:

Invalid argument during seek for read on /dev/sdb

A quick google search resulted in an entry on this page which told me that

the partition table is at the end of the disk

which is why the partition information is missing.

As the .img file is too small, i can't use the "w" command to write the changes as mentioned on the same page. I'm currently struggling to enlarge this file.

Jojomatik
  • 3
  • Did you put the image on a disk that has at least the size of the original disk (/dev/sdb)? Please [edit] your question to answer. – Bodo Feb 13 '21 at 23:00
  • If I was in this situation I would start from a working linux, and then mount the partitions from the image (readonly) to access the files I wanted. Trying to boot this image seems too much like performing heart surgery on yourself. Of course you might not have any choice. – icarus Feb 13 '21 at 23:07
  • "using ls the most important files seem to be there" - should read "using ls the directory entries for the most important files seem to be there". – icarus Feb 13 '21 at 23:09
  • @Bodo updated my question. – Jojomatik Feb 14 '21 at 09:48
  • @icarus You are probably right. If I understand you correctly, you are basically saying that I might only have the pointers to the files, but not the files themselves? – Jojomatik Feb 14 '21 at 09:49
  • This is too long for a comment, but it is not an answer either. As an analogy think of a book. At the start of the book there might be an index. It tells you the name of the chapters. If the book has had the last 2/3rds ripped off you might know all the names of the chapters but still not be able to read them. Particularly annoying if it is a murder mystery! There is a lot wrong with this analogy - starting with directory entries not being at the start of the disk. – icarus Feb 14 '21 at 18:12

1 Answers1

0

Coincidentally, I did something similar quite recently (although it was due to a hard-drive failure). In my case, ext4 happened to store the pay-load at the beginning of the disk, as shown by this graph (ext4 group usage over group position):

ext4 group usage over group position

Way A: use a disk drive

  1. Have a disk big enough to contain the entire image
  2. dd the damaged image onto that disk

Way B: enlarge the raw image file

  1. truncate -s 16 G image.img (appends space filled with zeros)
  2. Mount the imag udisksctl loop-setup --file image.img

Common for A and B:

  1. run the appropriate file system checker, e.g. fsck.ext4 $device with $device being the partition on the drive or loop device
  2. file-system is now mountable

If the damage is small enough, the system might still be bootable.

Hermann
  • 6,148
  • Sounds similar to what I had in mind. Could you elaborate a bit more on step 3? – Jojomatik Feb 14 '21 at 10:49
  • Also: How would I even create a larger .img file? I tried dd if=image.img of=image_resize.img bs=1M count=16384 to create a 16GB file but that just copied the same 4.3GB file – Jojomatik Feb 14 '21 at 11:41
  • @Jojomatik I assumed you had a disk you want to boot from. For working with the image, I recommend using truncate to set the image file size. I extended the answer. – Hermann Feb 14 '21 at 12:07
  • I have mounted the img in another ubuntu installation. If I nano a certain file it only reads ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@. To me this seems like this file system is competely broken. Tbh, I would expect something more gibrish and less ordered that those two characters if it were broken? Could this also be some permission thing or sth else I'm missing? fsck reports are lot of missing inodes, this doesn't seem right to me anyway. – Jojomatik Feb 14 '21 at 15:38
  • fsck reports missing inodes because they were never copied. Their information is lost. Many other inodes have survived, but only their metadata. The file content can be anything which was lingering in the unused blocks of the disk holding the image. On SSDs, they contain default value (usually zeros). – Hermann Feb 14 '21 at 17:13
  • Thank you for the clarification. I cannot test your answer for sure, but it seems like it would probably work. I will accept it for others in a similar situation to see. – Jojomatik Feb 14 '21 at 17:58
  • "ext4 happened to store the pay-load at the beginning of the disk" is misleading. See https://www.kernel.org/doc/html/latest/filesystems/ext4/index.html for the details. – icarus Feb 14 '21 at 18:25
  • @icarus I am aware that I have only that one point of data. I adjusted the wording to emphasis this is not an universal truth. The link you gave is rather generic. Can you specify what you mean in particular. https://www.kernel.org/doc/html/latest/filesystems/ext4/overview.html#block-and-inode-allocation-policy? – Hermann Feb 14 '21 at 18:39
  • The partition is divided into block groups, the inode tables are in each block group rather than all at the beginning. The original unix file system had the inodes all at the start. One of the improvements of the Berkeley 'fast file system' was that it spread the inode table across the disk to reduce the amount of seek time. – icarus Feb 14 '21 at 18:46
  • I used dumpe2fs to look at the used blocks within a block group. I am under the impression that a block group stores some inodes and their data. I do not see the documentation saying otherwise. I have probably misunderstood something. Is it possible for an inode of one block group to manage data which is physically located in another block group? – Hermann Feb 14 '21 at 18:58