2

How can I compress dd images and work with them at the same time?

On Windows, I can mark it as compressed and work with it like a normal file.

I can't re-partition, but I can mount some drives (For example NTFS/Win32)

Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
Olav
  • 175
  • You could try fusecompress but you'd have to compile it yourself. It didn't work for me, but I never applied much effort to be honest. YMMV. – garethTheRed Jun 17 '16 at 06:36

3 Answers3

3

There is a Linux module called cloop (compressed loop), packaged along with some corresponding utilities, e.g. on Debian.

You can mount these images in compressed state. Caveat: Read-Only. Also, you have to specifically create them, sda1.img.gz wont just mount.

In an nutshell:

# create_compressed_fs - blocksize < /dev/xyz > imageName.cloop
# insmod cloop.ko file=/path/to/imageName.cloop
# mount -o ro -t whatever /dev/cloop /mnt/compressed

You could convert an existing image with this as a first step:

# gunzip oldImg.gz |create_compressed_fs - blocksize > imageName.cloop

Attention: Make sure you have enough Virtual Memory. (see README)

About the achievable Compression ratio: https://en.wikipedia.org/wiki/Cloop:

A compression ratio of about 2.5:1 is common for software. The Knoppix cloop image, for example, is 700MB compressed and around 1.8GB uncompressed.

This module has seen multi-year production use, due to its use e.g. on the Knoppix Linux live Boot-Disk. Packaged/updated regularly since 2005. Software seems mature, e.g. README was last modified 2008,

Another popular choice for an overlapping group of use-cases: Squashfs, since 2010.

Here is a similar Q, with a nice writeup of a squashfs example. It comes down to installing squashfs-tools and running

# mkdir empty-dir
# mksquashfs empty-dir squash.img -p 'sda_backup.img f 444 root root dd if=/dev/sda bs=4M'
# mount squash.img /mnt/compressed
Community
  • 1
Alex Stragies
  • 6,016
  • Unfortunately, available on Debian doesn't necessarily mean available on CentOS (and vice-versa); just as in this case, where cloop isn't in the CentOS repos. – garethTheRed Jun 17 '16 at 06:35
  • yum search cloop returns Warning: No matches found for: cloop. This message is quite common in the RHEL/CentOS world as they are quite conservative when it comes to selecting packages for inclusion :-( – garethTheRed Jun 17 '16 at 06:44
  • 1
    @garethTheRed that's one of the reasons why i use ubuntu when name/brand recognition is required (for suits) and debian otherwise. – cas Jun 17 '16 at 12:59
  • How far will Fat32/NTFS compression work? Can I perhaps compress it in Windows, and then use it ro in Linux? – Olav Jun 21 '16 at 09:22
2

I suppose that dd image means a compressed copy of some partition that contained a filesystem.

In general, you can't mount such a compressed image, as it is not suitable for random access. You would need to decompress it, then you can mount it and access the filesystem.

If you want a filesystem which compresses data automatically (like the Windows filesystem you mentioned) have a look at BTRFS. But to use that, you need to create a partition, so it might not be an option for you.

Baba
  • 3,279
Hans-Martin Mosner
  • 1,949
  • 1
    squashfs is a compressed filesystem with good random access, albeit read-only (which may be all that's needed in this case). – trr Jul 31 '20 at 04:29
1

To create a compressed image on the fly:

gzip < /path/to/inpurt >/path/to/image.gz

To read a compressed image on the fly:

zcat /path/to/image.gz > /path/to/output
DopeGhoti
  • 76,081
  • 2
    There's no need for the dd in these cases. Use gzip </path/to/input >/path/to/image.gz and zcat /path/to/image.gz >/path/to/output directly. – Chris Davies Jun 17 '16 at 08:31
  • Naturally. I've adjusted my answer to no longer use dd, which I mainly kept because it was referenced in the original question. – DopeGhoti Jun 18 '16 at 21:29