8

I've seen numerous questions, answers, and guides relating to use of dd, cat, and clonezilla to facilitate partition and device cloning. Rather than continue those discussions I'm hoping to give very targeted questions to dd behavior within this post.

I'm using Ubuntu 18.04 LTS live, booted from USB. I have a 500gb HDD with Windows7 (/dev/sda). I've resized the partitions such that slightly more than 420gb is unallocated. I aim to backup the boot table and partitions to a 120gb SSD (/dev/sdb). The SSD has no partitions and shows 110gb unallocated.

sudo dd if=/dev/sda of=/dev/sdb hits failure and conveys 'No space left on device'. Spot checking partitions using gparted, I see /dev/sdb contains the same partitions, labels, size, and unused. The only noticeable difference is unallocated space.

I am able to boot Windows from the SSD but am left with a few questions related to dd behavior.

  • did dd attempt to copy device content which was not allocated?
  • does dd have an order of operations? (something like partition 1->99, then misc drive blocks)
  • by chance have I just been lucky so far in my immediate use of this Windows drive? am I being overly paranoid in thinking the drive is missing content?

For what it's worth, I do plan to retain the HDD for the foreseeable future.

jgrump2012
  • 183
  • 1
    I'm voting to close this question as off-topic because this is a Unix group. If you have questions about Windows installer, SuperUser stack Exchange is probably more suited. – Rui F Ribeiro Aug 12 '18 at 18:35
  • 2
    @RuiFRibeiro - while his use case was applied to cloning a Windows thing, he used Unix tools to do it. Fair question I think since once you get past the back ground he's trying to determine exactly how dd works. – ivanivan Aug 12 '18 at 18:43
  • @ivanivan It is not about what was used, but what it is being asked about the integrity of the Windows copy. I honestly think this is a much better match to our sister site superuser.com. – Rui F Ribeiro Aug 12 '18 at 18:45
  • 2
    @RuiFRibeiro - if he said "mp3 collection" instead of "windows installation" would that be different? Do what you will, you've been here longer, etc. but I still think that at the heart it is a valid question – ivanivan Aug 12 '18 at 18:47
  • I appreciate the responses. Feel free to strike the 3rd question from the record if you believe it detracts from purpose of the community. As I stated in the opening post, my goal is to better understand dd behavior. – jgrump2012 Aug 13 '18 at 02:23

2 Answers2

10

dd does not know anything about partitions or any other structure on the disk. It doesn't recognize "allocated" or "unallocated" parts of the disk. From dd's point of view, a device given as parameter consists of a series of blocks. Thus, if you run dd if=/dev/sda of=/dev/sdb, dd will copy blocks starting from the beginning of /dev/sda to the correspondingly numbered blocks on /dev/sdb, until it has read all blocks from /dev/sda or until it hits an error. In your example, because /dev/sdb was smaller in size than /dev/sda, dd was terminated with the error message "No space left on device" (on /dev/sdb).

Johan Myréen
  • 13,168
  • maybe recommend ..sda1 and etc? – mikeserv Aug 12 '18 at 19:09
  • 1
    @mikeserv in the specific use case of the querent, I would actually recommend just sticking with the raw block devices and letting dd run till it hits the error; you want the copied disk to be bootable, and creating the partitions manually is much more faff. – Carcer Aug 12 '18 at 22:52
  • Thanks @johan-myréen, this is helpful to understand. Hopefully it's ok to give a followup here. Prior to execution I had used gparted to resize the large drive. Given this, can we establish confidence that the blocks on /dev/sda were reordered such that they were first to be copied during dd execution? – jgrump2012 Aug 13 '18 at 02:31
3

Johan Myréen eloquently explained why your approach resulted in an error. There are a couple of workarounds to copy a drive to a smaller one with dd:

  • As suggested by mikeserv, instead of copying the entire device (/dev/sda), you could just copy the specific partition that contains your data of interest, e.g. /dev/sda1. This has a couple of disadvantages: You have to know which partition contains your data, and if you wipe the original drive and want to restore it, you will need to first recreate the original partition structure (ideally you have notes of what it looked like) and then restore the specific partition. (Any of this is potentially error-prone.) If you plan to run your Windows from the SSD, this is probably your best option (although Windows sometimes objects to hardware changes and wants to be reinstalled, but that is apparently not a problem in this case).
  • If you don't mind storing the content of your drive in compressed form, you can try something like sudo dd if=/dev/sda bs=1M | gzip > /path/to/mounted/backup-drive/sda_backup.gz (if the backup file is not user-writeable, either use sudo -i for a root shell or wrap the entire command with sudo sh -c 'command'; I am setting bs because the default of 512 bytes is inefficient for large amounts of data; instead of gzip you can also use other compression programs). This can be restored with zcat sda_backup.gz | dd of=/dev/sdx bs=1M, but you cannot boot from the partition and even mounting it is tricky (see this question for an interesting discussion of this aspect; the answer by doug65536 also discusses "zero-washing" to improve the compression ratio).

The problem with your current state of affairs is that you have copied the partition table from the large drive to the smaller one, which means it effectively contains wrong information about anything beyond 120 GB, and although that does not prevent you from from booting and using the partition, it is not ideal. You could try to just fix the partition table with something like gparted (much quicker than recopying) and hope that nothing else got borked, or start from scratch.

Thomas
  • 31