Can I overwrite LUKS partition with it's decrypted content?

Question

I have HDD with my data (partial backup of most important data available on other HDD), currently it's formatted as Ext4 over LVM over LUKS. I want to remove LUKS layer, but reformatting and data restore from backup is too long/no fun. Is there any possibility/chance to overwrite LUKS partition with it's content without using big buffer and without data corruption?

Answer 1

You're going to have to copy all the data around anyway. You should definitely have a backup at this point. Unless your backup device is significantly slower than your active disk, restoring from backup is as fast as it can go.

A LUKS volume starts with a header (up to 2MB). If you lose the header, the data in the volume is lost. As long as the header is intact, you can access data in independent 512-byte sectors.

A strategy like cat /dev/mapper/encrypted >/dev/sdz99 will work, because the ciphertext is located at a positive offset (the header size) relative to the plaintext. However, this may well be slower than restoring from backup, because it's a copy on the same disk (with a disk-to-disk copy, reading and writing are done in parallel). For a same-disk copy, dd with a large block size is only very slightly faster than cat. There is a major caveat with this strategy: if there is a power failure or other system crash during the copy, your whole partition will be hosed, because the header has been overwritten first thing.

You can save the first 2MB of data elsewhere, and move the rest:

dd if=/dev/mapper/encrypted of=/dev/sdz99 bs=2M skip=1 seek=1

This way, you can resume after an interruption (don't try to assemble the logical volume though, let alone mount the filesystem!); however this requires knowing where you left off. This is practically impossible to determine (you'd have to use a copying tool that outputs a trail of the blocks it's copying and writes it to the disk, synchronously with the block copies).

If your backup storage is very slow, then you can use this shifting strategy — the plain cat shifting will do — and fall back to restoring from backup if anything bad happens.

If the backup storage is really unwieldy, then a different approach would be to:

1. Shrink the filesystem (resize2fs).
2. Shrink the logical volume (lvreduce).
3. Shrink the physical volume (pvresize).
4. Shrink the encrypted volume.
5. Shrink the partition (fdisk or gdisk), and create a new partition in the freed space.
6. Create a physical volume in the new partition (pvcreate) and add it to the volume group (vgextend).
7. Move as many physical extents as possible off the encrypted volume (pvmove).
8. If the encrypted volume isn't empty, repeat from step 1.
9. Get rid of the now unused physical volume (vgreduce then pvremove).

Long and tortuous? Yes. Again, my recommendation is to restore from backup.

Answer 2

LUKS puts a header on a volume which is usually 2MiB in size.

Thus you can copy the open LUKS volume content to the LUKS base volume by using dd.

Note that if you lose power during the copy, you'll be left with undecipherable data, since the volume header will have been lost.

If only a small share of the volume is covered with data then it might be faster to shrink the LUKS volume, create a new one behind it and copy the data on the filesystem level (cp -a).

Answer 3

Short answer: It's possible, but I wouldn't do that with my data due to no warranty about data safety and low speed (it, probably, will take ~10hours to convert my 1TB disk)

TL;DR:

I decided to check this in VirtualBox virtual machine with attached 2GB virtual HDD volume as /dev/sdb. I write some script (see end of answer) to test removing LUKS layer from file system stack. The second important thing (the first one is to have full backup of your data) is to dump LUKS header to external storage and open LUKS volume with that external header (because header on volume will be overwritten). Without it fail rate is ~50% (2 test from 4 failed). With external header 4 of 4 test was successful… Anyway, I was too lazy to do more checks (I only checked md5sum of one big file), so there no warranty for your data to be safe. You, probably, will need to edit script if you want to play with it (line with Disk=/dev/sdb)

#!/bin/bash

Disk=/dev/sdb

if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root. Exit"
   exit 1
fi
echo ----------------------------------------------------
echo "Removing old test data"
echo ----------------------------------------------------
vgremove test -f
umount -l /mnt/old
rm -rf /mnt/old
mkdir /mnt/old
umount -l /mnt/new
rm -rf /mnt/new
mkdir /mnt/new
rm -f /tmp/header
rm -f /tmp/key
echo ----------------------------------------------------
DiskCrypt="$(basename $Disk)_crypt"
echo "Creating LUKS device with name $DiskCrypt for disk $Disk"
echo ----------------------------------------------------
echo "password" > /tmp/key
cryptsetup -q luksFormat -c aes-xts-plain64 -s 512 -d /tmp/key $Disk
cryptsetup -q luksHeaderBackup $Disk --header-backup-file /tmp/header
cryptsetup -q luksOpen --header /tmp/header -d /tmp/key $Disk $DiskCrypt
echo ----------------------------------------------------
echo "Creating LVM volume group and logical volume"
echo ----------------------------------------------------
vgcreate test /dev/mapper/$DiskCrypt
lvcreate -n test -l 100%FREE test
pvdisplay
echo ----------------------------------------------------
echo "Format volume to Ext4"
echo ----------------------------------------------------
mkfs.ext4 /dev/mapper/test-test
mount /dev/mapper/test-test /mnt/old
chmod o+rwx /mnt/old
cd /mnt/old
rm -f ./test-file
echo ----------------------------------------------------
echo "Create file with random content to fill entire volume"
echo ----------------------------------------------------
dd if=/dev/urandom of=./test-file bs=64M
stat ./test-file
echo ----------------------------------------------------
echo "Checking MD5 sum"
echo ----------------------------------------------------
md5old=$(md5sum ./test-file)
# cp ./test-file /tmp/
cd /
umount -l /mnt/old
echo ----------------------------------------------------
echo "Disabling and exporting LVM volume group"
echo ----------------------------------------------------
vgchange -an test
vgexport test
echo ----------------------------------------------------
echo "Moving data using dd"
echo ----------------------------------------------------
dd if=/dev/mapper/$DiskCrypt of=$Disk bs=64M
cryptsetup -q luksClose $DiskCrypt
echo ----------------------------------------------------
echo "Importing LVM volume group and check file system"
echo ----------------------------------------------------
pvscan
vgimport test
vgchange -ay test
pvdisplay
# Device may be not ready yet
sleep 5
fsck -f /dev/test/test
resize2fs /dev/test/test
echo ----------------------------------------------------
echo "Checking MD5 sum"
echo ----------------------------------------------------
mount /dev/mapper/test-test /mnt/new
cd /mnt/new
stat ./test-file
md5new=$(md5sum ./test-file)
echo ====================================================
echo "Old File MD5 sum:"
echo "$md5old"
echo ====================================================
echo "New File MD5 sum:"
echo "$md5new"
echo ====================================================
if [ "$md5old" == "$md5new" ]
then
    echo "<< SUCCESS >>"
else
    echo ">> FAILED <<"
fi
echo ====================================================
cd /
umount -l /mnt/new