To shrink a LUKS-encrypted partition, see my other answer here instead: Unlock a LUKS-encrypted outer partition, shrink and edit the LVM volume(s) within it, and then shrink the outer LUKS partition using a combination of 3 GUIs: Gnome "Disks", "blivet-gui", and "GParted".
How to clone your hard drive (SSD or HDD) and expand your LUKS-encrypted partition to fill the new full disk space:
All I was missing were these 2 commands:
sudo e2fsck -f /dev/mapper/system-root
sudo resize2fs -p /dev/mapper/system-root
I found this problem quite challenging, but here are the steps I just took tonight (25 Oct. 2021) to get it done.
The steps below were determined by looking here: https://help.ubuntu.com/community/ResizeEncryptedPartitions#Detailed_resizing_.2BAH4_Enlarging_an_encrypted_partition, as well as by looking at most of the other answers here, as well as by doing a lot of trial and error and just figuring things out.
Tested using an Ubuntu 20.04.3 LTS live USB image, created using mkusb, as described here: https://www.howtogeek.com/howto/14912/create-a-persistent-bootable-ubuntu-usb-flash-drive/.
Hardware I used:
- PNY CS2130 2TB M.2 PCIe NVMe Gen3 x4 Internal Solid State Drive (SSD), Read up to 3,500 - M280CS2130-2TB-RB, $200
- Sabrent USB 3.2 Type-C Tool-Free Enclosure for M.2 PCIe NVMe and SATA SSDs (EC-SNVE), $27
Brief summary:
While running Linux from an external live USB, do the following:
- Carefully clone your disk using
ddrescue (see below for full details).
- Use gparted to unlock your LUKS-encrypted partition and grow (resize) it. Leave ~10% of the space at the end of the SSD unpartitioned for over-provisioning.
- With the now-unlocked/decrypted file system NOT mounted, run
e2fsck to check and fix the disk. In the command below, system-root is what I see as my most-logical option when I type sudo e2fsck -f /dev/mapper/ and then press Tab Tab to have it list out all valid auto-complete options.:
sudo e2fsck -f /dev/mapper/system-root
- Now run the
resize2fs command to expand the unlocked file system to fill the full size of your newly-expanded LUKS-encrypted partition it is in:
sudo resize2fs -p /dev/mapper/system-root
- Mount your unlocked and newly-resized LUKS partition file system to ensure that it actually grew
cd ~
mkdir mnt
sudo mount /dev/mapper/system-root mnt
df -h # look for the new size of the `/dev/mapper/system-root` file system
# optionally unmount
sudo umount ~/mnt
- Reboot and use your new disk as normal.
df -h will now show your expanded size! Everything will work exactly as it did before you cloned your drive!
Details and full steps:
How to clone your hard drive (SSD or HDD) and expand your LUKS-encrypted partition to fill the new full disk space:
1. clone your disk
I'm assuming you've just cloned your disk to a larger drive using dd or ddrescue, as I explain in my steps on my personal website article here: https://www.electricrcaircraftguy.com/2018/01/how-to-clone-your-hard-drive.html.
In short, while booted onto an Ubuntu live USB flash drive (make one with mkusb), do:
# Install GNU ddrescue (gddrescue)
sudo add-apt-repository universe
sudo apt update
sudo apt install gddrescue
Do the clone.
WARNING WARNING WARNING: do NOT get these drives backwards or you will
WIPE ALL YOUR DATA ON YOUR ORIGINAL DISK!!
A 500 GB clone will take:
1. ~3~6 hrs using an external spinning USB 2.0 HDD
2. ~2.5~3 hrs using an external spinning USB 3.0 HDD
3. ~20 minutes using a brand new m.2 SSD in a USB C external enclosure
time sudo ddrescue -f -n /dev/FROM_DRIVE /dev/TO_DRIVE dd_rescue.log
Try to copy bad sectors 3 more times each; takes a few seconds usually
time sudo ddrescue -d -f -r3 /dev/FROM_DRIVE /dev/TO_DRIVE dd_rescue.log
Example output of the first clone cmd above:
ubuntu@ubuntu:~$ time sudo ddrescue -f -n /dev/sda /dev/sdb dd_rescue.log
GNU ddrescue 1.23
Press Ctrl-C to interrupt
ipos: 512110 MB, non-trimmed: 0 B, current rate: 558 MB/s
opos: 512110 MB, non-scraped: 0 B, average rate: 410 MB/s
non-tried: 0 B, bad-sector: 0 B, error rate: 0 B/s
rescued: 512110 MB, bad areas: 0, run time: 20m 46s
pct rescued: 100.00%, read errors: 0, remaining time: n/a
time since last successful read: n/a
Finished
real 21m2.331s
user 0m6.264s
sys 8m24.021s
2. Use gparted to grow your partition
While still running Ubuntu from its installation media (a live USB probably), open gparted, which is the GNU Partition Editor GUI. Choose your external /dev/TO_DRIVE in the drop-down menu on the top right. Ex: mine is /dev/sdb. You'll see something like this in the destination disk after doing the clone:
Notice that ddrescue automatically cloned the partitions too, since it does a direct byte-for-byte copy, which is a lower-level copy than the file system even.
As you can see above, /dev/sdb3 shows that it is [Encrypted]. That is what a LUKS-encrypted partition looks like in gparted.
Right-click the LUKS-locked partition and go to Open Encryption:
Type in your LUKS password to unlock it:
That partition's name will now change from [Encrypted] to [Encrypted] lvm2 pv, indicating it is unlocked. The mount point will now show system, as shown here:
You can optionally right-click the partition and go to "Information" for more information.
Next, Right-click the partition and go to Resize/Move.
As shown below, drag the right-side of the graphical partition box to the right to grow the partition. If this is an ssd, leave about 10% of the total disk space UNUSED on the far right, for dynamic over-provisioning. See here for what that means:
- https://en.wikipedia.org/wiki/Write_amplification#Over-provisioning
- https://www.seagate.com/tech-insights/ssd-over-provisioning-benefits-master-ti/
Essentially, it improves SSD read and write speed and reduces wear, as the SSD will automatically use this unpartitioned space as a sort of SSD-firmware-level swap space.
Click "Resize/Move" when done, then click the green check mark at the top of the gparted window to apply these changes (hovering over the green check mark shows "Apply All Operations").
If you look at the gparted output details, you'll see it conducted the following steps to grow the partition (this screenshot is from the output HTML details if you save them):
Notice it ran cryptsetup -v resize 'sdb3_crypt' and lvm pvresize -v '/dev/mapper/sdb3_crypt' for you.
IMPORTANT: Once resizing the partition completes, if at any point from here on out you see this warning pop up, click the "Fix" button:
Libparted Warning
Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an extra 2906813952 blocks) or continue with the current setting?
Fix
Ignore
Now that you resized the partition, you'll see it has grown, as shown here:
If you mount this unlocked LUKS partition at this time (I have mount instructions further below if you're curious), however, and run df -h to see the disk usage, it will show that the mounted and decrypted disk has NOT yet grown. We'll fix that next.
3. Change out the disks so that you don't have duplicate disk UUIDs
Since you just did a byte-for-byte copy of the old disk to the new disk, your new disk now has an identical UUID number to the old disk. This will cause problems with software tools which try to manipulate the disks (including blivet-gui, which will now throw the error below if you try to open it). So, to resolve that, take out the old disk and put the newly-cloned new disk in its place.
Example error you'll see if you try to open blivet-gui, for instance, at this time, due to duplicate disk UUIDs:
Error
Unknown error occured. Blivet-gui will be terminated.
Failed to init blivet:
Duplicate UUID 'abcdef01-abcd-0123-abcd-0123456789ab' found for devices: 'sda1' and 'sdb1'.
So, assuming you want your cloned disk to be your new internal disk, make that hardware change now. Turn off the computer (shutdown from the live USB), take apart the computer, remove the old drive, and put in the newly-cloned drive as the primary drive inside the computer.
If you choose not to do that at this time, you should change the UUID of the new disk so it doesn't match the old disk (since you just did an exact clone). I don't remember those details so I'll let you Google if you really need to use both the old disk and the new disk at the same time in the same computer. If you're like me, however, and you're removing the old disk entirely, do that, put the new one in its place, then continue on to the next step.
4. Grow the decrypted file system to fit the newly-resized partition it is in
With your newly-cloned drive now in your computer as the main disk, and the original drive NOT plugged in at all, boot back onto your live USB.
You now have 2 options:
Option 1. [Newly-added as of 16 Mar. 2023--what I recommend now] Use a GUI to grow the LVM volume within the LUKS partition
In the Ubuntu live USB, install blivet-gui, following my instructions here. In short:
sudo apt update
sudo add-apt-repository universe
sudo apt --fix-broken install
sudo apt update
optional, if you want to try the other answer here recommending this tool too
sudo apt install partitionmanager
sudo apt install curl
On the Ubuntu 22.04 live USB
echo 'deb http://download.opensuse.org/repositories/home:/vtrefny/xUbuntu_22.04/ /' | sudo tee /etc/apt/sources.list.d/home:vtrefny.list
curl -fsSL https://download.opensuse.org/repositories/home:vtrefny/xUbuntu_22.04/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/home_vtrefny.gpg > /dev/null
sudo apt update
sudo apt install blivet-gui
Then:
- Use Disks to unlock the outer LUKS-encrypted partition: press the Windows key and type in "disks" to open up the
gnome-disks GUI tool. Click on your drive of interest containing the LUKS-encrypted partition, then find the locked LUKS-encrypted partition. This is what I'm calling the "outer partition", since it contains protected inner LVM volumes. Click the unlock icon (it looks like an unlocked padlock) and enter your encryption password to unlock the partition. It will take several seconds to complete unlocking. You'll see the unlocked inner "LVM2 PV" volumes now show up in the Disks GUI underneath that outer "LUKS" partition. Reminder: only the Disks tool can properly unlock the partition to work with the next step. Don't unlock it with blivet-gui nor GParted, or else blivet-gui will throw an error in the next step.
[Adapted from my instructions here] Use blivet-gui to grow and edit the inner LVM volume(s) within the outer LUKS-encrypted partition: press the Windows key and type in "blivet" to open the blivet-gui tool. Note: the blivet-gui should NOT already be open! You must do the unlock step above in "Disks" before opening blivet-gui. Under the bold "LVM" label is my vgubuntu Logical Volume which I just unlocked with the Disks tool above. I want to grow this LVM to fill the entire outer LUKS-encrypted partition space. Here is what I see in blivet-gui when I click on the vgubuntu LVM and then the "Logical View" tab:
You can see the vgubuntu-root 474.75 GiB section, which is what I just cloned to this new disk using ddrescue above. Let's delete the 980 MiB swap logical partition in this LVM and grow the main LVM to fill the whole (980 MiB + 1.3 TiB) space, as follows:
Right-click the 980 MiB logial swap partition and go to "Delete". Right-click the 474.75 GiB logical main partition and go to --> Edit --> Resize --> drag the slider all the way to the right to resize it to fill the whole LVM container within the LUKS partition.
Click "Resize", then click the little check-mark at the top of the main blivet-gui window --> you'll need to "Confirm scheduled actions" by clicking "Ok" to complete the operation. It will take about 20 seconds. Be patient. When it completes, click "OK" on the box which says "All queued actions have been processed."
Now close blivet-gui, Disks, and gparted (if you had it open too). Then, open your nautilus file manager and go to "Other Locations" in the left-hand pane --> click on "1.9 TB Volume" to auto-mount it. Once your new disk is mounted, run df -h in the terminal to view its new size and space avaiable. The output below is what I see now. The file-system I just expanded via blivet-gui is the very last line in this output, and is called Filesystem /dev/mapper/vgubuntu-root, mounted at path /media/ubuntu/abcdef01-abcd-0123-abcd-0123456789ab. You can see I now have 412 GB used and 1.3 TB available! I just cloned my nearly-full 512 GB disk to a new 2 TB disk, so that makes sense!
ubuntu@ubuntu:~$ df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 3.2G 11M 3.2G 1% /run
/dev/sda1 3.6G 3.6G 0 100% /cdrom
/cow 16G 581M 16G 4% /
/dev/disk/by-label/writable 11G 18M 11G 1% /var/log
tmpfs 16G 432K 16G 1% /dev/shm
tmpfs 5.0M 8.0K 5.0M 1% /run/lock
tmpfs 16G 324K 16G 1% /tmp
tmpfs 3.2G 184K 3.2G 1% /run/user/999
/dev/mapper/vgubuntu-root 1.8T 412G 1.3T 25% /media/ubuntu/abcdef01-abcd-0123-abcd-0123456789ab
You can now reboot onto your new internal disk and use it as normal. See the steps below for more details.
Basically, just shut down, remove your Ubuntu live USB, and boot back up into the internal drive as normal! That's it!
You'll type in your LUKS encryption password at boot, just like normal. You'll log in to your user account just like normal. Everything will be exactly as you last left it before the clone, except df -h will show that your root filesystem mounted at path / is now much bigger!
Done.
Option 2. [Original answer] Use the command-line to grow the LVM volume within the LUKS partition
Again, open gparted to decrypt/unlock the drive, as described previously above:
Right-Click the LUKS-encrypted partition --> Open Encryption --> type password.
Now run this, where system-root is what I see as my most-logical option when I type sudo e2fsck -f /dev/mapper/ and then press Tab Tab to have it list out all valid auto-complete options.
# WARNING WARNING WARNING: do NOT run this cmd while this partition is mounted!
# It says that file system corruption WILL occur.
sudo e2fsck -f /dev/mapper/system-root
This e2fsck step is REQUIRED to be run before the resize2fs step, which comes next. e2fsck checks and fixes the newly-cloned and decrypted (unlocked) file system. Example output:
ubuntu@ubuntu:~$ sudo e2fsck -f /dev/mapper/system-root
e2fsck 1.45.5 (07-Jan-2020)
Pass 1: Checking inodes, blocks, and sizes
Inode 27012476 extent tree (at level 2) could be narrower. Optimize<y>? yes
Inode 27013750 extent tree (at level 2) could be narrower. Optimize<y>? yes
Inode 27191704 extent tree (at level 2) could be narrower. Optimize<y>? yes
Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/system-root: ***** FILE SYSTEM WAS MODIFIED *****
/dev/mapper/system-root: 4106785/31121408 files (0.5% non-contiguous), 114717248/124455936 blocks
Now run the resize2fs command to expand the unlocked file system to fill the full size of your newly-expanded LUKS-encrypted partition it is in:
# This command can be run with this partition mounted, but if you're following
# my instructions exactly right now, it will *not* be mounted at this time.
sudo resize2fs -p /dev/mapper/system-root
Example output if you did have it mounted when you ran it:
ubuntu@ubuntu:~$ sudo resize2fs -p /dev/mapper/system-root
resize2fs 1.45.5 (07-Jan-2020)
Filesystem at /dev/mapper/system-root is mounted on /home/ubuntu/mnt; on-line resizing required
old_desc_blocks = 60, new_desc_blocks = 211
The filesystem on /dev/mapper/system-root is now 441932800 (4k) blocks long.
Now the file system is expanded! Next we will mount it and run df -h to prove it.
5. Mount the decrypted (unlocked) partition and run df -h to prove its file system is expanded
We are still running in the live USB, and we have already unlocked the LUKS partition with our password using gparted. Let's mount it so we can read and access it if we want!
cd ~
mkdir mnt
# Mount `/dev/mapper/system-root` to the local ~/mnt directory
sudo mount /dev/mapper/system-root mnt
Now run df -h and you'll see this. Notice my disk is only 28% used now, with 1.2 TB available!
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/system-root 1.7T 430G 1.2T 28% /home/ubuntu/mnt
Previously it was 97% used with only 14 GB available!
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/system-root 467G 430G 14G 97% /home/ubuntu/mnt
Optionally, unmount the file system when done with:
sudo umount ~/mnt
6. Reboot and use your new disk as normal
Shut down the live USB and unplug it from your computer. Turn your computer back on to boot from the newly-cloned and now-expanded new internal drive. Type in your LUKS password when prompted, as an early part of the boot process. Log in like normal. You'll see your computer operating system and files are all intact and everything operates as if nothing was changed!
df -h, however, shows your new root (/) mount point has much more space!:
new df -h:
Notice I am 28% used with 1.2 TB available!
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/system-root 1.7T 430G 1.2T 28% /
old df -h:
Notice I am 97% used with only 14 GB available!
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/system-root 467G 430G 14G 97% /
Extra notes
To use gparted to "eject and remove" a LUKS file system, after unlocking it with your password, do the following:
- In gparted, right-click the partition and go to --> Deactivate.
- Then, right-click the partition again and go to "Close Encryption".
- Now you can safely unplug it, as far as I know.
I don't know what happened or why, but somewhere along the way gparted went from showing the true partition usage when unlocked to NOT showing the true partition usage when unlocked. It now shows full 100% usage even when unlocked. I think I did something weird. See the image below. Everything seems to work ok though, so I guess I can ignore it? If anyone knows why it does NOT show my true disk usage for this partition anymore, please let me know!
Keywords: expand encrypted LUKS partition: how to clone your Ubuntu OS or hard drive and expand your LUKS-encrypted hard drive / file system when done! ssd over-provisioning; clone ssd; clone hdd
