Rotate disk encryption keys

Google Distributed Cloud (GDC) air-gapped appliance uses the Linux Unified Key Setup (LUKS) to encrypt data on disks when external HSM is not provided. A RAID5 array is created on top of the 4 data drives, and a single LUKS device is created on top of the RAID5 array. The LUKS KEK is randomly generated and can be rotated.

This document describes the steps to rotate the LUKS KEK that must be performed for:

regularly scheduled KEK rotation.
KEK exposure.

Before you begin

Complete the following steps:

Verify that you meet the laptop prerequisites.
Ensure that you can sign in to the three nodes as root user.

Rotate KEK

The Google Distributed Cloud (GDC) air-gapped appliance system utilizes a two-node NetApp ONTAP Select (OTS) cluster deployed on nodes bm01 and bm02 (typically without GPUs). Each node possesses 4 dedicated data drives configured as a RAID5 array for enhanced redundancy. A single LUKS encrypted device is layered on top of each RAID5 array to ensure data security.

The LUKS encryption KEK is stored on the boot disk located in /etc/luks. Rotating the LUKS KEK necessitates rotating the LUKS encryption on both nodes to maintain security across the cluster. To perform a regular KEK rotation or if the KEK is exposed, repeat the following steps on both bm01 and bm02:

Sign in to the node as root user.
Make sure the RAID device exists, and get the RAID_DEVICE full path.
```
ll /dev/md
```
Make sure the old LUKS key file exists, and get the OLD_KEY full path.
```
ll /etc/luks
```
Add the old KEK to key slot 1, so that both key slot 0 and 1 have the same KEK.
```
cryptsetup luksAddKey RAID_DEVICE OLD_KEY --key-slot 1 --key-file OLD_KEY
```
Replace RAID_DEVICE and OLD_KEY with the paths from the previous steps.

Remove the old KEK from key slot 0.

cryptsetup luksKillSlot RAID_DEVICE 0 --key-file OLD_KEY

Generate a new KEK, and store it in the new key file NEW_KEY.
```
dd if=/dev/urandom of=NEW_KEY bs=512 count=1
chmod u=r,go-rwx NEW_KEY
```
Replace NEW_KEY with the full path to the new key file.

Add the new KEK to key slot 0.

cryptsetup luksAddKey RAID_DEVICE NEW_KEY --key-slot 0 --key-file OLD_KEY

Set persistent config to open or close the LUKS device on machine boot or shutdown.

DEVICE_UUID=$(cryptsetup luksUUID RAID_DEVICE)
echo "luksrd5 UUID=${DEVICE_UUID} NEW_KEY luks,discard" > /etc/crypttab

Remove the old KEK from key slot 1.

cryptsetup luksKillSlot RAID_DEVICE 1 --key-file NEW_KEY

Verify the new KEK is working.

cryptsetup luksDump --dump-master-key RAID_DEVICE --key-file NEW_KEY

Verify the old KEK is not working anymore.

cryptsetup luksDump --dump-master-key RAID_DEVICE --key-file OLD_KEY

Remove the old KEK.
```
rm OLD_KEY
```

If a node is rebooted before the rotation completes, the LUKS device can still be opened after boot. You can resume the steps after the node is back.