Regional persistent disks provide synchronous replication of data between two zones in a region. Regional persistent disks can be a good building block to use when you implement high availability (HA) services in Compute Engine. Regional persistent disks are also designed to work with regional managed instance groups.
Failure scenarios
With regional persistent disks, when the device is in replicating mode, data is automatically replicated to two zones in a region. A write is acknowledged back to a VM when it is durably persisted in both replicas.
If replication to one zone fails or is very slow for a while, the disk switches to unreplicated mode. In this mode, write is acknowledged after it is durably persisted in one replica.
If and when Compute Engine detects that replication can be resumed, data previously written since the device entered unreplicated mode is synced to both zones and the disk returns to replicating mode. The transition is fully automated and transparent to the customer.
RPO and RTO are undefined while a device is in unreplicated mode. To minimize data and/or availability loss in the event of a failure of a disk operating in unreplicated mode, we recommend that you back up your regional persistent disks regularly using snapshots. You can recover a disk by restoring the snapshot.
Zonal failures
A regional persistent disk is replicated in two zones:
- One replica is located in the same zone as the VM instance to which it is attached (the primary zone).
- The other replica is located in an alternate zone in the same region (the secondary zone).
In the event that the primary zone fails, you can fail over your regional
persistent disk to a VM instance in another zone by using the --force-attach
flag with the attach-disk
command.
In this scenario, you might not be able to detach a disk from the instance because the instance can't be reached to perform the detach operation. Force-attach lets you attach a regional persistent disk to a VM instance even if that disk is currently attached to another instance.
After you complete the force-attach
operation, Compute Engine
prevents the original VM from writing to the disk. Using force-attach
lets you safely regain access to your data and recover your service. You also
have the option to manually shut down the instance
after you perform the force-attach
step.
In the event that the secondary zone fails, the unhealthy replica comes back into sync with the healthy replica automatically when the secondary zone recovers.
Initial state | Failure | New state | Action |
---|---|---|---|
Two healthy zones | The primary zone fails |
|
Force attach the disk to a VM in the healthy zone. |
Two healthy zones | The secondary zone fails |
|
No action needed. The unhealthy replica is brought back into sync when the zone recovers. |
|
The healthy zone fails |
|
|
Application and VM failures
In the event of outages caused by VM misconfiguration, an unsuccessful OS
upgrade, or other application failures, you can force-attach
your regional
persistent disk to a VM instance in the same zone.
Failure category and (probability) | Failure types | Action |
---|---|---|
Application failure (High) | Application unresponsive
Application admin actions (for example, upgrade) Human error (for example, misconfiguration of parameters such as SSL certificate or ACLs) |
Application control plane can trigger failover based on health check thresholds. |
VM failure (Medium) | Infrastructure/hardware failure
VM unresponsive due to CPU contention, intermediate network interruption |
VMs are usually autohealed. The application control plane can trigger failover based on health check thresholds. |
Application corruption (Low-Medium) | Application data corruption (for example, due to application bugs or an unsuccessful OS upgrade) |
Application recovery:
|
Failover your regional persistent disk using force-attach
To force attach an existing disk to a VM, perform the following steps:
Console
Go to the VM instances page.
Select your project.
Click the name of the VM you want to change.
On the details page, click Edit.
In the Additional disks section, click Attach additional disk.
Select the regional persistent disk from the drop-down list.
To force attach the disk, select the Force-attach disk checkbox.
Click Done, and then click Save.
You can perform the same steps to force-attach
a disk to the original
VM after the failure is resolved.
gcloud
In the gcloud CLI, use the
instances attach-disk
command
to attach the replica disk to a VM instance. Include
the --disk-scope
flag and set it to regional
.
gcloud compute instances attach-disk INSTANCE_NAME \
--disk DISK_NAME --disk-scope regional \
--force-attach
Replace the following:
INSTANCE_NAME
: the name of the new VM instance in the regionDISK_NAME
: the name of the disk
After you force-attach
the disk, mount the file systems on the disk,
if necessary. The instance can use the force-attached disk to continue read
and write operations.
API
Construct a POST
request to the
compute.instances.attachDisk
method
method, and include the URL to the persistent disk that you just created.
To attach the disk to the new VM instance, the forceAttach=true
query
parameter is required, even though the primary instance still has the disk.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/INSTANCE_NAME/attachDisk?forceAttach=true
{
"source": "projects/PROJECT_ID/regions/REGION/disks/DISK_NAME"
}
Replace the following:
PROJECT_ID
: your project IDZONE
: the location of your instanceINSTANCE_NAME
: the name of the instance where you are adding the new persistent diskREGION
: the region where your new regional persistent disk is locatedDISK_NAME
: the name of the new disk
After you attach the replica disk, mount the file systems on the disks if necessary. The instance can use the replica disk to continue read and write operations.
What's next
- Add or resize a regional persistent disk.
- Learn about using regional persistent disks for high availability services.