Using volume snapshots

In Google Kubernetes Engine (GKE), you can use the Kubernetes volume snapshot feature for persistent volumes in your GKE clusters.

Volume snapshots let you create a copy of your volume at a specific point in time. You can use this copy to bring a volume back to a prior state or to provision a new volume.

From GKE version 1.17 or later, you can provision and attach volume snapshots with the following components:

• VolumeSnapshotClass
• VolumeSnapshot
• VolumeSnapshotContent

Note: In Kubernetes version 1.20, the VolumeSnapshot API version was promoted to GA. In GKE 1.23, snapshot.storage.k8s.io/v1beta1 is deprecated. If you are using a GKE cluster version 1.21 and later, migrate to snapshot.storage.k8s.io/v1.

Requirements

To use volume snapshots on GKE, you must meet the following requirements:

• Use a CSI driver that supports snapshots. The in-tree persistent disk driver does not support snapshots.

  • The Compute Engine persistent disk CSI driver is installed by default on new Linux clusters running GKE version 1.18.10-gke.2100 or later, or version 1.19.3-gke.2100 or later. You can also enable the Compute Engine persistent disk CSI driver on an existing cluster.
  • You can also automatically deploy the Filestore CSI driver. For a list of all CSI drivers that support snapshots, see the "Other features" column in Drivers in the Kubernetes documentation.

• Use control plane versions 1.17 or later. To use the Compute Engine persistent disk CSI Driver in a VolumeSnapshot, use GKE versions 1.17.6-gke.4 or later. To use the Filestore CSI Driver in a VolumeSnapshot, use GKE versions 1.21 or later.

• Have an existing PersistentVolumeClaim to use for a snapshot. The PersistentVolume you use for a snapshot source must be managed by a CSI driver. You can verify that you're using a CSI driver by checking that the PersistentVolume spec has a csi section with driver: pd.csi.storage.gke.io or filestore.csi.storage.gke.io. If the PersistentVolume is dynamically provisioned by the CSI driver as described in the following sections, it's managed by the CSI driver.

Before you begin

Before you start, make sure you have performed the following tasks:

• Enable the Google Kubernetes Engine API.
Enable Google Kubernetes Engine API
• If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI.
  Note: For existing gcloud CLI installations, make sure to set the compute/region and compute/zone properties. By setting default locations, you can avoid errors in gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location.

Creating and using a volume snapshot

The examples in this document show you how to do the following tasks:

1. Create a PersistentVolumeClaim and Deployment.
2. Add a file to the PersistentVolume that the Deployment uses.
3. Create a VolumeSnapshotClass to configure the snapshot.
4. Create a volume snapshot of the PersistentVolume.
5. Delete the test file.
6. Restore the PersistentVolume to the snapshot you created.
7. Verify that the restoration worked.

To use a volume snapshot, you must complete the following steps:

1. Create a VolumeSnapshotClass object to specify the CSI driver and deletion policy for your snapshot.
2. Create a VolumeSnapshot object to request a snapshot of an existing PersistentVolumeClaim.
3. Reference the VolumeSnapshot in a PersistentVolumeClaim to restore a volume to that snapshot or create a new volume using the snapshot.

Create a PersistentVolumeClaim and a Deployment

1. To create the PersistentVolumeClaim object, save the following manifest as my-pvc.yaml:

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: my-pvc
   spec:
     storageClassName: standard-rwo
     accessModes:
     - ReadWriteOnce
     resources:
       requests:
         storage: 1Gi
   

   For spec.storageClassName, you can specify any storage class that uses a supported CSI driver. This example uses the standard-rwo storage class installed by default with the Compute Engine persistent disk CSI driver. For more information, refer to Using the Compute Engine persistent disk CSI driver.

2. Apply the manifest:

   kubectl apply -f my-pvc.yaml
   

3. To create a Deployment, save the following manifest as my-deployment.yaml:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: hello-app
   spec:
     selector:
       matchLabels:
         app: hello-app
     template:
       metadata:
         labels:
           app: hello-app
       spec:
         containers:
         - name: hello-app
           image: google/cloud-sdk:slim
           args: [ "sleep", "3600" ]
           volumeMounts:
           - name: sdk-volume
             mountPath: /usr/share/hello/
         volumes:
         - name: sdk-volume
           persistentVolumeClaim:
             claimName: my-pvc
   

4. Apply the manifest:

   kubectl apply -f my-deployment.yaml
   

5. Check the status of the Deployment:

   kubectl get deployment hello-app
   

   It might take some time for the Deployment to become ready. You can run the preceding command until you see an output similar to the following:

   NAME        READY   UP-TO-DATE   AVAILABLE   AGE
   hello-app   1/1     1            1           2m55s
   

Add a test file to the volume

1. List the Pods in the Deployment:

   kubectl get pods -l app=hello-app
   

   The output is similar to the following:

   NAME                         READY   STATUS    RESTARTS   AGE
   hello-app-6d7b457c7d-vl4jr   1/1     Running   0          2m56s
   

2. Create a test file in a Pod:

   kubectl exec POD_NAME \
       -- sh -c 'echo "Hello World!" > /usr/share/hello/hello.txt'
   

   Replace POD_NAME with the name of the Pod.

3. Verify that the file exists:

   kubectl exec POD_NAME \
       -- sh -c 'cat /usr/share/hello/hello.txt'
   

   The output is similar to the following:

   Hello World!
   

Create a VolumeSnapshotClass object

Create a VolumeSnapshotClass object to specify the CSI driver and deletionPolicy for your volume snapshot. You can reference VolumeSnapshotClass objects when you create VolumeSnapshot objects.

1. Save the following manifest as volumesnapshotclass.yaml.

   v1

   Use the v1 API version for clusters running versions 1.21 or later.

   apiVersion: snapshot.storage.k8s.io/v1
   kind: VolumeSnapshotClass
   metadata:
     name: my-snapshotclass
   driver: pd.csi.storage.gke.io
   deletionPolicy: Delete
   

   v1beta1

   Use the v1beta1 API version for clusters running versions 1.21 or earlier.

   apiVersion: snapshot.storage.k8s.io/v1beta1
   kind: VolumeSnapshotClass
   metadata:
     name: my-snapshotclass
   driver: pd.csi.storage.gke.io
   deletionPolicy: Delete
   

   In this example:

   • The driver field is used by the CSI driver to provision the snapshot. In this example, pd.csi.storage.gke.io uses the Compute Engine persistent disk CSI Driver.

   • ThedeletionPolicy field tells GKE what to do with the VolumeSnapshotContent object and the underlying snapshot when the bound VolumeSnapshot object is deleted. Specify Delete to delete the VolumeSnapshotContent object and the underlying snapshot. Specify Retain if you want to keep the VolumeSnapshotContent and the underlying snapshot.

   To use a custom storage location, add a storage-locations parameter to the snapshot class. To use this parameter, your clusters must use version 1.21 or later.

   apiVersion: snapshot.storage.k8s.io/v1
   kind: VolumeSnapshotClass
   metadata:
     name: my-snapshotclass
   parameters:
     storage-locations: us-east2
   driver: pd.csi.storage.gke.io
   deletionPolicy: Delete
   

2. Apply the manifest:

   kubectl apply -f volumesnapshotclass.yaml
   

Create a VolumeSnapshot

A VolumeSnapshot object is a request for a snapshot of an existing PersistentVolumeClaim object. When you create a VolumeSnapshot object, GKE automatically creates and binds it with a VolumeSnapshotContent object, which is a resource in your cluster like a PersistentVolume object.

1. Save the following manifest as volumesnapshot.yaml.

   v1

   Use the v1 API version for clusters running versions 1.21 or later.

   apiVersion: snapshot.storage.k8s.io/v1
   kind: VolumeSnapshot
   metadata:
     name: my-snapshot
   spec:
     volumeSnapshotClassName: my-snapshotclass
     source:
       persistentVolumeClaimName: my-pvc
   

   v1beta1

   Use the v1beta1 API version for clusters running versions 1.21 or earlier.

   apiVersion: snapshot.storage.k8s.io/v1beta1
   kind: VolumeSnapshot
   metadata:
     name: my-snapshot
   spec:
     volumeSnapshotClassName: my-snapshotclass
     source:
       persistentVolumeClaimName: my-pvc
   

2. Apply the manifest:

   kubectl apply -f volumesnapshot.yaml
   

   After you create a volume snapshot, GKE creates a corresponding VolumeSnapshotContent object in the cluster. This object stores the snapshot and bindings of VolumeSnapshot objects. You do not interact with VolumeSnapshotContents objects directly.

3. Confirm that GKE created the VolumeSnapshotContents object:

   kubectl get volumesnapshotcontents
   

   The output is similar to the following:

   NAME                                               AGE
   snapcontent-cee5fb1f-5427-11ea-a53c-42010a1000da   55s
   

After the volume snapshot content is created, the CSI driver you specified in the VolumeSnapshotClass creates a snapshot on the corresponding storage system. After GKE creates a snapshot on the storage system and binds it to a VolumeSnapshot object on the cluster, the snapshot is ready to use. You can check the status by running the following command:

kubectl get volumesnapshot \
  -o custom-columns='NAME:.metadata.name,READY:.status.readyToUse'

If the snapshot is ready to use, the output is similar to the following:

NAME               READY
my-snapshot        true

Delete the test file

1. Delete the test file that you created:

   kubectl exec POD_NAME \
       -- sh -c 'rm /usr/share/hello/hello.txt'
   

2. Verify that the file no longer exists:

   kubectl exec POD_NAME \
       -- sh -c 'cat /usr/share/hello/hello.txt'
   

   The output is similar to the following:

   cat: /usr/share/hello/hello.txt: No such file or directory
   

Restore the volume snapshot

You can reference a VolumeSnapshot in a PersistentVolumeClaim to provision a new volume with data from an existing volume or restore a volume to a state that you captured in the snapshot.

To reference a VolumeSnapshot in a PersistentVolumeClaim, add the dataSource field to your PersistentVolumeClaim.

In this example, you reference the VolumeSnapshot that you created in a new PersistentVolumeClaim and update the Deployment to use the new claim.

1. Save the following manifest as pvc-restore.yaml:

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: pvc-restore
   spec:
     dataSource:
       name: my-snapshot
       kind: VolumeSnapshot
       apiGroup: snapshot.storage.k8s.io
     storageClassName: standard-rwo
     accessModes:
       - ReadWriteOnce
     resources:
       requests:
         storage: 1Gi
   

   Note: The namespace of the PersistentVolumeClaim must be the same as the namespace of the VolumeSnapshot.

2. Apply the manifest:

   kubectl apply -f pvc-restore.yaml
   

3. Update the my-deployment.yaml file to use the new PersistentVolumeClaim:

   ...
   volumes:
   - name: my-volume
     persistentVolumeClaim:
       claimName: pvc-restore
   

4. Apply the updated manifest:

   kubectl apply -f my-deployment.yaml
   

Check that the snapshot restored successfully

1. Get the name of the new Pod that GKE creates for the updated Deployment:

    kubectl get pods -l app=hello-app
   

Verify that the test file exists:

   kubectl exec NEW_POD_NAME \
       -- sh -c 'cat /usr/share/hello/hello.txt'

Replace NEW_POD_NAME with the name of the new Pod that GKE created.

The output is similar to the following:

   Hello World!

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

1. Delete the VolumeSnapshot:

   kubectl delete volumesnapshot my-snapshot
   

   Note: If a volume snapshot's VolumeSnapshotClass has deletionPolicy:Delete, the corresponding VolumeSnapshotContent and the physical snapshot on the storage system are also deleted. If it has a deletionPolicy:Retain, GKE retains the underlying snapshot.

2. Delete the VolumeSnapshotClass:

   kubectl delete volumesnapshotclass my-snapshotclass
   

3. Delete the Deployment:

   kubectl delete deployments hello-app
   

4. Delete the PersistentVolumeClaim objects:

   kubectl delete pvc my-pvc pvc-restore
   

What's next

• Read the Kubernetes Volume Snapshot documentation.
• Learn about volume expansion.
• Learn how to manually install a CSI driver.