Storage overview

You create persistent storage for workloads running on GKE on Azure with Kubernetes objects.

Control plane storage

When you create a cluster, GKE on Azure provisions Premium SSD LRS disks for control plane and etcd volumes.

Use persistent storage in your GKE on Azure workloads

In GKE on Azure, you use the Kubernetes PersistentVolume (PV), PersistentVolumeClaim (PVC), and StorageClass resources to provide persistent file and block storage to workloads. You can create snapshots of persistent storage with the VolumeSnapshot and VolumeSnapshotClass resources. Snapshots are stored in your Azure account.

StorageClass

Clusters have a default Kubernetes StorageClass that dynamically provisions stateful storage for workloads on Azure Disk Storage (AzureDisk) volumes. You can also use a different StorageClass to provision other types of volumes. These options are described in the following section.

Azure storage accounts

GKE on Azure automatically creates an Azure storage account when a cluster is created. If you would like to use a specific storage account, create a custom storage class.

Configure cluster storage

If you want to provision storage volumes with a non-default StorageClass, you can create a new StorageClass in a cluster that uses different parameters or a different storage driver. Then, you can set the StorageClass as the cluster's default, or configure your workloads to use the StorageClass. For example, you can use a StatefulSet to set a specific StorageClass name.

Azure Disk Storage (AzureDisk)

GKE on Azure manages Azure Disk volumes with the azuredisk-csi-driver.

The CSI Driver version is tied to a GKE on Azure Kubernetes version. When you upgrade your cluster version, GKE on Azure updates the driver version automatically. This version is typically the latest driver available at the time that the GKE on Azure version is released. The legacy Kubernetes kubernetes.io/azure-disk volume type is not supported.

The driver pre-installed with GKE on Azure provides the following StorageClasses by default:

  • standard-rwo (default): Used for provisioning standard SSD Azure disks with LRS
  • premium-rwo: Used for provisioning premium SSD Azure disks with LRS

Azure Disk volumes support the Kubernetes ReadWriteOnce access mode.

Volume snapshots

You can use the VolumeSnapshot resource to create snapshots of storage volumes. Then, you can provision new persistent volumes from these snapshots.

Volume cloning

You can use CSI volume cloning to clone existing Azure Disk volumes. For more information, see the Volume cloning example.

Volume resizing

You can resize unattached Azure Disk volumes with volume resizing. For more information, see the Volume resizing example. Resizing while the volume is attached and mounted is not supported.

Azure shared disks

For workloads that are designed to coordinate writes to a shared raw block disk, GKE on Azure supports Azure shared disks. For more information, see Shared disk(Multi-node ReadWrite) in the Azure disk CSI documentation.

Azure File Storage (AzureFile)

GKE on Azure manages Azure Files volumes with the azurefile-csi-driver

The CSI Driver is tied to a GKE on Azure Kubernetes version. When you upgrade your cluster version, the new driver version applies automatically. This version is typically the latest driver available at the time that the GKE on Azure version is released. The legacy Kubernetes kubernetes.io/azure-file volume type is not supported.

The driver pre-installed with GKE on Azure provides the following StorageClasses by default:

  • standard-rwx: Used for provisioning standard SSD Azure file shares with LRS
  • premium-rwx: Used for provisioning premium SSD Azure file shares with LRS

Azure file volumes use the Kubernetes ReadWriteMany access mode.

Volume snapshots

You can use the VolumeSnapshot resource to create snapshots of Azure File volumes. For more information, see the Azure File Snapshot feature documentation. To restore a snapshot, you use the Azure portal. For more information, see Overview of share snapshots for Azure Files.

Volume resizing

You can resize Azure File volumes with volume resizing. For more information, see the Volume resizing example.

Additional storage drivers

Other third-party storage systems can be used with GKE on Azure through third-party Container Storage Interface (CSI) drivers (recommended) or in-tree Kubernetes volume plugins.

CSI drivers

The Container Storage Interface (CSI) is an open standard API that enables Kubernetes to attach arbitrary storage systems to containerized workloads. A non-comprehensive list of CSI drivers can be found in the Kubernetes CSI Developer Documentation. GKE on Azure supports CSI v1.x.

To use a CSI driver in your cluster, you need to install the CSI driver provided by your storage vendor. You can then configure workloads to use the driver's StorageClass or set it as the default StorageClass.

Azure-specific storage drivers

The following volume drivers can be used with GKE on Azure:

GKE on Azure doesn't manage Azure Blob storage directly. You can manually install the blob-csi-driver CSI driver, but support is not available from Google.

Kubernetes in-tree volume plugins

Kubernetes ships with in-tree (built-in) volume plugins. Supported in-tree drivers include the following:

  • configMap
  • emptyDir
  • hostPath
  • nfs
  • projected
  • secrets

What's next?