A new version of GKE on AWS was released on October 2. See the release notes for more information.

Using StorageClasses with your workloads

GKE on AWS automatically deploys the Container Storage Interface (CSI) Driver for Amazon Elastic Block Store (EBS) to provision and manage Amazon EBS volumes in your clusters.

The GKE on AWS EBS CSI Driver version is tied to a GKE on AWS Kubernetes version. The driver version is typically the latest available when the GKE version is released. When the cluster is upgraded, the drivers update automatically.

Before you begin

  • From your anthos-aws directory, use anthos-gke to switch context to your user cluster.
    cd anthos-aws
    env HTTP_PROXY=http://localhost:8118 \
    anthos-gke aws clusters get-credentials CLUSTER_NAME

Using the default StorageClass

Creating a PersistentVolumeClaim without the field spec.storageClassName set provisions a gp2 volume using the default GKE on AWS EBS CSI Driver StorageClass.

The following YAML creates a PersistentVolumeClaim (PVC) named mypvc with a size of 30 gibibytes.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mypvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 30Gi

Using another preinstalled StorageClass

GKE on AWS EBS CSI Driver also includes the premium-rwo StorageClass, which provisions higher-throughput io1 volumes.

You can use it by specifying it in the spec.storageclassName of the PVC.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mypvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 30Gi
  storageclassName: premium-rwo

Using a custom StorageClass

You can create additional StorageClasses for EBS volumes or use Container Storage Interface (CSI) Drivers.

  1. Choose if you are using an EBS volume or a specific CSI driver.

    EBS Volume

    You can create your own custom StorageClass that specifies an EBS volume type, file system type, and other parameters. You can find additional StorageClass parameters on the GKE on AWS EBS CSI Driver GitHub page.

    To configure a custom StorageClass, copy the following YAML manifest into a file named my-custom-class.yaml.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: CLASS_NAME
    provisioner: ebs.csi.aws.com
    volumeBindingMode: WaitForFirstConsumer
    

    Replace CLASS_NAME with the name of your new StorageClass.

    For example, the following YAML creates a new StorageClass that provisions Throughput Optimized HDD EBS volumes formatted with the XFS file system.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: my-custom-class
    provisioner: ebs.csi.aws.com
    volumeBindingMode: WaitForFirstConsumer
    parameters:
      csi.storage.k8s.io/fsType: xfs
      type: st1
    

    CSI Driver

    You can specify a different CSI driver in the provisioner field.

    To create a StorageClass with another CSI driver, you can use the example YAML below.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: CLASS_NAME
    provisioner: CSI_DRIVER_NAME
    volumeBindingMode: WaitForFirstConsumer
    parameters:
      ...
    

    Replace the following values:

    • CLASS_NAME: the name of the StorageClass (for example, my-custom-class).
    • CSI_DRIVER_NAME: the name of the CSI driver (for example, csi.example.com).
    • You configure subfields under parameters according to your CSI driver.
  2. Apply the YAML to your cluster.

    env HTTP_PROXY=http://localhost:8118 \
      kubectl apply -f my-custom-class.yaml
    

Creating a PersistentVolumeClaim with a custom StorageClass

  1. Once a custom StorageClass is created, you can specify it in a PVC. The example below creates a PVC named my-pvc that references the StorageClass my-custom-class.

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-pvc
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 30Gi
      storageclassName: my-custom-class
    

Setting the default StorageClass

GKE on AWS uses a default StorageClass called standard-rwo that provisions gp2 EBS volumes. You can change the default to another StorageClass.

To change the default StorageClass:

  1. Update the is-default-class annotation for the standard-rwo StorageClass with kubectl patch.

    env HTTP_PROXY=http://localhost:8118 \
      kubectl patch storageclass standard-rwo -p \
      '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
    
  2. Create a new StorageClass that has the annotation storageclass.kubernetes.io/is-default-class: true.

    The following example StorageClass uses the ebs.csi.aws.com driver. To install another storage driver, see Installing a CSI driver.

    Copy the following YAML into a file named my-custom-class.yaml.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: CLASS_NAME
      annotations:
        storageclass.kubernetes.io/is-default-class: true
    provisioner: ebs.csi.aws.com
    volumeBindingMode: WaitForFirstConsumer
    parameters:
      type: EBS_VOLUME_TYPE
    

    Replace the following:

    • CLASS_NAME: the name of your new StorageClass.
    • EBS_VOLUME_TYPE: the AWS EBS volume type that the StorageClass creates.

    For example, the following YAML creates a new default StorageClass that provisions Throughput Optimized HDD EBS volumes formatted with the XFS file system.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: my-custom-default-class
      annotations:
        storageclass.kubernetes.io/is-default-class: "true"
    provisioner: ebs.csi.aws.com
    volumeBindingMode: WaitForFirstConsumer
    parameters:
      csi.storage.k8s.io/fsType: xfs
      type: st1
    
  3. Apply the new custom class to your cluster.

    env HTTP_PROXY=http://localhost:8118 \
      kubectl apply -f my-custom-class.yaml
    

After applying this manifest, GKE on AWS uses the my-custom-default-class StorageClass for new storage requests.

Reference the StorageClass in a StatefulSet

To use your new StorageClass, you can reference it in a StatefulSet's volumeClaimTemplates.

When you reference a StorageClass in a StatefulSet's volumeClaimTemplates specification, Kubernetes provides stable storage using PersistentVolumes (PVs). Kubernetes calls the provisioner defined in the StorageClass to create a new storage volume. After the volume is provisioned, Kubernetes automatically creates a PV.

Here is a StatefulSet that references the my-custom-class StorageClass and provisions a 1 gibibyte volume:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: k8s.gcr.io/nginx-slim:0.8
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates: # This is the specification in which you reference the StorageClass
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
      storageClassName: my-custom-class # This field references the existing StorageClass

What's next