Using SMB CSI driver to access SMB volume on Windows Server nodes


This page provides an example of how to use the open source SMB CSI Driver for Kubernetes to access a NetApp Cloud Volumes Service SMB volume on a Google Kubernetes Engine (GKE) cluster with Windows server nodes.

Overview

Server Message Block (SMB) Protocol is a network file sharing protocol used for Microsoft Windows. To use SMB with GKE clusters using Windows Server node pools, you can use the open source SMB CSI Driver for Kubernetes.

Tasks

The following sections walk you through an example of how to access a NetApp Cloud Volumes Service SMB volume on a GKE cluster with Windows Server nodes. The example uses the open source SMB CSI Driver for Kubernetes.

Deploy Active Directory

This task creates an Active Directory. If you already have an Active Directory for use, you can skip this task.

To deploy a self-managed Active Directory, the following instructions use a Google Cloud Marketplace solution to create a new Active Directory domain, with two Active Directory Domain Controllers.

  1. In the Google Cloud console, go to the Microsoft Active Directory Cloud Marketplace page.
  2. Click Launch.
  3. Complete the deployment configuration. Ensure the DNS server is in the same region as the NetApp Cloud Volumes Service SMB volume. Check region availability.
  4. Click Deploy.

If you want to instead use Managed Service for Microsoft Active Directory (Managed Microsoft AD), complete the following steps:

  1. Create a Managed Microsoft AD domain.
  2. Configure domain peering between the domain and NetApp Cloud Volumes Service network.
  3. To perform the Active Directory related tasks, connect to the domain.

Create a private DNS forwarding zone

Create a private DNS forwarding zone that forwards DNS queries to your Active Directory domain controllers.

Update your firewall rule

To allow queries from Cloud DNS to reach your AD connection, in the firewall rule of your AD, add 35.199.192.0/19 as a source IP range to the Source Filter.

For more information, see Security considerations for SMB access of Cloud Volumes Service.

Create an Active Directory connection to Cloud Volumes Service

For instructions, see Creating an AD connection.

Create an SMB volume in Cloud Volumes Service

For instructions, see Creating an SMB volume.

Use the mount target of the new SMB volume as the source value in your PersistentVolume or StorageClass, specified in the following format:

"//SMB_SERVER_NAME/SHARE_NAME"

Mount targets and instructions are available from the Cloud Volumes Service volume listing page and on the individual volume details pages.

Create a cluster with nodes joined to the AD domain

Perform the instructions in Configure Windows Server nodes to automatically join an Active Directory domain.

Install the SMB CSI driver

  1. Install the open source SMB CSI Driver for Kubernetes.
  2. To access a SMB volume from a Pod, create a Secret that encodes the username and password.

    kubectl create secret generic SECRET_NAME \
        --from-literal username="USERNAME" \
        --from-literal password="PASSWORD"
    

    Replace the following:

    • SECRET_NAME: the name of your Secret.
    • USERNAME: the username. The username encoded in the Secret should include a domain name and have the following format: domain\$username. If your SMB share is not part of any domain, domain can be any string.
    • PASSWORD: the password for the user.

Access the SMB volume

To access the SMB volume, you can choose to use one of the following:

Use a StorageClass to access the SMB volume

To access the SMB volume through a StorageClass, perform the following tasks:

  1. Create a StorageClass. Here's a sample manifest file named sc-smb.yaml:

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: smb
    provisioner: smb.csi.k8s.io
    parameters:
      source: "//SMB_SERVER_NAME/SHARE_NAME"
      csi.storage.k8s.io/node-stage-secret-name: "SECRET_NAME"
      csi.storage.k8s.io/node-stage-secret-namespace: "default"
      createSubDir: "false"    # optional: create a sub dir for new volume
    reclaimPolicy: Retain      # only Retain is supported
    volumeBindingMode: Immediate
    mountOptions:
      - dir_mode=0777
      - file_mode=0777
      - uid=1001
      - gid=1001
    

    In this example, we use the mountOptions field, which is optional for Windows Server, but makes this StorageClass work for both Linux and Windows Server.

    Replace the following:

    • SMB_SERVER_NAME: the hostname of the SMB server, including the domain. For example, the hostname for mount path //adserver-faab.cvssmb.com/eager-hungry-skossi is adserver-faab.cvssmb.com.
    • SHARE_NAME: the name of the SMB share. For example, the share name for mount path //adserver-faab.cvssmb.com/eager-hungry-skossi is eager-hungry-skossi. Only use root share for an SMB share. For details, see the related Known issue.
    • SECRET_NAME: the name of the Secret contains the credential to access the SMB volume.
  2. Create the StorageClass resource based on the manifest file:

    kubectl create -f sc-smb.yaml
    
  3. Deploy a Pod that consumes the StorageClass. Here's a sample manifest file named statefulset-smb.yaml. The Pod deployed for this StatefulSet creates a data.txt file in the mounted SMB drive:

    apiVersion: v1
    kind: Service
    metadata:
      name: busybox
      labels:
        app: busybox
    spec:
      ports:
      - port: 80
        name: web
      clusterIP: None
      selector:
        app: busybox
    ---
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: statefulset-smb
      labels:
        app: busybox
    spec:
      serviceName: statefulset-smb
      replicas: 1
      template:
        metadata:
          labels:
            app: busybox
        spec:
          nodeSelector:
            "kubernetes.io/os": windows
          containers:
            - name: statefulset-smb
              image: e2eteam/busybox:1.29
              command:
                - "powershell.exe"
                - "-Command"
                - "while (1) { Add-Content -Encoding Ascii C:\\sc\\smb\\data.txt $(Get-Date -Format u); sleep 1 }"
              volumeMounts:
                - name: smb
                  mountPath: "/sc/smb"
          tolerations:
            - key: "node.kubernetes.io/os"
              operator: "Exists"
              effect: "NoSchedule"
      updateStrategy:
        type: RollingUpdate
      selector:
        matchLabels:
          app: busybox
      volumeClaimTemplates:
        - metadata:
            name: smb
            annotations:
              volume.beta.kubernetes.io/storage-class: smb
          spec:
            accessModes: ["ReadWriteMany"]
            resources:
              requests:
                storage: 10Gi
    
  4. Create the StatefulSet resource based on the manifest file:

    kubectl create -f statefulset-smb.yaml
    

Use a PersistentVolume and PersistentVolumeClaim to access the volume

To access the SMB volume through a PersistentVolume and PersistentVolumeClaim, perform the following tasks:

  1. Create a PersistentVolume. Here's a sample manifest file named pv-smb.yaml:

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: pv-smb
    spec:
      capacity:
        storage: 100Gi
      accessModes:
        - ReadWriteMany
      persistentVolumeReclaimPolicy: Retain
      mountOptions:
        - dir_mode=0777
        - file_mode=0777
        - vers=3.0
      csi:
        driver: smb.csi.k8s.io
        readOnly: false
        volumeHandle: VOLUME_ID
        volumeAttributes:
          source: "//SMB_SERVER_NAME/SHARE_NAME"
        nodeStageSecretRef:
          name: SECRET_NAME
          namespace: default
    

    In this example, we use the mountOptions field, which is optional for Windows Server, but makes this PersistentVolume work for both Linux and Windows Server.

    Replace the following:

    • VOLUME_ID: a unique ID for the volume.
    • SMB_SERVER_NAME: the hostname of the SMB server, including the domain.
    • SHARE_NAME: the name of the SMB share.
    • SECRET_NAME: the name of the Secret that contains the credential to access the SMB volume.
  2. Create a PersistentVolume resource based on the manifest file:

    kubectl create -f pv-smb.yaml
    
  3. Create a PersistentVolumeClaim. Here's a sample manifest file named pvc-smb.yaml:

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: pvc-smb
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 10Gi
      volumeName: pv-smb
      storageClassName: ""
    
  4. Create a PersistentVolumeClaim resource based on the manifest file:

    kubectl create -f pvc-smb.yaml
    
  5. Deploy a Pod that consumes the PersistentVolumeClaim. Here's a sample manifest file named busybox-smb.yaml for a Pod deployment that consumes pvc-smb. This deployment creates a data.txt file in the mounted SMB drive:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: busybox-smb
      labels:
        app: busybox
    spec:
      replicas: 1
      template:
        metadata:
          name: busybox
          labels:
            app: busybox
        spec:
          nodeSelector:
            "kubernetes.io/os": windows
          containers:
            - name: busybox
              image: e2eteam/busybox:1.29
              command:
                - "powershell.exe"
                - "-Command"
                - "while (1) { Add-Content -Encoding Ascii C:\\pv\\pv-smb\\data.txt $(Get-Date -Format u); sleep 1 }"
              volumeMounts:
                - name: smb
                  mountPath: "/pv/pv-smb"
          tolerations:
            - key: "node.kubernetes.io/os"
              operator: "Exists"
              effect: "NoSchedule"
          volumes:
            - name: smb
              persistentVolumeClaim:
                claimName: pvc-smb
      selector:
        matchLabels:
          app: busybox
    
  6. Create a Deployment from the manifest file:

    kubectl apply -f  busybox-smb.yaml
    

Test access to the SMB volume

To verify that you can access the data.txt file on the SMB volume, perform one of the following tasks:

  • Start a powershell session in the container and list the data.txt file:

    kubectl exec POD_NAME -- powershell.exe -c "ls PATH_TO_THE_FILE"
    
  • Open the SMB drive in another VM to confirm that the data.txt file was created successfully in the remote share.

Known issues

Mount error on Windows after reboot

Problem: For example, if \\smb-server\share\test1 is already mounted, you might get an error when mounting volume \\smb-server\share\test2 after the Windows node reboots.

Reason: The source field for both the StorageClass and PersistentVolume should only use root share for one SMB server in one cluster. Also, you should use the volumeMounts.subPath property in the deployment.

Workaround: Only use \\smb-server\share as the source.

For more known issues, see the Known issues page for the open source SMB CSI Driver for Kubernetes.

What's next