Using SMB CSI driver to access SMB volume on Windows Server nodes

Standard

---

This page provides an example of how to use the open source SMB CSI Driver for Kubernetes to access a NetApp Cloud Volumes Service SMB volume on a Google Kubernetes Engine (GKE) cluster with Windows server nodes.

Note: You can also use the open source SMB CSI Driver for Kubernetes to access a NetApp Cloud Volumes Service SMB volume on a Google Kubernetes Engine (GKE) cluster with Linux nodes starting from GKE version 1.20 for Container-Optimized OS or version 1.19 for Ubuntu.

Overview

Server Message Block (SMB) Protocol is a network file sharing protocol used for Microsoft Windows. To use SMB with GKE clusters using Windows Server node pools, you can use the open source SMB CSI Driver for Kubernetes.

Tasks

The following sections walk you through an example of how to access a NetApp Cloud Volumes Service SMB volume on a GKE cluster with Windows Server nodes. The example uses the open source SMB CSI Driver for Kubernetes.

Note: Support for the open source SMB CSI Driver for Kubernetes will be provided by Google on a best effort basis and is not covered under the GKE SLA.

Deploy a self-managed Active Directory

This task creates a self-managed Active Directory. If you already have an Active Directory for use, you can skip this task.

Note: You cannot use Managed Service for Microsoft Active Directory (Managed Microsoft AD) because peering isn't available between Managed Microsoft AD and NetApp Cloud Volumes Service.

To deploy a self-managed Active Directory, the following instructions use a Google Cloud Marketplace solution to create a new Active Directory domain, with two Active Directory Domain Controllers.

1. In the Google Cloud console, go to the Microsoft Active Directory Cloud Marketplace page.
2. Click Launch.
3. Complete the deployment configuration. Ensure the DNS server is in the same region as the NetApp Cloud Volumes Service SMB volume. Check region availability.
4. Click Deploy.

Create a private DNS forwarding zone

Create a private DNS forwarding zone that forwards DNS queries to your Active Directory domain controllers.

Update your firewall rule

To allow queries from Cloud DNS to reach your AD connection, in the firewall rule of your AD, add 35.199.192.0/19 as a source IP range to the Source Filter.

For more information, see Security considerations for SMB access of Cloud Volumes Service.

Create an Active Directory connection to Cloud Volumes Service

For instructions, see Creating an AD connection.

Create an SMB volume in Cloud Volumes Service

For instructions, see Creating an SMB volume.

Use the mount target of the new SMB volume as the source value in your PersistentVolume or StorageClass, specified in the following format:

"//SMB_SERVER_NAME/SHARE_NAME"

Mount targets and instructions are available from the Cloud Volumes Service volume listing page and on the individual volume details pages.

Note: The CSI Proxy component is available in GKE cluster versions 1.19 and later.

Create a cluster with nodes joined to the AD domain

Perform the instructions in Configure Windows Server nodes to automatically join an Active Directory domain.

Install the SMB CSI driver

1. Install the open source SMB CSI Driver for Kubernetes.

2. To access a SMB volume from a Pod, create a Secret that encodes the username and password.

   kubectl create secret generic SECRET_NAME \
       --from-literal username="USERNAME" \
       --from-literal password="PASSWORD"
   

   Replace the following:

   • SECRET_NAME: the name of your Secret.
   • USERNAME: the username. The username encoded in the Secret should include a domain name and have the following format: domain\$username. If your SMB share is not part of any domain, domain can be any string.
   • PASSWORD: the password for the user.

Access the SMB volume

To access the SMB volume, you can choose to use one of the following:

• StorageClass
• PersistentVolume

Warning: The Delete reclaim policy of a PersistentVolume or a StorageClass is not supported by the open source SMB CSI Driver for Kubernetes.

Use a StorageClass to access the SMB volume

To access the SMB volume through a StorageClass, perform the following tasks:

1. Create a StorageClass. Here's a sample manifest file named sc-smb.yaml:

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: smb
   provisioner: smb.csi.k8s.io
   parameters:
     source: "//SMB_SERVER_NAME/SHARE_NAME"
     csi.storage.k8s.io/node-stage-secret-name: "SECRET_NAME"
     csi.storage.k8s.io/node-stage-secret-namespace: "default"
     createSubDir: "false"    # optional: create a sub dir for new volume
   reclaimPolicy: Retain      # only Retain is supported
   volumeBindingMode: Immediate
   mountOptions:
     - dir_mode=0777
     - file_mode=0777
     - uid=1001
     - gid=1001
   

   In this example, we use the mountOptions field, which is optional for Windows Server, but makes this StorageClass work for both Linux and Windows Server.

   Replace the following:

   • SMB_SERVER_NAME: the hostname of the SMB server, including the domain. For example, the hostname for mount path //adserver-faab.cvssmb.com/eager-hungry-skossi is adserver-faab.cvssmb.com.
   • SHARE_NAME: the name of the SMB share. For example, the share name for mount path //adserver-faab.cvssmb.com/eager-hungry-skossi is eager-hungry-skossi. Only use root share for an SMB share. For details, see the related Known issue.
   • SECRET_NAME: the name of the Secret contains the credential to access the SMB volume.

2. Create the StorageClass resource based on the manifest file:

   kubectl create -f sc-smb.yaml
   

3. Deploy a Pod that consumes the StorageClass. Here's a sample manifest file named statefulset-smb.yaml. The Pod deployed for this StatefulSet creates a data.txt file in the mounted SMB drive:

   apiVersion: v1
   kind: Service
   metadata:
     name: busybox
     labels:
       app: busybox
   spec:
     ports:
     - port: 80
       name: web
     clusterIP: None
     selector:
       app: busybox
   ---
   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
     name: statefulset-smb
     labels:
       app: busybox
   spec:
     serviceName: statefulset-smb
     replicas: 1
     template:
       metadata:
         labels:
           app: busybox
       spec:
         nodeSelector:
           "kubernetes.io/os": windows
         containers:
           - name: statefulset-smb
             image: e2eteam/busybox:1.29
             command:
               - "powershell.exe"
               - "-Command"
               - "while (1) { Add-Content -Encoding Ascii C:\\sc\\smb\\data.txt $(Get-Date -Format u); sleep 1 }"
             volumeMounts:
               - name: smb
                 mountPath: "/sc/smb"
         tolerations:
           - key: "node.kubernetes.io/os"
             operator: "Exists"
             effect: "NoSchedule"
     updateStrategy:
       type: RollingUpdate
     selector:
       matchLabels:
         app: busybox
     volumeClaimTemplates:
       - metadata:
           name: smb
           annotations:
             volume.beta.kubernetes.io/storage-class: smb
         spec:
           accessModes: ["ReadWriteMany"]
           resources:
             requests:
               storage: 10Gi
   

4. Create the StatefulSet resource based on the manifest file:

   kubectl create -f statefulset-smb.yaml
   

Use a PersistentVolume and PersistentVolumeClaim to access the volume

To access the SMB volume through a PersistentVolume and PersistentVolumeClaim, perform the following tasks:

1. Create a PersistentVolume. Here's a sample manifest file named pv-smb.yaml:

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: pv-smb
   spec:
     capacity:
       storage: 100Gi
     accessModes:
       - ReadWriteMany
     persistentVolumeReclaimPolicy: Retain
     mountOptions:
       - dir_mode=0777
       - file_mode=0777
       - vers=3.0
     csi:
       driver: smb.csi.k8s.io
       readOnly: false
       volumeHandle: VOLUME_ID
       volumeAttributes:
         source: "//SMB_SERVER_NAME/SHARE_NAME"
       nodeStageSecretRef:
         name: SECRET_NAME
         namespace: default
   

   In this example, we use the mountOptions field, which is optional for Windows Server, but makes this PersistentVolume work for both Linux and Windows Server.

   Replace the following:

   • VOLUME_ID: a unique ID for the volume.
   • SMB_SERVER_NAME: the hostname of the SMB server, including the domain.
   • SHARE_NAME: the name of the SMB share.
   • SECRET_NAME: the name of the Secret that contains the credential to access the SMB volume.

2. Create a PersistentVolume resource based on the manifest file:

   kubectl create -f pv-smb.yaml
   

3. Create a PersistentVolumeClaim. Here's a sample manifest file named pvc-smb.yaml:

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: pvc-smb
   spec:
     accessModes:
       - ReadWriteMany
     resources:
       requests:
         storage: 10Gi
     volumeName: pv-smb
     storageClassName: ""
   

4. Create a PersistentVolumeClaim resource based on the manifest file:

   kubectl create -f pvc-smb.yaml
   

5. Deploy a Pod that consumes the PersistentVolumeClaim. Here's a sample manifest file named busybox-smb.yaml for a Pod deployment that consumes pvc-smb. This deployment creates a data.txt file in the mounted SMB drive:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: busybox-smb
     labels:
       app: busybox
   spec:
     replicas: 1
     template:
       metadata:
         name: busybox
         labels:
           app: busybox
       spec:
         nodeSelector:
           "kubernetes.io/os": windows
         containers:
           - name: busybox
             image: e2eteam/busybox:1.29
             command:
               - "powershell.exe"
               - "-Command"
               - "while (1) { Add-Content -Encoding Ascii C:\\pv\\pv-smb\\data.txt $(Get-Date -Format u); sleep 1 }"
             volumeMounts:
               - name: smb
                 mountPath: "/pv/pv-smb"
         tolerations:
           - key: "node.kubernetes.io/os"
             operator: "Exists"
             effect: "NoSchedule"
         volumes:
           - name: smb
             persistentVolumeClaim:
               claimName: pvc-smb
     selector:
       matchLabels:
         app: busybox
   

6. Create a Deployment from the manifest file:

   kubectl apply -f  busybox-smb.yaml
   

Test access to the SMB volume

To verify that you can access the data.txt file on the SMB volume, perform one of the following tasks:

• Start a powershell session in the container and list the data.txt file:

  kubectl exec POD_NAME -- powershell.exe -c "ls PATH_TO_THE_FILE"
  

• Open the SMB drive in another VM to confirm that the data.txt file was created successfully in the remote share.

Known issues

Mount error on Windows after reboot

Problem: For example, if \\smb-server\share\test1 is already mounted, you might get an error when mounting volume \\smb-server\share\test2 after the Windows node reboots.

Reason: The source field for both the StorageClass and PersistentVolume should only use root share for one SMB server in one cluster. Also, you should use the volumeMounts.subPath property in the deployment.

Workaround: Only use \\smb-server\share as the source.

For more known issues, see the Known issues page for the open source SMB CSI Driver for Kubernetes.

What's next

• Learn how to deploy a Windows application.
• Learn more about the NetApp Cloud Volumes Service.