Quickstart: Access Cloud Storage buckets with the FUSE CSI driver

Quickstart: Access Cloud Storage buckets with the FUSE CSI driver

Autopilot

---

In this quickstart guide, you learn how to access Cloud Storage buckets from within a Google Kubernetes Engine (GKE) application as if they were local file systems. This quickstart provides a sample Kubernetes manifest to create a Pod that mounts the bucket, and shows you how to interact with the bucket's contents using the Pod's file system.

This page is for Developers, Storage specialists, and anyone else who wants to provision and manage cloud resources, and deploy workloads. To learn more about common roles, see Common GKE Enterprise user roles and tasks.

Before reading this page, ensure that you're familiar with Kubernetes and Cloud Storage FUSE CSI driver.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the GKE and Cloud Storage APIs.

   Enable the APIs

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

6. Make sure that billing is enabled for your Google Cloud project.

7. Enable the GKE and Cloud Storage APIs.

   Enable the APIs

8. Make sure that you have the following role or roles on the project: roles/container.admin, roles/storage.admin, roles/storage.objectUser, roles/iam.serviceAccountUser

   Check for the roles

   1. In the Google Cloud console, go to the IAM page.

      Go to IAM
   2. Select the project.

   3. In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.

   4. For all rows that specify or include you, check the Role colunn to see whether the list of roles includes the required roles.

   Grant the roles

   1. In the Google Cloud console, go to the IAM page.

      Go to IAM
   2. Select the project.
   3. Click Grant access.

   4. In the New principals field, enter your user identifier. This is typically the email address for a Google Account.

   5. In the Select a role list, select a role.
   6. To grant additional roles, click Add another role and add each additional role.
   7. Click Save.

Create a GKE Autopilot cluster

1. In the Google Cloud console, go to the GKE Clusters page.

   Go to Clusters

2. Click Create.

3. In the Create cluster dialog, click Configure for the Autopilot mode.

4. Under Cluster basics, do the following:

   1. In the Name field, enter a name for the cluster.

   2. Keep the default values for the rest of the settings.

5. To create the cluster, click Create.

Create a Cloud Storage bucket

1. In the Google Cloud console, go to the Cloud Storage Buckets page.
   

   Go to Buckets

2. Click Create.

3. On the Create a bucket page, in the Get started section, do the following:

   1. Enter a globally unique name for your bucket that meets the bucket naming requirements.

   2. Keep the default values for the rest of the settings.

4. To create the bucket, click Create.

Upload objects to your Cloud Storage bucket

1. In the Google Cloud console, go to the Cloud Storage Buckets page.
   

   Go to Buckets

2. In the list of buckets, click the name of the bucket that you just created.

3. In the Objects tab for the bucket, do either of the following:

   • Drag files from your desktop or file manager to the main pane in the Google Cloud console.

   • Click the Upload Files button. In the dialog that appears, select the files that you want to upload, and click Open.

   Note: If you're using the Chrome browser, folder uploads are also supported.

Configure access to your Cloud Storage bucket

To make sure your Cloud Storage bucket is accessible by your GKE cluster, follow the steps in Configure access to Cloud Storage buckets.

Deploy a Pod to access your Cloud Storage bucket by using FUSE

1. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

   A Cloud Shell session opens inside a frame on the console.

2. To set the default project, use the project ID of the project where you want to deploy the Pod:

   gcloud config set project PROJECT_ID
   

   Replace PROJECT_ID with your project ID.

3. Create and apply the Pod manifest with the following content:

   cat << EOF | kubectl apply -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: gcsfuse-test
     namespace: default
     annotations:
       gke-gcsfuse/volumes: "true"
   spec:
     terminationGracePeriodSeconds: 60
     containers:
     - image: busybox
       name: busybox
       command: ["sleep"]
       args: ["infinity"]
       volumeMounts:
       - name: gcsfuse-test
         mountPath: /data
         readOnly: true
     serviceAccountName: default
     volumes:
     - name: gcsfuse-test
       csi:
         driver: gcsfuse.csi.storage.gke.io
         volumeAttributes:
           bucketName: BUCKET_NAME
           mountOptions: "implicit-dirs"
   EOF
   

   • Replace BUCKET_NAME with your Cloud Storage bucket name.
   • If you're using a different namespace or a service account, replace the namespace and serviceAccountName fields accordingly.

   The content of this manifest starts a Pod called gcsfuse-test that mounts the bucket on its /data path.

4. Verify that the Pod is running:

   kubectl get pod gcsfuse-test
   

   The output is similar to the following:

   NAME           READY   STATUS    RESTARTS   AGE
   gcsfuse-test   2/2     Running   0          12s
   

   The Cloud Storage FUSE CSI driver attaches a sidecar container in your Pod to manage interactions with Cloud Storage.

   If the Pod isn't running, you can fetch the event logs to help you diagnose the issue by running the kubectl describe pod gcsfuse-test command. For more information, see Troubleshooting on GitHub.

5. After the Pod is running, you can explore its file system:

   kubectl exec -it gcsfuse-test -- find /data
   

   Success: You've successfully accessed your Cloud Storage bucket as a local file system!

Clean up

To avoid incurring charges to your Google Cloud account for the resources that you used in this tutorial, delete the Pod:

kubectl delete pod gcsfuse-test

What's next

• Learn how the Cloud Storage FUSE CSI driver for GKE works..
• Learn more about the capabilities of Cloud Storage FUSE.