[[["容易理解","easyToUnderstand","thumb-up"],["確實解決了我的問題","solvedMyProblem","thumb-up"],["其他","otherUp","thumb-up"]],[["難以理解","hardToUnderstand","thumb-down"],["資訊或程式碼範例有誤","incorrectInformationOrSampleCode","thumb-down"],["缺少我需要的資訊/範例","missingTheInformationSamplesINeed","thumb-down"],["翻譯問題","translationIssue","thumb-down"],["其他","otherDown","thumb-down"]],["上次更新時間：2025-09-04 (世界標準時間)。"],[],[],null,["# About GKE Volume Populator\n\n[Autopilot](/kubernetes-engine/docs/concepts/autopilot-overview) [Standard](/kubernetes-engine/docs/concepts/choose-cluster-mode)\n\n*** ** * ** ***\n\nThe Google Kubernetes Engine (GKE) Volume Populator can help you automate and streamline the process of preloading data from [Cloud Storage buckets](/storage/docs/buckets) to destination [PersistentVolumeClaims (PVCs)](/kubernetes-engine/docs/concepts/persistent-volumes#persistentvolumeclaims) during [dynamic provisioning](/kubernetes-engine/docs/concepts/persistent-volumes#dynamic_provisioning).\n\n\u003cbr /\u003e\n\nHow GKE Volume Populator works\n------------------------------\n\nGKE Volume Populator leverages the core [Kubernetes Volume Populator](https://kubernetes.io/blog/2022/05/16/volume-populators-beta/) concept. Instead of provisioning an empty volume, the GKE Volume Populator allows a PVC to reference a [`GCPDataSource`](/kubernetes-engine/docs/reference/crds/gcpdatasource) custom resource. This custom resource specifies the source Cloud Storage bucket and the necessary credentials.\n\nWhen you create a PVC with a `dataSourceRef` pointing to a `GCPDataSource` resource, the GKE Volume Populator initiates the data transfer. It copies data from the specified Cloud Storage bucket URI into the underlying persistent storage volume before making the volume available to your Pods.\n\nThis process reduces your need to use manual data transfer scripts or CLI commands, and automates the transfer of large datasets to persistent volumes. GKE Volume Populator supports data transfers between the following source and destination types:\n\n- Cloud Storage to [Parallelstore](/kubernetes-engine/docs/concepts/parallelstore-for-gke)\n- Cloud Storage to [Hyperdisk ML](/kubernetes-engine/docs/concepts/hyperdisk#hyperdisk-ml)\n\nGKE Volume Populator is a GKE managed component that's enabled by default on both Autopilot and Standard clusters. You primarily interact with GKE Volume Populator through the [gcloud CLI](/sdk/gcloud) and [kubectl](https://kubernetes.io/docs/reference/kubectl/) CLI.\n\n### Architecture\n\nThe following diagram shows how data flows from the source storage to the destination\nstorage, and how the [PersistentVolume](/kubernetes-engine/docs/concepts/persistent-volumes) for the destination storage is created by using GKE Volume Populator.\n\n1. You create a PVC that references a [`GCPDataSource`](/kubernetes-engine/docs/reference/crds/gcpdatasource) custom resource.\n2. The GKE Volume Populator detects the PVC and initiates a data transfer Job.\n3. The transfer Job runs on an existing node pool, or a new one is created if node auto-provisioning is enabled.\n4. The transfer Job copies data from the Cloud Storage bucket specified in the `GCPDataSource` resource to the destination storage volume.\n5. After the transfer is complete, the PVC is bound to the destination storage volume, making the data available to the workload Pod.\n\nKey benefits\n------------\n\nThe GKE Volume Populator offers several benefits:\n\n- **Automated data population**: automatically populate volumes with data from Cloud Storage during provisioning, which helps reduce operational overhead.\n- **Seamless data portability**: move data from object storage to high-performance file (Parallelstore) or block storage (Hyperdisk) systems to help optimize for price or performance based on your workload needs.\n- **Simplified workflows**: reduce the need for separate data loading Jobs, or manual intervention to prepare persistent volumes.\n- **Integration with [Identity and Access Management (IAM)](/iam/docs/overview)** : use IAM-based authentication through [Workload Identity Federation for GKE](/kubernetes-engine/docs/concepts/workload-identity) to help ensure secure data transfer with fine-grained access control.\n- **Accelerated AI/ML workloads**: quickly preload large datasets, models, and weights directly into high-performance storage to help speed up training and inference tasks.\n\nUse cases for GKE Volume Populator\n----------------------------------\n\nYou can use GKE Volume Populator to load large training datasets for AI/ML. Imagine you have a multi-terabyte dataset for training a large language model (LLM) stored in a Cloud Storage bucket. Your training Job runs on GKE and requires high I/O performance. Instead of manually copying the data, you can use the GKE Volume Populator to automatically provision a [Parallelstore](/kubernetes-engine/docs/how-to/persistent-volumes/volume-populator) or [Hyperdisk ML](/kubernetes-engine/docs/how-to/persistent-volumes/volume-populator-hdml) volume, and populate it with the dataset from Cloud Storage when the PVC is created. This automated process helps ensure that your training Pods start with immediate, high-speed access to the data.\n\nHere are some more examples where you can use the GKE Volume Populator:\n\n- Pre-caching AI/ML model weights and assets from Cloud Storage into Hyperdisk ML volumes to accelerate model loading times for inference serving.\n- Migrating data from Cloud Storage to persistent volumes for stateful applications requiring performant disk access.\n\nWhat's next\n-----------\n\n- [Automate data transfer to Parallelstore by using GKE Volume Populator](/kubernetes-engine/docs/how-to/persistent-volumes/volume-populator).\n- [Automate data transfer to Hyperdisk ML by using GKE Volume Populator](/kubernetes-engine/docs/how-to/persistent-volumes/volume-populator-hdml).\n- [Troubleshooting GKE Volume Populator data transfer issues](/kubernetes-engine/docs/troubleshooting/troubleshooting-gke-volume-populator)."]]
