建立及管理自訂限制

本頁說明如何使用 Organization Policy Service 自訂限制，限制對下列 Google Cloud 資源執行的特定作業：

  • gkeonprem.googleapis.com/BareMetalAdminCluster
  • gkeonprem.googleapis.com/BareMetalCluster
  • gkeonprem.googleapis.com/BareMetalNodePool
  • gkeonprem.googleapis.com/VmwareAdminCluster
  • gkeonprem.googleapis.com/VmwareCluster
  • gkeonprem.googleapis.com/VmwareNodePool

如要進一步瞭解機構政策，請參閱自訂機構政策。

關於機構政策和限制

Google Cloud 機構政策服務可讓您透過程式以集中方式控管機構的資源。身為機構政策管理員，您可以定義機構政策，也就是一組稱為「限制」的限制，適用於Google Cloud 資源和Google Cloud 資源階層中這些資源的子系。您可以在機構、資料夾或專案層級強制執行機構政策。

機構政策提供各種 Google Cloud 服務的內建代管限制。不過，如要更精細地自訂機構政策中受限的特定欄位，也可以建立自訂限制，並在機構政策中使用這些自訂限制。

政策繼承

根據預設，機構政策會由您強制執行政策的資源子系繼承。舉例來說，如果您對資料夾強制執行政策， Google Cloud 系統會對該資料夾中的所有專案強制執行政策。如要進一步瞭解這項行為及如何變更，請參閱「階層評估規則」。

事前準備

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. Install the Google Cloud CLI.

  5. If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  6. To initialize the gcloud CLI, run the following command: 

    gcloud init

  7. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  8. Make sure that billing is enabled for your Google Cloud project.

  9. Install the Google Cloud CLI.

  10. If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  11. To initialize the gcloud CLI, run the following command: 

    gcloud init
  12. 請確認您知道機構 ID。

    13. 必要的角色

    如要取得管理自訂機構政策所需的權限，請要求管理員為您授予機構資源的機構政策管理員 (roles/orgpolicy.policyAdmin) IAM 角色。如要進一步瞭解如何授予角色，請參閱「管理專案、資料夾和機構的存取權」。

    您或許還可透過自訂角色或其他預先定義的角色取得必要權限。

    建立自訂限制

    自訂限制是在 YAML 檔案中定義，當中包含您要強制執行機構政策的服務所支援的資源、方法、條件和動作。自訂限制的條件是使用一般運算語言 (CEL) 定義。如要進一步瞭解如何使用 CEL 在自訂限制中建構條件，請參閱「建立及管理自訂限制」一文的 CEL 一節。

    如要建立自訂限制，請使用下列格式建立 YAML 檔案：

    name: organizations/ORGANIZATION_ID/customConstraints/CONSTRAINT_NAME
resourceTypes:
- RESOURCE_NAME
methodTypes:
- CREATE
condition: "CONDITION"
actionType: ACTION
displayName: DISPLAY_NAME
description: DESCRIPTION

    更改下列內容：

    • ORGANIZATION_ID：您的機構 ID，例如 123456789。

    • CONSTRAINT_NAME：新自訂限制的名稱。自訂限制條件必須以 custom. 開頭，且只能包含大寫英文字母、小寫英文字母或數字。例如，custom.denyHighNumvCPUs。這個欄位的長度上限為 70 個字元。

    • RESOURCE_NAME：包含要限制物件和欄位的Google Cloud 資源完整名稱。例如：gkeonprem.googleapis.com/gkeonprem.googleapis.com/VmwareNodePool。

    • CONDITION：針對支援服務資源的代表項目編寫的 CEL 條件。這個欄位的長度上限為 1000 個字元。如要進一步瞭解可編寫條件的資源，請參閱「支援的資源」。例如：resource.config.cpus > 4。

    • ACTION：如果符合 condition，應採取的動作。只能是 ALLOW。

    • DISPLAY_NAME：限制條件的易記名稱。這個欄位的長度上限為 200 個字元。

    • DESCRIPTION：違反政策時，要以錯誤訊息形式顯示的限制說明。這個欄位的長度上限為 2000 個字元。

    如要進一步瞭解如何建立自訂限制，請參閱「定義自訂限制」。

    設定自訂限制

    為新的自訂限制建立 YAML 檔案後，您必須進行設定，才能在貴機構的機構政策中使用該檔案。如要設定自訂限制，請使用 gcloud org-policies set-custom-constraint 指令： 
    gcloud org-policies set-custom-constraint CONSTRAINT_PATH
     將 CONSTRAINT_PATH 替換為自訂限制檔案的完整路徑。例如：/home/user/customconstraint.yaml。 完成後，自訂限制就會顯示在 Google Cloud 機構政策清單中，做為機構政策使用。如要確認自訂限制存在，請使用 gcloud org-policies list-custom-constraints 指令：
    gcloud org-policies list-custom-constraints --organization=ORGANIZATION_ID
     將 ORGANIZATION_ID 替換為機構資源的 ID。 詳情請參閱「查看機構政策」。

    強制執行自訂機構政策

    如要強制執行限制，請建立參照該限制的機構政策，然後將該政策套用至 Google Cloud 資源。

    控制台

    1. 前往 Google Cloud 控制台的「Organization policies」(組織政策) 頁面。

      前往「機構政策」

    2. 在專案選擇工具中，選取要設定機構政策的專案。
    3. 在「Organization policies」(機構政策) 頁面上的清單中選取限制條件，即可查看該限制條件的「Policy details」(政策詳情) 頁面。
    4. 如要設定這項資源的機構政策，請按一下「管理政策」。
    5. 在「編輯政策」頁面中，選取「覆寫上層政策」。
    6. 按一下「新增規則」。
    7. 在「Enforcement」(強制執行) 區段中，選取是否要強制執行這項機構政策。
    8. 選用：如要根據標記設定機構政策條件，請按一下「新增條件」。請注意，如果為組織政策新增條件式規則，您必須至少新增一項無條件規則，否則無法儲存政策。詳情請參閱「使用標記設定組織政策」。
    9. 按一下「測試變更」，模擬機構政策的影響。舊版受管理限制不支援政策模擬。詳情請參閱「 使用 Policy Simulator 測試組織政策變更」。
    10. 如要完成並套用機構政策，請按一下「設定政策」。這項政策最多需要 15 分鐘才會生效。

    gcloud

    如要建立含有布林值規則的機構政策，請建立參照限制的政策 YAML 檔案：

          name: projects/PROJECT_ID/policies/CONSTRAINT_NAME
      spec:
        rules:
        - enforce: true

    取代下列項目：

    • PROJECT_ID：要強制執行限制的專案。
    • CONSTRAINT_NAME：您為自訂限制定義的名稱。例如：custom.denyHighNumvCPUs。

    如要強制執行包含限制的機構政策，請執行下列指令： 

        gcloud org-policies set-policy POLICY_PATH

    將 POLICY_PATH 替換為機構政策 YAML 檔案的完整路徑。這項政策最多需要 15 分鐘才會生效。

    測試自訂機構政策

    以下範例說明如何拒絕在 Google Distributed Cloud (僅限軟體) VMware 使用者叢集中，更新超過四個 vCPU 的節點集區。

    開始之前，請務必瞭解下列事項：

    • 您的機構 ID
    • 專案 ID

    建立限制

    1. 將下列檔案儲存為 constraint-node-pool-vcpus.yaml：

      name: organizations/ORGANIZATION_ID/customConstraints/custom.denyHighNumvCPUs
resourceTypes:
- gkeonprem.googleapis.com/VmwareNodePool
methodTypes:
- CREATE
- UPDATE
condition: resource.config.cpus > 4
actionType: DENY
displayName: Node pool vCPU constraint
description: Node pools must have 4 or fewer vCPUs.

      這項限制會將節點集區中的 vCPU 數量限制為 4 個以下。如果節點集區建立或更新時的 vCPU 數量大於 4，系統會拒絕作業。

    2. 套用限制：

      gcloud org-policies set-custom-constraint ~/constraint-node-pool-vcpus.yaml

    3. 確認限制條件是否存在：

      gcloud org-policies list-custom-constraints --organization=ORGANIZATION_ID

      輸出結果會與下列內容相似：

      CUSTOM_CONSTRAINT         ACTION_TYPE  METHOD_TYPES   RESOURCE_TYPES                            DISPLAY_NAME
custom.denyHighNumvCPUs   DENY         CREATE         gkeonprem.googleapis.com/VmwareNodePool   Node pool vCPU constraint
...

    建立政策

    1. 將下列檔案儲存為 policy-deny-node-pool-high-vcpus.yaml：

      name: projects/PROJECT_ID/policies/custom.denyHighNumvCPUs
spec:
  rules:
  - enforce: true

      將 PROJECT_ID 替換為您的專案 ID。

    2. 套用政策：

      gcloud org-policies set-policy ~/policy-deny-node-pool-high-vcpus.yaml

    3. 確認政策存在：

      gcloud org-policies list --project=PROJECT_ID

      輸出結果會與下列內容相似：

      CONSTRAINT               LIST_POLICY    BOOLEAN_POLICY    ETAG
custom.denyHighNumvCPUs  -              SET               COCsm5QGENiXi2E=

    套用政策後，請等待約 15 分鐘， Google Cloud 就會開始強制執行政策。

    測試政策

    嘗試更新違反政策的節點集區。

    1. (選用) 列出使用者叢集，取得叢集名稱和區域：

      gcloud container vmware clusters list \
    --project=PROJECT_ID\
    --location=-

      設定 --location=- 時，表示要列出所有區域的所有使用者叢集。如要縮小清單範圍，請將 --location 設為您建立叢集時指定的區域，或在 GKE On-Prem API 中註冊叢集時指定的區域。

    2. (選用) 列出節點集區，取得要更新的節點集區名稱：

      gcloud container vmware node-pools list \
    --cluster=USER_CLUSTER_NAME \
    --project=PROJECT_ID \
    --location=REGION

      更改下列內容：

      • USER_CLUSTER_NAME：節點集區所在使用者叢集的名稱。

      • REGION：與使用者叢集相關聯的 Google Cloud 區域。

    3. 執行下列指令，嘗試將 vCPU 數量更新為 6：

      gcloud container vmware node-pools update NODE_POOL_NAME \
    --cluster=USER_CLUSTER_NAME  \
    --project=PROJECT_ID \
    --location=REGION \
    --cpus=6

      將 NODE_POOL_NAME 替換為節點集區的名稱。

    作業遭拒。輸出內容如下：

    Operation denied by custom org policy on resource 'projects/PROJECT_ID/locations/REGION/entryGroups/org-policy-test': ["customConstraints/custom.denyHighNumvCPUs"]

    常見用途的自訂機構政策範例

    下表提供一些常見用途的自訂限制語法：

    說明 限制語法
    拒絕版本 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.VersionDeny
    resourceTypes:
    - gkeonprem.googleapis.com/BareMetalCluster
    methodTypes:
    - CREATE
    - UPDATE
    condition: resource.bareMetalVersion.startsWith("1.29")
    actionType: DENY
    displayName: Bare metal user cluster version denied
    description: The bare metal user cluster version is no longer supported
    僅允許在有金鑰時使用 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.AllowAnnotation
    resourceTypes:
    - gkeonprem.googleapis.com/BareMetalAdminCluster
    methodTypes:
    - CREATE
    - UPDATE
    condition: '"allowKey" in resource.annotations'
    actionType: ALLOW
    displayName: Bare metal admin cluster annotation keys allowed
    description: Annotation keys are allowed on bare metal admin clusters

    GKE On-Prem API 支援的資源

    下表列出可在自訂限制中參照的 GKE On-Prem API 資源。

    資源 欄位
    gkeonprem.googleapis.com/BareMetalAdminCluster resource.annotations
    resource.bareMetalVersion
    resource.binaryAuthorization.evaluationMode
    resource.clusterOperations.enableApplicationLogs
    resource.controlPlane.apiServerArgs.argument
    resource.controlPlane.apiServerArgs.value
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.kubeletConfig.registryBurst
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.kubeletConfig.registryPullQps
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.kubeletConfig.serializeImagePullsDisabled
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.nodeConfigs.nodeIp
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.operatingSystem
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.taints.effect
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.taints.key
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.taints.value
    resource.description
    resource.loadBalancer.manualLbConfig.enabled
    resource.loadBalancer.portConfig.controlPlaneLoadBalancerPort
    resource.loadBalancer.vipConfig.controlPlaneVip
    resource.maintenanceConfig.maintenanceAddressCidrBlocks
    resource.name
    resource.networkConfig.islandModeCidr.podAddressCidrBlocks
    resource.networkConfig.islandModeCidr.serviceAddressCidrBlocks
    resource.nodeAccessConfig.loginUser
    resource.nodeConfig.maxPodsPerNode
    resource.osEnvironmentConfig.packageRepoExcluded
    resource.proxy.noProxy
    resource.proxy.uri
    resource.securityConfig.authorization.adminUsers.username
    resource.storage.lvpNodeMountsConfig.path
    resource.storage.lvpNodeMountsConfig.storageClass
    resource.storage.lvpShareConfig.lvpConfig.path
    resource.storage.lvpShareConfig.lvpConfig.storageClass
    resource.storage.lvpShareConfig.sharedPathPvCount
    gkeonprem.googleapis.com/BareMetalCluster resource.adminClusterMembership
    resource.annotations
    resource.bareMetalVersion
    resource.binaryAuthorization.evaluationMode
    resource.clusterOperations.enableApplicationLogs
    resource.controlPlane.apiServerArgs.argument
    resource.controlPlane.apiServerArgs.value
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.kubeletConfig.registryBurst
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.kubeletConfig.registryPullQps
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.kubeletConfig.serializeImagePullsDisabled
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.nodeConfigs.nodeIp
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.operatingSystem
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.taints.effect
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.taints.key
    resource.controlPlane.controlPlaneNodePoolConfig.nodePoolConfig.taints.value
    resource.description
    resource.loadBalancer.bgpLbConfig.addressPools.addresses
    resource.loadBalancer.bgpLbConfig.addressPools.avoidBuggyIps
    resource.loadBalancer.bgpLbConfig.addressPools.manualAssign
    resource.loadBalancer.bgpLbConfig.addressPools.pool
    resource.loadBalancer.bgpLbConfig.asn
    resource.loadBalancer.bgpLbConfig.bgpPeerConfigs.asn
    resource.loadBalancer.bgpLbConfig.bgpPeerConfigs.controlPlaneNodes
    resource.loadBalancer.bgpLbConfig.bgpPeerConfigs.ipAddress
    resource.loadBalancer.bgpLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.kubeletConfig.registryBurst
    resource.loadBalancer.bgpLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.kubeletConfig.registryPullQps
    resource.loadBalancer.bgpLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.kubeletConfig.serializeImagePullsDisabled
    resource.loadBalancer.bgpLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.nodeConfigs.nodeIp
    resource.loadBalancer.bgpLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.operatingSystem
    resource.loadBalancer.bgpLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.taints.effect
    resource.loadBalancer.bgpLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.taints.key
    resource.loadBalancer.bgpLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.taints.value
    resource.loadBalancer.manualLbConfig.enabled
    resource.loadBalancer.metalLbConfig.addressPools.addresses
    resource.loadBalancer.metalLbConfig.addressPools.avoidBuggyIps
    resource.loadBalancer.metalLbConfig.addressPools.manualAssign
    resource.loadBalancer.metalLbConfig.addressPools.pool
    resource.loadBalancer.metalLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.kubeletConfig.registryBurst
    resource.loadBalancer.metalLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.kubeletConfig.registryPullQps
    resource.loadBalancer.metalLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.kubeletConfig.serializeImagePullsDisabled
    resource.loadBalancer.metalLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.nodeConfigs.nodeIp
    resource.loadBalancer.metalLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.operatingSystem
    resource.loadBalancer.metalLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.taints.effect
    resource.loadBalancer.metalLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.taints.key
    resource.loadBalancer.metalLbConfig.loadBalancerNodePoolConfig.nodePoolConfig.taints.value
    resource.loadBalancer.portConfig.controlPlaneLoadBalancerPort
    resource.loadBalancer.vipConfig.controlPlaneVip
    resource.loadBalancer.vipConfig.ingressVip
    resource.maintenanceConfig.maintenanceAddressCidrBlocks
    resource.name
    resource.networkConfig.advancedNetworking
    resource.networkConfig.islandModeCidr.podAddressCidrBlocks
    resource.networkConfig.islandModeCidr.serviceAddressCidrBlocks
    resource.networkConfig.multipleNetworkInterfacesConfig.enabled
    resource.networkConfig.srIovConfig.enabled
    resource.nodeAccessConfig.loginUser
    resource.nodeConfig.containerRuntime
    resource.nodeConfig.maxPodsPerNode
    resource.osEnvironmentConfig.packageRepoExcluded
    resource.proxy.noProxy
    resource.proxy.uri
    resource.securityConfig.authorization.adminUsers.username
    resource.storage.lvpNodeMountsConfig.path
    resource.storage.lvpNodeMountsConfig.storageClass
    resource.storage.lvpShareConfig.lvpConfig.path
    resource.storage.lvpShareConfig.lvpConfig.storageClass
    resource.storage.lvpShareConfig.sharedPathPvCount
    resource.upgradePolicy.policy
    gkeonprem.googleapis.com/BareMetalNodePool resource.annotations
    resource.displayName
    resource.name
    resource.nodePoolConfig.kubeletConfig.registryBurst
    resource.nodePoolConfig.kubeletConfig.registryPullQps
    resource.nodePoolConfig.kubeletConfig.serializeImagePullsDisabled
    resource.nodePoolConfig.nodeConfigs.nodeIp
    resource.nodePoolConfig.operatingSystem
    resource.nodePoolConfig.taints.effect
    resource.nodePoolConfig.taints.key
    resource.nodePoolConfig.taints.value
    resource.upgradePolicy.parallelUpgradeConfig.concurrentNodes
    resource.upgradePolicy.parallelUpgradeConfig.minimumAvailableNodes
    gkeonprem.googleapis.com/VmwareAdminCluster resource.addonNode.autoResizeConfig.enabled
    resource.annotations
    resource.antiAffinityGroups.aagConfigDisabled
    resource.authorization.viewerUsers.username
    resource.autoRepairConfig.enabled
    resource.bootstrapClusterMembership
    resource.controlPlaneNode.cpus
    resource.controlPlaneNode.memory
    resource.controlPlaneNode.replicas
    resource.description
    resource.enableAdvancedCluster
    resource.imageType
    resource.loadBalancer.f5Config.address
    resource.loadBalancer.f5Config.partition
    resource.loadBalancer.f5Config.snatPool
    resource.loadBalancer.manualLbConfig.addonsNodePort
    resource.loadBalancer.manualLbConfig.controlPlaneNodePort
    resource.loadBalancer.manualLbConfig.ingressHttpNodePort
    resource.loadBalancer.manualLbConfig.ingressHttpsNodePort
    resource.loadBalancer.manualLbConfig.konnectivityServerNodePort
    resource.loadBalancer.metalLbConfig.enabled
    resource.loadBalancer.vipConfig.addonsVip
    resource.loadBalancer.vipConfig.controlPlaneVip
    resource.name
    resource.networkConfig.dhcpIpConfig.enabled
    resource.networkConfig.haControlPlaneConfig.controlPlaneIpBlock.gateway
    resource.networkConfig.haControlPlaneConfig.controlPlaneIpBlock.ips.hostname
    resource.networkConfig.haControlPlaneConfig.controlPlaneIpBlock.ips.ip
    resource.networkConfig.haControlPlaneConfig.controlPlaneIpBlock.netmask
    resource.networkConfig.hostConfig.dnsSearchDomains
    resource.networkConfig.hostConfig.dnsServers
    resource.networkConfig.hostConfig.ntpServers
    resource.networkConfig.podAddressCidrBlocks
    resource.networkConfig.serviceAddressCidrBlocks
    resource.networkConfig.staticIpConfig.ipBlocks.gateway
    resource.networkConfig.staticIpConfig.ipBlocks.ips.hostname
    resource.networkConfig.staticIpConfig.ipBlocks.ips.ip
    resource.networkConfig.staticIpConfig.ipBlocks.netmask
    resource.networkConfig.vcenterNetwork
    resource.onPremVersion
    resource.platformConfig.requiredPlatformVersion
    resource.vcenter.address
    resource.vcenter.caCertData
    resource.vcenter.cluster
    resource.vcenter.datacenter
    resource.vcenter.dataDisk
    resource.vcenter.datastore
    resource.vcenter.folder
    resource.vcenter.resourcePool
    resource.vcenter.storagePolicyName
    gkeonprem.googleapis.com/VmwareCluster resource.adminClusterMembership
    resource.annotations
    resource.antiAffinityGroups.aagConfigDisabled
    resource.authorization.adminUsers.username
    resource.autoRepairConfig.enabled
    resource.binaryAuthorization.evaluationMode
    resource.controlPlaneNode.autoResizeConfig.enabled
    resource.controlPlaneNode.cpus
    resource.controlPlaneNode.memory
    resource.controlPlaneNode.replicas
    resource.controlPlaneNode.vsphereConfig.datastore
    resource.controlPlaneNode.vsphereConfig.storagePolicyName
    resource.dataplaneV2.advancedNetworking
    resource.dataplaneV2.dataplaneV2Enabled
    resource.dataplaneV2.forwardMode
    resource.dataplaneV2.windowsDataplaneV2Enabled
    resource.description
    resource.disableBundledIngress
    resource.enableAdvancedCluster
    resource.enableControlPlaneV2
    resource.loadBalancer.f5Config.address
    resource.loadBalancer.f5Config.partition
    resource.loadBalancer.f5Config.snatPool
    resource.loadBalancer.manualLbConfig.controlPlaneNodePort
    resource.loadBalancer.manualLbConfig.ingressHttpNodePort
    resource.loadBalancer.manualLbConfig.ingressHttpsNodePort
    resource.loadBalancer.manualLbConfig.konnectivityServerNodePort
    resource.loadBalancer.metalLbConfig.addressPools.addresses
    resource.loadBalancer.metalLbConfig.addressPools.avoidBuggyIps
    resource.loadBalancer.metalLbConfig.addressPools.manualAssign
    resource.loadBalancer.metalLbConfig.addressPools.pool
    resource.loadBalancer.vipConfig.controlPlaneVip
    resource.loadBalancer.vipConfig.ingressVip
    resource.name
    resource.networkConfig.controlPlaneV2Config.controlPlaneIpBlock.gateway
    resource.networkConfig.controlPlaneV2Config.controlPlaneIpBlock.ips.hostname
    resource.networkConfig.controlPlaneV2Config.controlPlaneIpBlock.ips.ip
    resource.networkConfig.controlPlaneV2Config.controlPlaneIpBlock.netmask
    resource.networkConfig.dhcpIpConfig.enabled
    resource.networkConfig.hostConfig.dnsSearchDomains
    resource.networkConfig.hostConfig.dnsServers
    resource.networkConfig.hostConfig.ntpServers
    resource.networkConfig.podAddressCidrBlocks
    resource.networkConfig.serviceAddressCidrBlocks
    resource.networkConfig.staticIpConfig.ipBlocks.gateway
    resource.networkConfig.staticIpConfig.ipBlocks.ips.hostname
    resource.networkConfig.staticIpConfig.ipBlocks.ips.ip
    resource.networkConfig.staticIpConfig.ipBlocks.netmask
    resource.networkConfig.vcenterNetwork
    resource.onPremVersion
    resource.storage.vsphereCsiDisabled
    resource.upgradePolicy.controlPlaneOnly
    resource.vcenter.caCertData
    resource.vcenter.cluster
    resource.vcenter.datacenter
    resource.vcenter.datastore
    resource.vcenter.folder
    resource.vcenter.resourcePool
    resource.vcenter.storagePolicyName
    resource.vmTrackingEnabled
    gkeonprem.googleapis.com/VmwareNodePool resource.annotations
    resource.config.bootDiskSizeGb
    resource.config.cpus
    resource.config.enableLoadBalancer
    resource.config.image
    resource.config.imageType
    resource.config.memoryMb
    resource.config.replicas
    resource.config.taints.effect
    resource.config.taints.key
    resource.config.taints.value
    resource.config.vsphereConfig.datastore
    resource.config.vsphereConfig.hostGroups
    resource.config.vsphereConfig.tags.category
    resource.config.vsphereConfig.tags.tag
    resource.displayName
    resource.name
    resource.nodePoolAutoscaling.maxReplicas
    resource.nodePoolAutoscaling.minReplicas
    resource.onPremVersion

    後續步驟