ContainerNodePool
Property | Value |
---|---|
Google Cloud Service Name | Kubernetes Engine |
Google Cloud Service Documentation | /kubernetes-engine/docs/ |
Google Cloud REST Resource Name | v1.projects.locations.clusters.nodePools |
Google Cloud REST Resource Documentation | /kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters.nodePools |
Config Connector Resource Short Names | gcpcontainernodepool gcpcontainernodepools containernodepool |
Config Connector Service Name | container.googleapis.com |
Config Connector Resource Fully Qualified Name | containernodepools.container.cnrm.cloud.google.com |
Can Be Referenced by IAMPolicy/IAMPolicyMember | No |
Config Connector Default Average Reconcile Interval In Seconds | 600 |
Custom Resource Definition Properties
Annotations
Fields | |
---|---|
cnrm.cloud.google.com/project-id |
Spec
Schema
autoscaling:
locationPolicy: string
maxNodeCount: integer
minNodeCount: integer
totalMaxNodeCount: integer
totalMinNodeCount: integer
clusterRef:
external: string
name: string
namespace: string
initialNodeCount: integer
location: string
management:
autoRepair: boolean
autoUpgrade: boolean
maxPodsPerNode: integer
namePrefix: string
networkConfig:
additionalNodeNetworkConfigs:
- networkRef:
external: string
name: string
namespace: string
subnetworkRef:
external: string
name: string
namespace: string
additionalPodNetworkConfigs:
- maxPodsPerNode: integer
secondaryPodRange: string
subnetworkRef:
external: string
name: string
namespace: string
createPodRange: boolean
enablePrivateNodes: boolean
podCidrOverprovisionConfig:
disabled: boolean
podIpv4CidrBlock: string
podRange: string
nodeConfig:
advancedMachineFeatures:
threadsPerCore: integer
bootDiskKMSCryptoKeyRef:
external: string
name: string
namespace: string
confidentialNodes:
enabled: boolean
diskSizeGb: integer
diskType: string
ephemeralStorageConfig:
localSsdCount: integer
ephemeralStorageLocalSsdConfig:
localSsdCount: integer
fastSocket:
enabled: boolean
gcfsConfig:
enabled: boolean
guestAccelerator:
- count: integer
gpuDriverInstallationConfig:
gpuDriverVersion: string
gpuPartitionSize: string
gpuSharingConfig:
gpuSharingStrategy: string
maxSharedClientsPerGpu: integer
type: string
gvnic:
enabled: boolean
hostMaintenancePolicy:
maintenanceInterval: string
imageType: string
kubeletConfig:
cpuCfsQuota: boolean
cpuCfsQuotaPeriod: string
cpuManagerPolicy: string
podPidsLimit: integer
labels:
string: string
linuxNodeConfig:
cgroupMode: string
sysctls:
string: string
localNvmeSsdBlockConfig:
localSsdCount: integer
localSsdCount: integer
loggingVariant: string
machineType: string
metadata:
string: string
minCpuPlatform: string
nodeGroupRef:
external: string
name: string
namespace: string
oauthScopes:
- string
preemptible: boolean
reservationAffinity:
consumeReservationType: string
key: string
values:
- string
resourceLabels:
string: string
sandboxConfig:
sandboxType: string
serviceAccountRef:
external: string
name: string
namespace: string
shieldedInstanceConfig:
enableIntegrityMonitoring: boolean
enableSecureBoot: boolean
soleTenantConfig:
nodeAffinity:
- key: string
operator: string
values:
- string
spot: boolean
tags:
- string
taint:
- effect: string
key: string
value: string
workloadMetadataConfig:
mode: string
nodeMetadata: string
nodeCount: integer
nodeLocations:
- string
placementPolicy:
policyNameRef:
external: string
name: string
namespace: string
tpuTopology: string
type: string
resourceID: string
upgradeSettings:
blueGreenSettings:
nodePoolSoakDuration: string
standardRolloutPolicy:
batchNodeCount: integer
batchPercentage: float
batchSoakDuration: string
maxSurge: integer
maxUnavailable: integer
strategy: string
version: string
Fields | |
---|---|
Optional |
Configuration required by cluster autoscaler to adjust the size of the node pool to the current cluster usage. To disable autoscaling, set minNodeCount and maxNodeCount to 0. |
Optional |
Location policy specifies the algorithm used when scaling-up the node pool. "BALANCED" - Is a best effort policy that aims to balance the sizes of available zones. "ANY" - Instructs the cluster autoscaler to prioritize utilization of unused reservations, and reduces preemption risk for Spot VMs. |
Optional |
Maximum number of nodes per zone in the node pool. Must be >= min_node_count. Cannot be used with total limits. |
Optional |
Minimum number of nodes per zone in the node pool. Must be >=0 and <= max_node_count. Cannot be used with total limits. |
Optional |
Maximum number of all nodes in the node pool. Must be >= total_min_node_count. Cannot be used with per zone limits. |
Optional |
Minimum number of all nodes in the node pool. Must be >=0 and <= total_max_node_count. Cannot be used with per zone limits. |
Required |
|
Optional |
Allowed value: The `name` field of a `ContainerCluster` resource. |
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
Optional |
Immutable. The initial number of nodes for the pool. In regional or multi-zonal clusters, this is the number of nodes per zone. Changing this will force recreation of the resource. |
Required |
Immutable. The location (region or zone) of the cluster. |
Optional |
Node management configuration, wherein auto-repair and auto-upgrade is configured. |
Optional |
Whether the nodes will be automatically repaired. |
Optional |
Whether the nodes will be automatically upgraded. |
Optional |
Immutable. The maximum number of pods per node in this node pool. Note that this does not work on node pools which are "route-based" - that is, node pools belonging to clusters that do not have IP Aliasing enabled. |
Optional |
Immutable. Creates a unique name for the node pool beginning with the specified prefix. Conflicts with name. |
Optional |
Networking configuration for this NodePool. If specified, it overrides the cluster-level defaults. |
Optional |
Immutable. We specify the additional node networks for this node pool using this list. Each node network corresponds to an additional interface. |
Optional |
|
Optional |
Immutable. Name of the VPC where the additional interface belongs. |
Optional |
Allowed value: The `selfLink` field of a `ComputeNetwork` resource. |
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
Optional |
Immutable. Name of the subnetwork where the additional interface belongs. |
Optional |
Allowed value: The `selfLink` field of a `ComputeSubnetwork` resource. |
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
Optional |
Immutable. We specify the additional pod networks for this node pool using this list. Each pod network corresponds to an additional alias IP range for the node. |
Optional |
|
Optional |
Immutable. The maximum number of pods per node which use this pod network. |
Optional |
Immutable. The name of the secondary range on the subnet which provides IP address for this pod range. |
Optional |
Immutable. Name of the subnetwork where the additional pod network belongs. |
Optional |
Allowed value: The `selfLink` field of a `ComputeSubnetwork` resource. |
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
Optional |
Immutable. Whether to create a new range for pod IPs in this node pool. Defaults are provided for pod_range and pod_ipv4_cidr_block if they are not specified. |
Optional |
Whether nodes have internal IP addresses only. |
Optional |
Immutable. Configuration for node-pool level pod cidr overprovision. If not set, the cluster level setting will be inherited. |
Required* |
|
Optional |
Immutable. The IP address range for pod IPs in this node pool. Only applicable if create_pod_range is true. Set to blank to have a range chosen with the default size. Set to /netmask (e.g. /14) to have a range chosen with a specific netmask. Set to a CIDR notation (e.g. 10.96.0.0/14) to pick a specific range to use. |
Optional |
Immutable. The ID of the secondary range for pod IPs. If create_pod_range is true, this ID is used for the new range. If create_pod_range is false, uses an existing secondary range with this ID. |
Optional |
Immutable. The configuration of the nodepool. |
Optional |
Immutable. Specifies options for controlling advanced machine features. |
Required* |
Immutable. The number of threads per physical core. To disable simultaneous multithreading (SMT) set this to 1. If unset, the maximum number of threads supported per core by the underlying processor is assumed. |
Optional |
|
Optional |
Allowed value: The `selfLink` field of a `KMSCryptoKey` resource. |
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
Optional |
Immutable. Configuration for the confidential nodes feature, which makes nodes run on confidential VMs. Warning: This configuration can't be changed (or added/removed) after pool creation without deleting and recreating the entire pool. |
Required* |
Immutable. Whether Confidential Nodes feature is enabled for all nodes in this pool. |
Optional |
Immutable. Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB. |
Optional |
Immutable. Type of the disk attached to each node. Such as pd-standard, pd-balanced or pd-ssd. |
Optional |
Immutable. Parameters for the ephemeral storage filesystem. If unspecified, ephemeral storage is backed by the boot disk. |
Required* |
Immutable. Number of local SSDs to use to back ephemeral storage. Uses NVMe interfaces. Each local SSD must be 375 or 3000 GB in size, and all local SSDs must share the same size. |
Optional |
Immutable. Parameters for the ephemeral storage filesystem. If unspecified, ephemeral storage is backed by the boot disk. |
Required* |
Immutable. Number of local SSDs to use to back ephemeral storage. Uses NVMe interfaces. Each local SSD must be 375 or 3000 GB in size, and all local SSDs must share the same size. |
Optional |
Enable or disable NCCL Fast Socket in the node pool. |
Required* |
Whether or not NCCL Fast Socket is enabled. |
Optional |
Immutable. GCFS configuration for this node. |
Required* |
Immutable. Whether or not GCFS is enabled. |
Optional |
Immutable. List of the type and count of accelerator cards attached to the instance. |
Optional |
|
Required* |
Immutable. The number of the accelerator cards exposed to an instance. |
Optional |
Immutable. Configuration for auto installation of GPU driver. |
Required* |
Immutable. Mode for how the GPU driver is installed. |
Optional |
Immutable. Size of partitions to create on the GPU. Valid values are described in the NVIDIA mig user guide (https://docs.nvidia.com/datacenter/tesla/mig-user-guide/#partitioning). |
Optional |
Immutable. Configuration for GPU sharing. |
Required* |
Immutable. The type of GPU sharing strategy to enable on the GPU node. Possible values are described in the API package (https://pkg.go.dev/google.golang.org/api/container/v1#GPUSharingConfig). |
Required* |
Immutable. The maximum number of containers that can share a GPU. |
Required* |
Immutable. The accelerator type resource name. |
Optional |
Immutable. Enable or disable gvnic in the node pool. |
Required* |
Immutable. Whether or not gvnic is enabled. |
Optional |
Immutable. The maintenance policy for the hosts on which the GKE VMs run on. |
Required* |
Immutable. . |
Optional |
The image type to use for this node. Note that for a given image type, the latest version of it will be used. |
Optional |
Node kubelet configs. |
Optional |
Enable CPU CFS quota enforcement for containers that specify CPU limits. |
Optional |
Set the CPU CFS quota period value 'cpu.cfs_period_us'. |
Required* |
Control the CPU management policy on the node. |
Optional |
Controls the maximum number of processes allowed to run in a pod. |
Optional |
The map of Kubernetes labels (key/value pairs) to be applied to each node. These will added in addition to any default label(s) that Kubernetes may apply to the node. |
Optional |
Parameters that can be configured on Linux nodes. |
Optional |
cgroupMode specifies the cgroup mode to be used on the node. |
Optional |
The Linux kernel parameters to be applied to the nodes and all pods running on the nodes. |
Optional |
Immutable. Parameters for raw-block local NVMe SSDs. |
Required* |
Immutable. Number of raw-block local NVMe SSD disks to be attached to the node. Each local SSD is 375 GB in size. |
Optional |
Immutable. The number of local SSD disks to be attached to the node. |
Optional |
Type of logging agent that is used as the default value for node pools in the cluster. Valid values include DEFAULT and MAX_THROUGHPUT. |
Optional |
Immutable. The name of a Google Compute Engine machine type. |
Optional |
Immutable. The metadata key/value pairs assigned to instances in the cluster. |
Optional |
Immutable. Minimum CPU platform to be used by this instance. The instance may be scheduled on the specified or newer CPU platform. |
Optional |
Immutable. Setting this field will assign instances of this pool to run on the specified node group. This is useful for running workloads on sole tenant nodes. |
Optional |
Allowed value: The `name` field of a `ComputeNodeGroup` resource. |
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
Optional |
Immutable. The set of Google API scopes to be made available on all of the node VMs. |
Optional |
|
Optional |
Immutable. Whether the nodes are created as preemptible VM instances. |
Optional |
Immutable. The reservation affinity configuration for the node pool. |
Required* |
Immutable. Corresponds to the type of reservation consumption. |
Optional |
Immutable. The label key of a reservation resource. |
Optional |
Immutable. The label values of the reservation resource. |
Optional |
|
Optional |
The GCE resource labels (a map of key/value pairs) to be applied to the node pool. |
Optional |
Immutable. Sandbox configuration for this node. |
Required* |
Type of the sandbox to use for the node (e.g. 'gvisor'). |
Optional |
|
Optional |
Allowed value: The `email` field of an `IAMServiceAccount` resource. |
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
Optional |
Immutable. Shielded Instance options. |
Optional |
Immutable. Defines whether the instance has integrity monitoring enabled. |
Optional |
Immutable. Defines whether the instance has Secure Boot enabled. |
Optional |
Immutable. Node affinity options for sole tenant node pools. |
Required* |
Immutable. . |
Required* |
|
Required* |
Immutable. . |
Required* |
Immutable. . |
Required* |
Immutable. . |
Required* |
|
Optional |
Immutable. Whether the nodes are created as spot VM instances. |
Optional |
The list of instance tags applied to all nodes. |
Optional |
|
Optional |
List of Kubernetes taints to be applied to each node. |
Optional |
|
Required* |
Effect for taint. |
Required* |
Key for taint. |
Required* |
Value for taint. |
Optional |
The workload metadata configuration for this node. |
Optional |
Mode is the configuration for how to expose metadata to workloads running on the node. |
Optional |
DEPRECATED. Deprecated in favor of mode. NodeMetadata is the configuration for how to expose metadata to the workloads running on the node. |
Optional |
The number of nodes per instance group. This field can be used to update the number of nodes per instance group but should not be used alongside autoscaling. |
Optional |
The list of zones in which the node pool's nodes should be located. Nodes must be in the region of their regional cluster or in the same region as their cluster's zone for zonal clusters. If unspecified, the cluster-level node_locations will be used. |
Optional |
|
Optional |
Immutable. Specifies the node placement policy. |
Optional |
Immutable. If set, refers to the name of a custom resource policy supplied by the user. The resource policy must be in the same project and region as the node pool. If not found, InvalidArgument error is returned. |
Optional |
Allowed value: The `selfLink` field of a `ComputeResourcePolicy` resource. |
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
Optional |
TPU placement topology for pod slice node pool. https://cloud.google.com/tpu/docs/types-topologies#tpu_topologies. |
Required* |
Type defines the type of placement policy. |
Optional |
Immutable. Optional. The name of the resource. Used for creation and acquisition. When unset, the value of `metadata.name` is used as the default. |
Optional |
Specify node upgrade settings to change how many nodes GKE attempts to upgrade at once. The number of nodes upgraded simultaneously is the sum of max_surge and max_unavailable. The maximum number of nodes upgraded simultaneously is limited to 20. |
Optional |
Settings for BlueGreen node pool upgrade. |
Optional |
Time needed after draining entire blue pool. After this period, blue pool will be cleaned up. |
Required* |
Standard rollout policy is the default policy for blue-green. |
Optional |
Number of blue nodes to drain in a batch. |
Optional |
Percentage of the blue pool nodes to drain in a batch. |
Optional |
Soak time after each batch gets drained. |
Optional |
The number of additional nodes that can be added to the node pool during an upgrade. Increasing max_surge raises the number of nodes that can be upgraded simultaneously. Can be set to 0 or greater. |
Optional |
The number of nodes that can be simultaneously unavailable during an upgrade. Increasing max_unavailable raises the number of nodes that can be upgraded in parallel. Can be set to 0 or greater. |
Optional |
Update strategy for the given nodepool. |
Optional |
|
* Field is required when parent field is specified
Status
Schema
conditions:
- lastTransitionTime: string
message: string
reason: string
status: string
type: string
instanceGroupUrls:
- string
managedInstanceGroupUrls:
- string
observedGeneration: integer
observedState:
version: string
operation: string
Fields | |
---|---|
conditions |
Conditions represent the latest available observation of the resource's current state. |
conditions[] |
|
conditions[].lastTransitionTime |
Last time the condition transitioned from one status to another. |
conditions[].message |
Human-readable message indicating details about last transition. |
conditions[].reason |
Unique, one-word, CamelCase reason for the condition's last transition. |
conditions[].status |
Status is the status of the condition. Can be True, False, Unknown. |
conditions[].type |
Type is the type of the condition. |
instanceGroupUrls |
The resource URLs of the managed instance groups associated with this node pool. |
instanceGroupUrls[] |
|
managedInstanceGroupUrls |
List of instance group URLs which have been assigned to this node pool. |
managedInstanceGroupUrls[] |
|
observedGeneration |
ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource. |
observedState |
The observed state of the underlying GCP resource. |
observedState.version |
|
operation |
|
Sample YAML(s)
Basic Node Pool
# Copyright 2020 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: container.cnrm.cloud.google.com/v1beta1
kind: ContainerNodePool
metadata:
labels:
label-one: "value-one"
name: containernodepool-sample-basic
spec:
location: us-east1-c
autoscaling:
minNodeCount: 1
maxNodeCount: 3
nodeConfig:
machineType: n1-standard-1
diskSizeGb: 100
diskType: pd-standard
tags:
- tagone
- tagtwo
preemptible: false
minCpuPlatform: "Intel Haswell"
oauthScopes:
- "https://www.googleapis.com/auth/logging.write"
- "https://www.googleapis.com/auth/monitoring"
guestAccelerator:
- type: "nvidia-tesla-t4"
count: 1
metadata:
disable-legacy-endpoints: "true"
management:
autoRepair: true
autoUpgrade: true
clusterRef:
name: containernodepool-dep-basic
---
apiVersion: container.cnrm.cloud.google.com/v1beta1
kind: ContainerCluster
metadata:
annotations:
cnrm.cloud.google.com/remove-default-node-pool: "true"
name: containernodepool-dep-basic
spec:
location: us-east1-c
initialNodeCount: 1
Sole Tenant Node Pool
# Copyright 2020 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: container.cnrm.cloud.google.com/v1beta1
kind: ContainerNodePool
metadata:
name: containernodepool-sample-soletenancy
spec:
location: us-central1-b
autoscaling:
minNodeCount: 1
maxNodeCount: 3
nodeConfig:
machineType: n1-standard-2
nodeGroupRef:
name: containernodepool-dep-soletenancy
clusterRef:
name: containernodepool-dep-soletenancy
---
apiVersion: compute.cnrm.cloud.google.com/v1beta1
kind: ComputeNodeGroup
metadata:
name: containernodepool-dep-soletenancy
spec:
description: A single sole-tenant node in the us-central1-b zone.
size: 1
nodeTemplateRef:
name: containernodepool-dep-soletenancy
zone: us-central1-b
---
apiVersion: compute.cnrm.cloud.google.com/v1beta1
kind: ComputeNodeTemplate
metadata:
name: containernodepool-dep-soletenancy
spec:
region: us-central1
nodeType: n1-node-96-624
---
apiVersion: container.cnrm.cloud.google.com/v1beta1
kind: ContainerCluster
metadata:
annotations:
cnrm.cloud.google.com/remove-default-node-pool: "true"
name: containernodepool-dep-soletenancy
spec:
description: A cluster using the Compute Engine sole-tenant node.
location: us-central1-b
initialNodeCount: 1