Cluster API resources

This topic outlines the Kubernetes Cluster API resources used in GKE on-prem.

Overview

GKE on-prem supports the following Cluster API resources:

  • Cluster
  • Machine
  • MachineDeployment
  • MachineSet
  • MachineClasses

GKE on-prem automatically generates these resources in your clusters during installation; generally, you don't need to interact directly with them. Additionally, GKE on-prem documentation references these resources to illustrate concepts like high-availability.

You can check which Cluster API CustomResourceDefinitions (CRDs) are available in a cluster:

kubectl get crds | grep "cluster.k8s.io"
clusters.cluster.k8s.io
machineclasses.cluster.k8s.io
machinedeployments.cluster.k8s.io
machines.cluster.k8s.io
machinesets.cluster.k8s.io

The following sections discuss the Cluster API resources currently relevant to understanding GKE on-prem.

Cluster

A Cluster resource represents an instance of Kubernetes running on a host. All GKE on-prem clusters have a Cluster resource that describes the cluster.

To see Cluster resources in a cluster, run the following command, where [KUBECONFIG] is the path to the cluster's kubeconfig file:

kubectl get clusters --kubeconfig [KUBECONFIG]

Below is an example of a Cluster resource that you might see in a GKE on-prem user cluster:

apiVersion: cluster.k8s.io/v1alpha1
kind: Cluster
metadata:
  annotations:
    addons.gke.io/loadbalancerip: ""
    bundle.gke.io/target-version:
    bundle.gke.io/version:
    onprem.gke.io/provider: vsphere
  creationTimestamp:
  finalizers:
  - cluster.cluster.k8s.io
  generation:
  name: example-user-cluster
  namespace: default
  resourceVersion: ""
  selfLink: /apis/cluster.k8s.io/v1alpha1/namespaces/default/clusters/example-user-cluster
  uid:
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      -
    serviceDomain: cluster.local
    services:
      cidrBlocks:
      -
  providerSpec:
    value:
      apiVersion: vsphereproviderconfig.k8s.io/v1alpha1
      controlPlaneReplicas:
      controlPlaneVersion:
      kind: VsphereClusterProviderConfig
      loadBalancerIP:
      loadBalancerNodePort:
      metadata:
        creationTimestamp:
      networkSpec:
        dns:
        ntp: ""
        reservedAddresses:
      oidcSpec:
      sniCert:
      vsphereDatacenter:
      vsphereDefaultDatastore:
      vsphereNetwork:
      vsphereResourcePool:
status:
  apiEndpoints:
  - host:
    port:

Machine

In GKE on-prem, a Machine resource is a declarative specification for an individual virtual machine (VM) that lives in vSphere. Each Machine resource has a corresponding Node resource that is created by GKE on-prem based on the Machine's specification. These resources reference the same physical object: a VM living in your vSphere environment.

To see Machine resources in a cluster, run the following command, where [KUBECONFIG] is the path to the cluster's kubeconfig file:

kubectl get machines --kubeconfig [KUBECONFIG]

Below is an example of a Machine resource that you might see in a GKE on-prem cluster:

apiVersion: cluster.k8s.io/v1alpha1
kind: Machine
metadata:
  annotations:
    control-plane-version: ""
    kubelet-version:
    onprem.gke.io/provider: vsphere
    vm-ip-address:
    vsphere-machine-state:
    vsphere-machine-uuid:
  creationTimestamp:
  finalizers:
  - machine.cluster.k8s.io
  generateName: example-cluster-node
  generation:
  labels:
    kubernetes.googleapis.com/cluster-name: example-cluster-node
    kubernetes.googleapis.com/cluster-namespace:
    machine-template-hash: ""
    set: node
  name: example-cluster-node
  namespace: default
  ownerReferences:
  - apiVersion: cluster.k8s.io/v1alpha1
    blockOwnerDeletion:
    controller:
    kind: MachineSet
    name: example-cluster-node
    uid:
  resourceVersion: ""
  selfLink: /apis/cluster.k8s.io/v1alpha1/namespaces/default/machines/example-cluster-node
  uid:
spec:
  metadata:
    creationTimestamp:
  providerSpec:
    value:
      apiVersion: vsphereproviderconfig.k8s.io/v1alpha1
      kind: VsphereMachineProviderConfig
      machineVariables:
        datacenter:
        datastore:
        disk_label:
        disk_size: ""
        memory: ""
        network:
        num_cpus: ""
        resource_pool:
        vm_template:
      metadata:
        creationTimestamp:
      networkSpec:
        address:
        dns:
        ntp: ""
        useIPAM:
      vsphereMachine:
  versions:
    kubelet:
status:
  lastUpdated:
  nodeRef:
    kind: Node
    name:
    uid:

MachineDeployment

A MachineDeployment manages a group of Machines: it specifies the GKE on-prem cluster in which the Machines run, declares their vSphere configuration, and rolls out changes to configurations. You can think of MachineDeployments and Machines being like Deployments and Pods.

To see MachineDeployment resources in a cluster, run the following command, where [KUBECONFIG] is the path to the cluster's kubeconfig file:

kubectl get machinedeployments --kubeconfig [KUBECONFIG]

Below is an example of a MachineDeployment resource that you might see in a GKE on-prem cluster:

apiVersion: cluster.k8s.io/v1alpha1
kind: MachineDeployment
metadata:
  annotations:
    machinedeployment.clusters.k8s.io/revision: "1"
    onprem.gke.io/provider: vsphere
  creationTimestamp:
  generation: 2
  labels:
    kubernetes.googleapis.com/cluster-name: example-cluster
    kubernetes.googleapis.com/cluster-namespace:
    set: node
  name: example-cluster
  namespace: default
  resourceVersion: ""
  selfLink: /apis/cluster.k8s.io/v1alpha1/namespaces/default/machinedeployments/example-cluster
  uid:
spec:
  minReadySeconds: 0
  progressDeadlineSeconds:
  replicas:
  revisionHistoryLimit:
  selector:
    matchLabels:
      kubernetes.googleapis.com/cluster-name: example-cluster
      kubernetes.googleapis.com/cluster-namespace: default
      set: node
  strategy:
    rollingUpdate:
      maxSurge:
      maxUnavailable:
    type: RollingUpdate
  template:
    metadata:
      annotations:
        onprem.gke.io/provider: vsphere
      creationTimestamp: null
      labels:
        kubernetes.googleapis.com/cluster-name: example-cluster
        kubernetes.googleapis.com/cluster-namespace:
        set: node
    spec:
      metadata:
        creationTimestamp:
      providerSpec:
        value:
          apiVersion: vsphereproviderconfig.k8s.io/v1alpha1
          kind: VsphereMachineProviderConfig
          machineVariables:
            datacenter:
            datastore:
            disk_label:
            disk_size: ""
            memory: ""
            network:
            num_cpus: ""
            resource_pool: example-cluster
            vm_template:
          metadata:
            creationTimestamp:
          networkSpec:
            address:
            dns:
            ntp: ""
            useIPAM:
          vsphereMachine:
      versions:
        kubelet:
status:
  availableReplicas:
  observedGeneration:
  readyReplicas:
  replicas:
  updatedReplicas: