GKE on-prem configuration file

Use the following example of the GKE on-prem configuration file to understand how to structure and configure the attributes of your admin and user clusters.

Depending on the installation flow that you choose, you generate one of the following configuration files:

• If you use the gkeadm command, the default filename is admin-ws-config.yaml.

  View the admin-ws-config.yaml example.

  gcp:
    # Path of the allowlisted service account's JSON key file
    whitelistedServiceAccountKeyPath: ""
  # Specify which vCenter resources to use
  vCenter:
    # The credentials and address GKE On-Prem should use to connect to vCenter
    credentials:
      address: ""
      username: ""
      password: ""
    datacenter: ""
    datastore: ""
    cluster: ""
    network: ""
    resourcePool: ""
    # Provide the path to vCenter CA certificate pub key for SSL verification
    caCertPath: ""
  # The URL of the proxy for the jump host
  proxyUrl: ""
  adminWorkstation:
    name: gke-admin-ws-1.3.1-200422-165849
    cpus: 4
    memoryMB: 8192
    # The disk size of the admin workstation in GB. It is recommended to use a disk
    # with at least 50 GB to host images decompressed from the bundle.
    diskGB: 50
    network:
      # The IP allocation mode: 'dhcp' or 'static'
      ipAllocationMode: ""
      # # The host config in static IP mode. Do not include if using DHCP
      # hostConfig:
      #   # The IPv4 static IP address for the admin workstation
      #   ip: ""
      #   # The IP address of the default gateway of the subnet in which the admin workstation
      #   # is to be created
      #   gateway: ""
      #   # The subnet mask of the network where you want to create your admin workstation
      #   netmask: ""
      #   # The list of DNS nameservers to be used by the admin workstation
      #   dns:
      #   - ""
    # The URL of the proxy for the admin workstation
    proxyUrl: ""
    ntpServer: ntp.ubuntu.com

  
  
• If you use the gkectl create-config command (Static IP | DHCP), the default filename is config.yaml.

  View the config.yaml example.

  # Absolute path to a GKE bundle on disk
  bundlepath: ""
  # Specify which vCenter resources to use for deployment
  vcenter:
    # The credentials and address GKE should use to connect to vCenter
    credentials:
      address: ""
      username: ""
      password: ""
    datacenter: ""
    datastore: ""
    cluster: ""
    network: ""
    resourcepool: ""
    # Provide the name for the persistent disk to be used by the deployment (ending
    # in .vmdk). Any directory in the supplied path must be created before deployment.
    # Not required when adding additional user clusters
    datadisk: ""
    # Provide the path to vCenter CA certificate pub key for SSL verification
    cacertpath: ""
  # Specify the proxy configuration.
  proxy:
    # The URL of the proxy
    url: ""
    # The domains and IP addresses excluded from proxying
    noproxy: ""
  # Specify admin cluster settings for a fresh GKE On-Prem deployment. Omit this section
  # and use the --kubeconfig flag when adding a new user cluster to an existing deployment
  admincluster:
    # In-Cluster vCenter configuration
    vcenter:
      # If specified it overwrites the network field in global vcenter configuration
      network: ""
    # # The absolute or relative path to the yaml file to use for static IP allocation.
    # # Do not include if using DHCP
    # ipblockfilepath: ""
    # # Specify pre-defined nodeports if using "manual" load balancer mode
    # manuallbspec:
    #   ingresshttpnodeport: 32527
    #   ingresshttpsnodeport: 30139
    #   controlplanenodeport: 30968
    #   addonsnodeport: 31405
    # Specify the already-existing partition and credentials to use with F5
    bigip:
      # To re-use credentials across clusters we recommend using YAML node anchors.
      # See https://yaml.org/spec/1.2/spec.html#id2785586
      credentials:
        address: ""
        username: ""
        password: ""
      partition: ""
      # # Optionally specify a pool name if using SNAT
      # snatpoolname: ""
    # The VIPs to use for load balancing
    vips:
      # Used to connect to the Kubernetes API
      controlplanevip: ""
      # Shared by all services for ingress traffic
      ingressvip: ""
      # # Used for admin cluster addons (needed for multi cluster features). Must be the same
      # # across clusters
      # addonsvip: ""
    # The Kubernetes service CIDR range for the cluster. Must not overlap with the pod
    # CIDR range
    serviceiprange: 10.96.232.0/24
    # The Kubernetes pod CIDR range for the cluster. Must not overlap with the service
    # CIDR range
    podiprange: 192.168.0.0/16
  # Specify settings when deploying a new user cluster. Used both with a fresh deployment
  # or when adding a new cluster to an existing deployment.
  usercluster:
    # In-Cluster vCenter configuration
    vcenter:
      # If specified it overwrites the network field in global vcenter configuration
      network: ""
    # # The absolute or relative path to the yaml file to use for static IP allocation.
    # # Do not include if using DHCP
    # ipblockfilepath: ""
    # # Specify pre-defined nodeports if using "manual" load balancer mode
    # manuallbspec:
    #   ingresshttpnodeport: 30243
    #   ingresshttpsnodeport: 30879
    #   controlplanenodeport: 30562
    #   addonsnodeport: 0
    # Specify the already-existing partition and credentials to use with F5
    bigip:
      # To re-use credentials across clusters we recommend using YAML node anchors.
      # See https://yaml.org/spec/1.2/spec.html#id2785586
      credentials:
        address: ""
        username: ""
        password: ""
      partition: ""
      # # Optionally specify a pool name if using SNAT
      # snatpoolname: ""
    # The VIPs to use for load balancing
    vips:
      # Used to connect to the Kubernetes API
      controlplanevip: ""
      # Shared by all services for ingress traffic
      ingressvip: ""
      # # Used for admin cluster addons (needed for multi cluster features). Must be the same
      # # across clusters
      # addonsvip: ""
    # A unique name for this cluster
    clustername: ""
    # User cluster control plane nodes must have either 1 or 3 replicas
    masternode:
      cpus: 4
      memorymb: 8192
      # How many machines of this type to deploy
      replicas: 1
    # The number of worker nodes to deploy and their size. Min. 2 replicas
    workernode:
      cpus: 4
      memorymb: 8192
      # How many machines of this type to deploy
      replicas: 3
    # (Optional) Node pools with customizable labels, taints, etc.
    nodepools:
      - name: pool-1
        cpus: 4
        memorymb: 8192
        replicas: 5
      - name: pool-2
        cpus: 8
        memorymb: 16384
        replicas: 3
    # The Kubernetes service CIDR range for the cluster
    serviceiprange: 10.96.0.0/12
    # The Kubernetes pod CIDR range for the cluster
    podiprange: 192.168.0.0/16
    # # Uncomment this section to use OIDC authentication
    # oidc:
    #   issuerurl: ""
    #   kubectlredirecturl: ""
    #   clientid: ""
    #   clientsecret: ""
    #   username: ""
    #   usernameprefix: ""
    #   group: ""
    #   groupprefix: ""
    #   scopes: ""
    #   extraparams: ""
    #   # Set value to string "true" or "false"
    #   usehttpproxy: ""
    #   # # The absolute or relative path to the CA file (optional)
    #   # capath: ""
    # # Optionally provide an additional serving certificate for the API server
    # sni:
    #   certpath: ""
    #   keypath: ""
    # # Specify whether or not to enable the GKE usage metering feature
    # usagemetering:
    #   bigqueryprojectid: ""
    #   # The ID of the BigQuery Dataset in which the usage metering data will be stored
    #   bigquerydatasetid: ""
    #   # The absolute or relative path to the key file for a GCP service account used by
    #   # gke-usage-metering to report to BigQuery
    #   bigqueryserviceaccountkeypath: ""
    #   # Whether or not to enable the consumption-based metering feature
    #   enableconsumptionmetering: false
  # Which load balancer mode to use "Manual" or "Integrated"
  lbmode: Integrated
  # Specify which GCP project to connect your GKE clusters to
  gkeconnect:
    projectid: ""
    # The absolute or relative path to the key file for a GCP service account used to
    # register the cluster
    registerserviceaccountkeypath: ""
    # The absolute or relative path to the key file for a GCP service account used by
    # the GKE connect agent
    agentserviceaccountkeypath: ""
  # Specify which GCP project to connect your logs and metrics to
  stackdriver:
    projectid: ""
    # A GCP region where you would like to store logs and metrics for this cluster.
    clusterlocation: ""
    enablevpc: false
    # The absolute or relative path to the key file for a GCP service account used to
    # send logs and metrics from the cluster
    serviceaccountkeypath: ""
  # Specify Cloud Run configuration
  cloudrun:
    enabled: true
  # # Optionally use a private Docker registry to host GKE images
  # privateregistryconfig:
  #   # Do not include the scheme with your registry address
  #   credentials:
  #     address: ""
  #     username: ""
  #     password: ""
  #   # The absolute or relative path to the CA certificate for this registry
  #   cacertpath: ""
  # The absolute or relative path to the GCP service account key that will be used to
  # pull GKE images
  gcrkeypath: ""
  # Configure kubernetes apiserver audit logging
  cloudauditlogging:
    projectid: ""
    # A GCP region where you would like to store audit logs for this cluster.
    clusterlocation: ""
    # The absolute or relative path to the key file for a GCP service account used to
    # send audit logs from the cluster
    serviceaccountkeypath: ""

  
  

You then modify that generated file to suit your needs before you use it to create your admin cluster and user clusters.