This page shows how to create an admin workstation that you can use to create GKE on-prem clusters.
In this topic, you create an admin workstation that has a static IP address. That is consistent with the basic installation journey, which creates clusters that use static IP addresses.
If you want to create an admin workstation that uses Dynamic Host Configuration Protocol (DHCP) to get its IP address, see Creating an admin workstation using DHCP.
The steps in this topic use the
gkeadm command-line tool, which is available
for Linux, Windows 10, Windows Server 2019, and macOS Catalina.
gkeadm command-line tool and make it executable:
gsutil cp gs://gke-on-prem-release-public/gkeadm/1.5.1-gke.8/linux/gkeadm ./ chmod +x gkeadm
gkeadm command-line tool and make it executable:
gsutil cp gs://gke-on-prem-release-public/gkeadm/1.5.1-gke.8/darwin/gkeadm ./ chmod +x gkeadm
gkeadm command-line tool:
gsutil cp gs://gke-on-prem-release-public/gkeadm/1.5.1-gke.8/windows/gkeadm.exe ./
Generating a template for your configuration file
To create an admin workstation, the
gkeadm tool requires a configuration file.
To generate a template for your configuration file:
./gkeadm create config
The output is a file named
The following fields in your configuration file are populated with default or generated values. For eample:
adminWorkstation: name: gke-admin-ws-200617-113711 cpus: 4 memoryMB: 8192 diskGB: 50 dataDiskName: gke-on-prem-admin-workstation-data-disk/gke-admin-ws-200617-113711-data-disk.vmdk dataDiskMB: 512 ntpServer: ntp.ubuntu.com
Keep all of the default and generated values.
Notice the generated value of
dataDiskName. This virtual machine disk (VMDK)
is created for you. Later, if you manually delete your admin workstation, the
VMDK is automatically deleted.
Filling in the rest of your configuration file
In your configuration file, enter field values as described in this section.
String. The path of the JSON key file for your component access service account. This is the account that GKE on-prem uses to download components from Container Registry. For information on creating a JSON key file, see Service accounts and keys.
gkeadm creates your admin workstation, it activates your component access
service account on the admin workstation.
gcp: whitelistedServiceAccountKeyPath: "my-key-folder/component-access-key.json"
String. The IP address or the hostname of your vCenter server.
Before you fill in the
address field, download and inspect
the serving certificate of your vCenter server. Enter the following command to
download the certificate and save it to a file named
true | openssl s_client -connect [VCENTER_IP]:443 -showcerts 2>/dev/null | sed -ne '/-BEGIN/,/-END/p' > vcenter.pem
where [VCENTER_IP] is the IP address of your vCenter Server.
Open the certificate file to see the Subject Common Name and the Subject Alternative Name:
openssl x509 -in vcenter.pem -text -noout
The output shows the
Subject Common Name (CN). This might be an IP address, or
it might be a hostname. For example:
Subject: ... CN = 203.0.113.100
Subject: ... CN = my-host.my-domain.example
The output might also include one or more DNS names under
Subject Alternative Name:
X509v3 Subject Alternative Name: DNS:vcenter.my-domain.example
Subject Common Name or one of the DNS names under
Subject Alternative Name to use as the value of
in your configuration file. For example:
vCenter: credentials: address: "203.0.113.1"
vCenter: credentials: address: "my-host.my-domain.example"
String. A vCenter Server user account. The user account should have the Administrator role or equivalent privileges. See vSphere requirements.
vCenter: credentials: username: "email@example.com"
String. The password for the vCenter Server user account. For example:
vCenrer: credentials: password: "#STyZ2T#Ko2o"
String. The name of your vSphere datacenter. For example:
vCenter: datacenter: "MY-DATACENTER"
String. The name of your vSphere datastore. For example:
vCenter: datastore: "MY-DATASTORE"
String. The name of your vSphere cluster. For example:
vCenter: cluster: "MY-CLUSTER"
String. The name of the vSphere network where you want to create your admin workstation.
If the name contains a special character, you must use an escape sequence for it.
|Special characters||Escape sequence|
|Percent sign (
If the network name is not unique, it is possible to specify a path to the
network, such as
vCenter: network: "MY-VM-NETWORK"
String. If you are using a non-default resource pool, provide the name of your vSphere resource pool. For example:
vCenter: resourcePool: "MY-POOL"
If you are using the default resource pool, provide the following value:
vCenter: resourcePool: "[MY_CLUSTER]/Resources"
where [MY_CLUSTER] is the name of your vSphere cluster.
String. When a client, like GKE on-prem, sends a request to your vCenter server, the server must prove its identity to the client by presenting a certificate or a certificate bundle. To verify the certificate or bundle, GKE on-prem must have the root certificate in the chain of trust.
vCenter.caCertPath to the path of the root certificate. For example:
vCenter: caCertPath: "/usr/local/google/home/me/certs/vcenter-ca-cert.pem"
Your VMware installation has a certificate authority (CA) that issues a certificate to your vCenter server. The root certificate in the chain of trust is a self-signed certificate created by VMware.
If you do not want to use the VMWare CA, which is the default, you can configure VMware to use a different certificate authority.
If your vCenter server uses a certificate issued by the default VMware CA, download the certificate as follows:
curl -k "https://[SERVER_ADDRESS]/certs/download.zip" > download.zip
where [SERVER_ADDRESS] is the address of your vCenter server.
unzip command and unzip the certificate file:
sudo apt-get install unzip unzip downloads.zip
If the unzip command doesn't work the first time, enter the command again.
Find the certificate file in
String: If the machine you are using to run
gkeadm uses a proxy server for access to
the internet, set this field to the URL of the proxy server. For example:
String. Set this to
"static". For example:
adminWorkstation: network: ipAllocationMode: "static"
String. An IP address of your choice for your admin workstation. For example:
adminWorkstation: network: hostconfig: ip: "172.16.5.1"
String. The IP address of the default gateway for the network that contains your admin workstation. For example:
adminWorkstation: network: hostconfig: gateway: "172.16.6.254"
String. The netmask for the network that contains your admin workstation. For example:
adminWorkstation: network: hostconfig: netmask: "255.255.248.0"
String array. An array of IP addresses for DNS servers that your admin workstation can use. For example:
adminWorkstation: network: hostconfig: dns: - "172.16.255.1" - "172.16.255.2"
String. If your network is behind a proxy server, and you want both your admin
workstation and your GKE on-prem clusters to use the same proxy
server, then set
adminworkstation.proxyURL to the URL of the proxy server.
adminworkstation: proxyUrl: "http://aw-proxy.example"
String. The hostname or IP address of the Network Time Protocol server that your admin workstation should use. For example:
adminWorkstation: ntpServer: "126.96.36.199"
If you leave this empty, GKE on-prem uses
Here is an example of a completed
gcp: whitelistedServiceAccountKeyPath: "my-key-folder/component-access-key.json" vCenter: credentials: address: "203.0.113.1" username: "administrator.vsphere.local" password: "#STyZ2T#Ko2o" datacenter: "MY-DATACENTER" datastore: "MY-DATASTORE" cluster: "MY-CLUSTER" network: "MY-VM-NETWORK" resourcePool: "MY-POOL" caCertPath: "/usr/local/google/home/me/certs/the-root.cert" proxyUrl: "" adminWorkstation: name: "gke-admin-ws-200617-113711" cpus: 4 memoryMB: 8192 diskGB: 50 dataDiskName: "gke-on-prem-admin-workstation-data-disk/gke-admin-ws-200617-113711-data-disk.vmdk" dataDiskMB: 512 network: ipAllocationMode: "static" hostConfig: ip: "172.16.5.1" gateway: "172.16.6.254" netmask: "255.255.248.0" dns: - "172.16.255.1" - "172.16.255.2" proxyUrl: "" ntpServer: "ntp.ubuntu.com"
Creating your admin workstation
In this step, you use
gkeadm to create an admin workstation and a set of
service accounts. If you prefer to create and configure your service accounts
manually, you can omit the
Create your admin workstation and a set of service accounts:
./gkeadm create admin-workstation --auto-create-service-accounts
The output gives detailed information about the creation of your admin
workstation. For example, you can see that
gkeadm performed these tasks:
Ran preflight checks.
Downloaded an OVA image and uploaded it as a vSphere template.
Created an SSH key.
Created an admin workstation VM.
gkectland Docker versions installed on the admin workstation.
Printed information about the NTP server on the admin workstation.
Created service accounts and their JSON key files. These service accounts are members of the project that is the parent of your component access service account.
Assigned the appropriate IAM roles to the service accounts. These roles are granted on the project that is the parent of your component access service account.
Copied JSON key files for the service accounts to the admin workstation.
Copied the CA certificate file for your vCenter server to the admin workstation.
user-cluster.yamlon the admin workstation.n Later you will use these configuration files to create clusters.
... - Validation Category: vCenter - [SUCCESS] Credentials - [SUCCESS] Version - [SUCCESS] Datacenter - [SUCCESS] Datastore - [SUCCESS] Resource Pool - [SUCCESS] Folder - [SUCCESS] Network - [SUCCESS] Datadisk All validation results were SUCCESS. ****************************************** Admin workstation VM successfully created: - Name: my-admin-workstation - IP: 172.16.5.1 - SSH Key: /usr/local/google/home/me/.ssh/gke-admin-workstation ****************************************** Printing gkectl and docker versions on admin workstation... gkectl version gkectl 1.4.0-gke.13 (git-b39331512) docker version Client: Version: 19.03.2 API version: 1.40 Go version: go1.12.9 Git commit: 6a30dfca03 Built: Mon Sep 30 22:47:02 2019 OS/Arch: linux/amd64 Experimental: false Getting ... service account...< Creating other service accounts and JSON key files... - connect-agent-sa-2007081316 - connect-register-sa-2007081316 - log-mon-sa-2007081316 Enabling APIs... - project my-project - serviceusage.googleapis.com - iam.googleapis.com - cloudresourcemanager.googleapis.com Configuring IAM roles for service accounts... - my-component-access-sa for project my-project - roles/serviceusage.serviceUsageViewer - roles/iam.serviceAccountCreator - roles/iam.roleViewer - connect-agent-sa-2007081316 for project my-project - roles/gkehub.connect - connect-register-sa-2007081316 for project my-project - roles/gkehub.admin - roles/serviceusage.serviceUsageViewer - log-mon-sa-2007081316 for project my-project - roles/stackdriver.resourceMetadata.writer - roles/logging.logWriter - roles/monitoring.metricWriter Copying files to admin workstation... - /usr/local/google/home/me/certs/vcenter-ca-cert.pem - /usr/local/google/home/me/service-account-keys/component-access-key.json - connect-agent-sa-2007081316.json - connect-register-sa-2007081316.json - log-mon-sa-2007081316.json Preparing "admin-cluster.yaml" for gkectl... Preparing "user-cluster.yaml" for gkectl... ******************************************************************** Admin workstation is ready to use. Admin workstation information saved to /usr/local/google/home/me/my-admin-workstation This file is required for future upgrades SSH into the admin workstation with the following command: ssh -i /usr/local/google/home/me/.ssh/gke-admin-workstation firstname.lastname@example.org ********************************************************************
Getting an SSH connection to your admin workstation
Near the end of the preceding output there is a command you can use to SSH into your admin workstation. Enter that command now. For example:
ssh -i /usr/local/google/home/me/.ssh/gke-admin-workstation email@example.com
List the files on your admin workstation:
In the output, you can see two cluster configuration files, your CA certificate file, and the JSON key files for your service accounts:
admin-cluster.yaml connect-agent-sa-2007081316.json connect-register-sa-2007081316.json log-mon-sa-2007081316.json user-cluster.yaml vcenter-ca-cert.pem component-access-key.json
Inspecting the cluster configuration files
On your admin workstation,
gkeadm created two cluster configuration
user-cluster.yaml You use these files later
to create your admin and user clusters.
Use a text editor to open
admin-cluster.yaml. Notice that many of the
fields are already populated with values that you provided in your
admin-ws-config.yaml file. For example:
apiVersion: v1 kind: AdminCluster bundlePath: /var/lib/gke/bundles/gke-onprem-vsphere-1.4.0-gke.13-full.tgz vCenter: address: 203.0.113.1 datacenter: MY-DATACENTER cluster: MY-CLUSTER resourcePool: MY-POOL datastore: MY-DATASTORE caCertPath: vcenter-ca-cert.pem credentials: username: firstname.lastname@example.org password: STyZ2T#Ko2o ... gcrKeyPath: component-access-key.json stackdriver: projectID: my-project-id serviceAccountKeyPath: log-mon-sa-2007081316.json
user-cluster.yaml. Notice that many of the fields are already
populated with values that you provided in your
apiVersion: v1 kind: UserCluster gkeOnPremVersion: 1.4.0-gke.13 ... stackdriver: projectID: my-project-id serviceAccountKeyPath: log-mon-sa-2007081316.json gkeConnect: projectID: my-project-id registerServiceAccountKeyPath: connect-register-sa-2007081316.json agentServiceAccountKeyPath: connect-agent-sa-2007081316.json
Using a single Cloud project
The GKE on-prem configuration file has several fields where you can
specify a Google Cloud project ID. For example, you can specify one
project ID for
gkeconnect and another project ID for
gkeconnect: projectid: "" ... stackdriver: projectid: ""
gkeadm prepared your GKE on-prem configuration file, it set
all of the
projectid fields to the project that is the parent of your
component access service account. Also,
gkeadm granted roles to your
service accounts on the project that is the parent of your service
account. In short,
gkeadm used a single Cloud project.
If you want the flexibility of using multiple Google Cloud projects, you
must manually fill in the
projectid fields and manually grant roles to your
service accounts. For more information, see
Using Multiple Google Cloud projects.