Creating an admin workstation

This document shows how to create an admin workstation that you can use to create GKE on-prem clusters.

In this document, you create an admin workstation that has a static IP address. This is consistent with the basic installation journey, which creates clusters that use static IP addresses.

If you want to create an admin workstation that uses Dynamic Host Configuration Protocol (DHCP) to get its IP address, see Creating an admin workstation using DHCP.

The steps in this document use the gkeadm command-line tool, which is available only for Linux. In a future version of GKE on-prem, gkeadm will be available for Windows and MAC OS. If you want to use a computer running Windows or MAC OS to create an admin workstation, see Creating an admin workstation with a static IP address or Creating an admin workstation using DHCP.

Downloading gkeadm

Download the gkeadm command-line tool, and make it executable:

gcloud storage cp gs://gke-on-prem-release-public/gkeadm/1.3.2-gke.1/linux/gkeadm ./
chmod +x gkeadm

Generating a template for your configuration file

To create an admin workstation, the gkeadm tool requires a configuration file. To generate a template for your configuration file:

./gkeadm create config

The output is a file named admin-ws-config.yaml.

gcp:
  # Path of the allowlisted service account's JSON key file
  whitelistedServiceAccountKeyPath: ""
# Specify which vCenter resources to use
vCenter:
  # The credentials and address GKE On-Prem should use to connect to vCenter
  credentials:
    address: ""
    username: ""
    ...
adminWorkstation:
  ...
  network:
    # The IP allocation mode: 'dhcp' or 'static'
    ipAllocationMode: ""
    # # The host config in static IP mode. Do not include if using DHCP
    # hostConfig:
    #   # The IPv4 static IP address for the admin workstation
    ...

Filling in your configuration file

In your configuration file, enter field values as described in this section.

gcp.whitelistedServiceAccountKeyPath

String. The path of the JSON key file for your allowlisted service account. For information about creating a JSON key file, see allowlisted service account. For example:

whitelistedServiceAccountKeyPath: "my-key-folder/whitelisted-key.json"

vCenter.credentials.address

String. The IP address or the hostname of your vCenter server.

Before you fill in the address field, download and inspect the serving certificate of your vCenter server. Enter the following command to download the certificate and save it to a file named vcenter.pem.

true | openssl s_client -connect VCENTER_IPVCENTER_IP:443 -showcerts 2>/dev/null | sed -ne '/-BEGIN/,/-END/p' > vcenter.pem

Replace VCENTER_IP with the IP address of your vCenter Server.

Open the certificate file to see the Subject Common Name and the Subject Alternative Name:

openssl x509 -in vcenter.pem -text -noout

The output shows the Subject Common Name (CN). This might be an IP address, or it might be a hostname. For example:

Subject: ... CN = 203.0.113.100
 
Subject: ... CN = my-host.my-domain.example

The output might also include one or more DNS names under Subject Alternative Name:

X509v3 Subject Alternative Name:
    DNS:vcenter.my-domain.example

Choose the Subject Common Name or one of the DNS names under Subject Alternative Name to use as the value of vcenter.credentials.address in your configuration file. For example:

vcenter:
  credentials:
    address: "203.0.113.1"
    ...

vcenter:
  credentials:
    address: "my-host.my-domain.example"
    ...

vCenter.credentials.username

String. A vCenter Server user account. The user account should have the Administrator role or equivalent privileges. See vSphere requirements.

For example:

username: "administrator@vsphere.local"

vCenter.credentials.password

String. The password for the vCenter Server user account. For example:

password: "#STyZ2T#Ko2o"

vCenter.datacenter

String. The name of your vSphere datacenter. For example:

datacenter: "MY-DATACENTER"

vCenter.datastore

String. The name of your vSphere datastore. For example:

datastore: "MY-DATASTORE"

vCenter.cluster

String. The name of your vSphere cluster. For example:

cluster: "MY-CLUSTER"

vCenter.network

String. The name of the vSphere network where you want to create your admin workstation. For example:

network: "MY-VM-NETWORK"

vCenter.resourcePool

String. If you are using a non-default resource pool, provide the name of your vSphere resource pool. For example:

resourcePool: "MY-POOL"

If you are using the default resource pool, provide the following value:

resourcePool: "MY_CLUSTER/Resources"

Replace MY_CLUSTER with the name of your vSphere cluster.

See Specifying the root resource pool for a standalone host.

vCenter.caCertPath

String. When a client, like GKE on-prem, sends a request to your vCenter server, the server must prove its identity to the client by presenting a certificate or a certificate bundle. To verify the certificate or bundle, GKE on-prem must have the root certificate in the chain of trust.

Set vCenter.caCertPath to the path of the root certificate. For example:

caCertPath: "/usr/local/google/home/me/certs/vcenter-ca-cert.pem"

Your VMware installation has a certificate authority (CA) that issues a certificate to your vCenter server. The root certificate in the chain of trust is a self-signed certificate created by VMware.

If you do not want to use the VMWare CA, which is the default, you can configure VMware to use a different certificate authority.

If your vCenter server uses a certificate issued by the default VMware CA, download the certificate as follows:

curl -k "https://SERVER_ADDRESS/certs/download.zip" > download.zip

Replace SERVER_ADDRESS with the address of your vCenter server.

Install the unzip command and unzip the certificate file:

sudo apt-get install unzip
unzip downloads.zip

If the unzip command doesn't work the first time, enter the command again.

Find the certificate file in certs/lin.

proxyUrl

String: If the machine you are using to run gkeadm uses a proxy server for access to the internet, set this field to the URL of the proxy server. For example:

proxyUrl: "https://my-proxy.example.local"

adminWorkstation.name

String. A name of your choice for your admin workstation. For example:

name: "my-admin-workstation"

adminWorkstation.cpus

Integer. The number of virtual CPUs for your admin workstation. For example:

cpus: 4

adminWorkstation.memoryMB

Integer. The number of megabytes of memory for your admin workstation. For example:

memoryMB: 8192

adminWorkstation.diskGB

Integer. The number of gigabytes of virtual disk space for your admin workstation. Must be at least 50. For example:

diskGB: 50

adminWorkstation.network.ipAllocationMode

String. Set this to "static". For example:

ipAllocationMode: "static"

adminWorkstation.network.hostConfig.ip

String. An IP address of your choice for your admin workstation. For example:

ip: "172.16.5.1"

adminWorkstation.network.hostConfig.gateway

String. The IP address of the default gateway for the network that contains your admin workstation. For example:

gateway: "172.16.6.254"

adminWorkstation.network.hostConfig.netmask

String. The netmask for the network that contains your admin workstation. For example:

netmask: "255.255.248.0"

adminWorkstation.network.hostConfig.dns

String array. An array of IP addresses for DNS servers that your admin workstation can use. For example:

dns:
- "172.16.255.1"
- "172.16.255.2"

adminWorkstation.proxyUrl

String. If your network is behind a proxy server, and you want both your admin workstation and your GKE on-prem clusters to use the same proxy server, then set adminworkstation.proxyURL to the URL of the proxy server. For example:

adminworkstation:
  proxyUrl: "http://aw-proxy.example"

adminWorkstation.ntpServer

String. The hostname or IP address of the Network Time Protocol server that your admin workstation is to use. For example:

ntpServer: "216.239.35.0"

If you leave this empty, GKE on-prem uses "ntp.ubuntu.com".

Here is an example of a completed admin-ws-config.yaml file:

gcp:
  whitelistedServiceAccountKeyPath: "my-key-folder/whitelisted-key.json"
vCenter:
  credentials:
    address: "203.0.113.1"
    username: "administrator.vsphere.local"
    password: "#STyZ2T#Ko2o"
  datacenter: "MY-DATACENTER"
  datastore: "MY-DATASTORE"
  cluster: "MY-CLUSTER"
  network: "MY-VM-NETWORK"
  resourcePool: "MY-POOL"
  caCertPath: "/usr/local/google/home/me/certs/the-root.cert"
proxyUrl: ""
adminWorkstation:
  name: "my-admin-workstation"
  cpus: 4
  memoryMB: 8192
  diskGB: 50
  network:
    ipAllocationMode: "static"
    hostConfig:
      ip: "172.16.5.1"
      gateway: "172.16.6.254"
      netmask: "255.255.248.0"
      dns:
      - "172.16.255.1"
      - "172.16.255.2
  proxyUrl: ""
  ntpServer: "216.239.35.0"

Creating your admin workstation

To use gkeadm to create an admin workstation and a set of service accounts, you must grant the resourcemanager.projectIamAdmin and serviceusage.serviceUsageAdmin IAM roles to the default Cloud Billing account. To view the default account:

gcloud config get-value account

To grant the required roles:

gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="user:DEFAULT_ACCOUNT" \
    --role="roles/resourcemanager.projectIamAdmin"

gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="user:DEFAULT_ACCOUNT" \
    --role="roles/serviceusage.serviceUsageAdmin"

Replace the following:

  • PROJECT_ID: the ID of your connect project.
  • DEFAULT_ACCOUNT: the default Cloud Billing account.

Create your admin workstation and a set of service accounts. If you prefer to create and configure your service accounts manually, you can omit the --auto-create-service-accounts flag.

./gkeadm create admin-workstation --auto-create-service-accounts

The output gives detailed information about the creation of your admin workstation. For example, you can see that gkeadm performed these tasks:

  • Ran preflight checks.

  • Downloaded an OVA image and uploaded it as a vSphere template.

  • Created an SSH key.

  • Created an admin workstation VM.

  • Printed the gkectl and Docker versions installed on the admin workstation.

  • Printed information about the NTP server on the admin workstation.

  • Created service accounts and their JSON key files. These service accounts are members of the project that is the parent of your allowlisted service account.

  • Assigned the appropriate IAM roles to the service accounts. These roles are granted on the project that is the parent of your allowlisted service account.

  • Copied JSON key files for the service accounts to the admin workstation.

  • Copied the CA certificate file for your vCenter server to the admin workstation.

  • Prepared config.yaml on the admin workstation. Later, you will use config.yaml to create clusters.

...
- Validation Category: vCenter
    - [SUCCESS] Credentials
    - [SUCCESS] Version
    - [SUCCESS] Datacenter
    - [SUCCESS] Datastore
    - [SUCCESS] Resource Pool
    - [SUCCESS] Network

All validation results were SUCCESS.

******************************************
Admin workstation VM successfully created:
- Name:    my-admin-workstation
- IP:      172.16.5.1
- SSH Key: /usr/local/google/home/me/.ssh/gke-admin-workstation
******************************************

Printing gkectl and docker versions on admin workstation...
gkectl version
gkectl 1.3.1-gke.0 (git-3aaf91927)

docker version
Client:
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.9
 Git commit:        6a30dfca03
 Built:             Mon Sep 30 22:47:02 2019
 OS/Arch:           linux/amd64
 Experimental:      false

...

Getting allowlisted service account...

Creating other service accounts and JSON key files...
    - log-mon-sa-2004211642
    - connect-agent-sa-2004211642
    - connect-register-sa-2004211642

Enabling APIs...
    - project my-project
        - serviceusage.googleapis.com
        - iam.googleapis.com
        - cloudresourcemanager.googleapis.com

Configuring IAM roles for service accounts...
    - my-allowlisted-sa for project my-project
        - roles/serviceusage.serviceUsageViewer
        - roles/iam.serviceAccountCreator
        - roles/iam.roleViewer
    - log-mon-sa-2004211642 for project my-project
        - roles/stackdriver.resourceMetadata.writer
        - roles/logging.logWriter
        - roles/monitoring.metricWriter
    - connect-agent-sa-2004211642 for project my-project
        - roles/gkehub.connect
    - connect-register-sa-2004211642 for project my-project
        - roles/gkehub.admin
        - roles/serviceusage.serviceUsageViewer

Copying files to admin workstation...
    - /usr/local/google/home/me/certs/vcenter-ca-cert.pem
    - /usr/local/google/home/me/service-account-keys/whitelisted-key.json
    - log-mon-sa-2004211642.json
    - connect-agent-sa-2004211642.json
    - connect-register-sa-2004211642.json

Preparing config.yaml for gkectl...


********************************************************************
Admin workstation is ready to use.

SSH into the admin workstation with the following command:
ssh -i /usr/local/google/home/me/.ssh/gke-admin-workstation ubuntu@172.16.5.1
********************************************************************

Admin workstation information saved to /usr/local/google/home/me/my-admin-workstation

Getting an SSH connection to your admin workstation

Near the end of the preceding output there is a command you can use to get an SSH connection to your admin workstation. Enter that command now. For example:

ssh -i /usr/local/google/home/me/.ssh/gke-admin-workstation ubuntu@172.16.5.1

List the files on your admin workstation:

ls -1

In the output, you can see a GKE on-prem configuration file, your CA certificate file, and the JSON key files for your service accounts:

config.yaml
connect-agent-sa-200303-143027.json
connect-register-sa-200303-143027.json
log-mon-sa-200303-143027.json
the-root.crt
whitelisted-key.json

Inspecting the GKE on-prem configuration file

On your admin workstation, your configuration file was created and named config.yaml. You use this file later to create your admin and user clusters. Use a text editor to open config.yaml. Notice that many of the fields are already populated with values that you provided in your admin-ws-config.yaml file. For example:

vcenter:
  credentials:
    address: 203.0.113.1
    username: administrator@vsphere.local
    password: STyZ2T#Ko2o
  datacenter: MY-DATACENTER
  datastore: MY-DATASTORE
  cluster: MY-CLUSTER
  network: MY-VM-NETWORK
…
gkeconnect:
  projectid: my-project-id
  registerserviceaccountkeypath: /home/ubuntu/connect-register-sa-xxx.json
  agentserviceaccountkeypath: /home/ubuntu/connect-agent-sa-xxx.json
stackdriver:
  projectid: my-project-id
  clusterlocation: ""
  enablevpc: false
  serviceaccountkeypath: /home/ubuntu/log-mon-sa-xxx.json

Using a single Google Cloud project

The GKE on-prem configuration file has several fields where you can specify a Google Cloud project ID. For example, you can specify one project ID for gkeconnect and another project ID for stackdriver:

gkeconnect:
  projectid: ""
...
stackdriver:
  projectid: ""

When gkeadm prepared your GKE on-prem configuration file, it set all of the projectid fields to the project that is the parent of your allowlisted service account. Also, gkeadm granted roles to your service accounts on the project that is the parent of your allowlisted service account. In short, gkeadm used a single Google Cloud project.

If you want the flexibility of using multiple Google Cloud projects, you must manually fill in the projectid fields and manually grant roles to your service accounts. For more information, see Using Multiple Google Cloud projects.