Getting started

This page takes you through some steps you should take before you install GKE On-Prem in your environment.

Before you begin

Review the following topics:

• System requirements
• GKE On-Prem overview

Placeholder values in GKE On-Prem documentation

As you use GKE On-Prem's documentation, you can change placeholder values in code blocks by clicking them:

Enter your name: [YOUR_NAME]

You might find this useful if you want to fill in placeholder values before copying and running commands.

Limitations

Note: For current known issues, refer to the Release notes.

Limitation	Description
Maximum and minimum limits for clusters and nodes	See Quotas and limits. Your environment's performance might impact these limits.
One admin cluster per project	You can only register one admin cluster per Google Cloud project. If you need to run multiple admin clusters, you need to use a separate Google Cloud project.

Creating a Google Cloud project

Create a Google Cloud project, if you don't already have one. You need a project to run GKE On-Prem.

Installing the required command-line interface tools

• Install Google Cloud CLI, which includes gcloud, the command-line interface (CLI) to Google Cloud.
• Install govc, the CLI to VMware vSphere.
• Install Terraform 0.11, which includes the terraform CLI. Follow Terraform's installation instructions to verify the installation and set up your PATH variable.

Authorizing gcloud to access Google Cloud

After you install gcloud CLI, authorize gcloud to access Google Cloud:

gcloud auth login

Setting a default Google Cloud project

Setting a default Google Cloud causes all gcloud CLI commands to run against the project, so that you don't need to specify your project for each command. To set a default project, run the following command:

gcloud config set project [PROJECT_ID]

Replace [PROJECT_ID] with your project ID. (You can find your project ID in Google Cloud console, or by running gcloud config get-value project.)

Creating Google Cloud service accounts

Before you install GKE On-Prem for the first time, you use gcloud to create four Google Cloud service accounts. GKE On-Prem uses these service accounts to complete tasks on your behalf; the following sections describe each account's purpose.

Access service account

Note: If you purchased GKE Enterprise, you might have already created a allowlisted service account. If so, skip this section.

You use this service account to download GKE On-Prem's binaries from Cloud Storage. It is the only service account that Google allowlists.

Run the following command to create access-service-account:

gcloud iam service-accounts create access-service-account

Register service account

Connect uses this service account to register your GKE On-Prem clusters with Google Cloud console.

Run the following command to create register-service-account:

gcloud iam service-accounts create register-service-account

Connect service account

Note: It is a best practice to have separate connect service accounts for each user cluster you create.

Connect uses this service account to maintain a connection between GKE On-Prem clusters and Google Cloud.

Run the following command to create connect-service-account:

gcloud iam service-accounts create connect-service-account

Google Cloud Observability service account

This service account allows GKE On-Prem to write logging and monitoring data to Google Cloud Observability:

Run the following command to create stackdriver-service-account:

gcloud iam service-accounts create stackdriver-service-account

Allowlisting your project and accounts

After you purchase GKE Enterprise, Google allowlists the following to grant you access to GKE On-Prem and Connect:

• Your Google Cloud project.
• Your Google account, and individual Google accounts of team members.
• Your access service account.

If you want to use a different project or service account, or if you'd like to enable additional users, Google Cloud Support or your Technical Account Manager can help. Open a support case via Google Cloud console or the Google Cloud Support Center.

Enabling the required APIs in your project

You need to enable the following APIs in your Google Cloud project:

• cloudresourcemanager.googleapis.com
• container.googleapis.com
• gkeconnect.googleapis.com
• gkehub.googleapis.com
• serviceusage.googleapis.com
• stackdriver.googleapis.com
• monitoring.googleapis.com
• logging.googleapis.com

Note: Pods in your cluster must be able to reach www.googleapis.com and gkeconnect.googleapis.com addresses, either directly or by using a configured proxy server. To learn how to list APIs you've already enabled in your projects, refer to Listing Services in the Service Usage documentation.

To enable these APIs, run the following command:

gcloud services enable \
cloudresourcemanager.googleapis.com \
container.googleapis.com \
gkeconnect.googleapis.com \
gkehub.googleapis.com \
serviceusage.googleapis.com \
stackdriver.googleapis.com \
monitoring.googleapis.com \
logging.googleapis.com

Assigning Identity and Access Management roles to your service accounts

IAM grants accounts permissions to call Google Cloud APIs. Assign dedicated IAM roles to these service accounts for privilege isolation.

Note: In the all of the following commands, the --member parameter accepts the service account's email address. The email address is comprised of its display name and your project ID. To view the email addresses of all service accounts in your project, run gcloud iam service-accounts list.

List service accounts' email addresses

First, list the service accounts in your Google Cloud project:

gcloud iam service-accounts list

For a Google Cloud project named my-gcp-project, this command's output looks like this:

gcloud iam service-accounts list
NAME                                    EMAIL
                                        [e-mail]
                                        [e-mail]
                                        [e-mail]
                                        [e-mail]

Take note of each accounts' email address. For each of the following sections, you provide the relevant account's email account.

Register service account

Grant the gkehub.admin and serviceuserage.serviceUsageViewer roles to your register service account:

gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member="serviceAccount:[REGISTER_SERVICE_ACCOUNT_EMAIL] \
--role="roles/gkehub.admin"

gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member="serviceAccount:[REGISTER_SERVICE_ACCOUNT_EMAIL] \
--role="roles/serviceusage.serviceUsageViewer"

Connect service account

Grant the gkehub.connect role to your connect service account:

gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member="serviceAccount:[CONNECT_SERVICE_ACCOUNT_EMAIL] \
--role="roles/gkehub.connect"

Google Cloud Observability service account

Grant the stackdriver.resourceMetadata.writer, logging.logWriter, and monitoring.metricWriter roles to your Google Cloud Observability service account:

gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member "serviceAccount:[STACKDRIVER_SERVICE_ACCOUNT_EMAIL] \
--role "roles/stackdriver.resourceMetadata.writer"

gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member "serviceAccount:[STACKDRIVER_SERVICE_ACCOUNT_EMAIL] \
--role "roles/logging.logWriter"

gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member "serviceAccount:[STACKDRIVER_SERVICE_ACCOUNT_EMAIL] \
--role "roles/monitoring.metricWriter"

Configuring Logging and Monitoring

Stackdriver Logging and Stackdriver Monitoring are enabled by default for GKE On-Prem.

Allowlisting Google and HashiCorp addresses for your proxy

If your organization requires Internet access to pass through an HTTP proxy, allowlist the following Google addresses in the proxy:

• binaryauthorization.googleapis.com
• googlecode.l.googleusercontent.com
• storage.l.googleusercontent.com
• storage.googleapis.com

You use HashiCorp Terraform version 0.11 to create an admin workstation VM in vSphere. To run Terraform in an environment with a proxy or firewall, you need to allowlist the following HashiCorp addresses:

• checkpoint-api.hashicorp.com
• releases.hashicorp.com

Note: Terraform runs only on the machine that you use to create the admin workstation. GKE On-Prem itself does not rely on Terraform.

Preparing your load balancer

GKE On-Prem clusters can run with one of two load balancing modes, "Integrated" and "Manual." With Integrated mode, GKE On-Prem clusters run with the F5 BIG-IP load balancer. With Manual mode, you manually configure a different load balancer.

Preparing F5 BIG-IP partitions

If you choose to use the Integrated mode, you need to create an F5 BIG-IP partition to handle load balancing for each GKE On-Prem cluster you intend to create.

Initially, you need to create at least two partitions: one for the admin cluster, and one for a user cluster. You must create a partition before you create the corresponding cluster.

Do not use your cluster partitions for anything else. Each of your clusters must have a partition that is for the sole use of that cluster.

To learn how to create partitions, read Creating an administrative partition in the F5 BIG-IP documentation.

Using Manual load balancing mode

The Manual load balancing mode requires more configuration than the Integrated mode. For details, see Enabling manual load balancing.