Authenticating with OIDC and Google

Caution: Be sure to plan your installation ahead of time. Your clusters' configurations, as declared in the GKE on-prem configuration file, become immutable after you create the clusters. You can't change most aspects of your clusters after they are created, including networking and authentication. The only configurable exceptions are:

• Resizing user clusters
• Creating additional user clusters
• Upgrading clusters
• Creating and managing node pools

This page shows how to use OpenID Connect (OIDC) with GKE on-prem. To use OIDC, you must specify an OpenID Provider. This topic shows how to configure GKE on-prem to use Google as the OpenID Provider.

For an overview of the authentication flow, see Authentication. For general information about using OpenID providers with GKE on-prem, see Authenticating with OpenID Connect.

Limitations

The Google OpenID provider does not support groups. When you use Kubernetes role-based access control (RBAC) to grant roles to authenticated users, you must grant roles to individual users, not groups.

Overview

GKE on-prem supports OpenID Connect (OIDC) as one of the authentication mechanisms for interacting with a user cluster's Kubernetes API server. With OIDC, you can manage access to Kubernetes clusters by using the standard procedures in your organization for creating, enabling, and disabling employee accounts.

There are two ways an employee can use the OIDC authentication flow:

• An employee can use kubectl to initiate an OIDC flow. To make this flow automatic, GKE on-prem provides the Anthos Plugin for Kubectl, a kubectl plugin.

• An employee can use Google Cloud console to initiate an OIDC authentication flow.

In this exercise, you configure both options: kubectl and Google Cloud console.

Before you begin

Before you read this topic, you should be familiar with OAuth 2.0 and OpenID Connect. You should also be familiar with OpenID scopes and claims.

Personas

This topic refers to three personas:

• Organization administrator: This person chooses an OpenID provider and registers client applications with the provider.

• Cluster administrator: This person creates one or more user clusters and creates authentication configuration files for developers who use the clusters.

• Developer: This person runs workloads on one or more clusters and uses OIDC to authenticate.

Creating a redirect URL for the Anthos Plugin for Kubectl

This section is for organization administrators.

As part of configuring Google to be your OpenID provider, you must specify a redirect URL that Google can use to return ID tokens to the Anthos Plugin for Kubectl. The Anthos Plugin for Kubectl runs on each developer's local machine and listens on a port of your choice. Choose a port number greater than 1024 that is suitable for this purpose. Then, the redirect URL is:

http://localhost:[PORT]/callback

where [PORT] is your port number.

When you configure the Google OpenID provider, specify http://localhost:[PORT]/callback as one of your redirect URLs.

Configuring a redirect URL for Google Cloud console

This section is for organization administrators.

In addition to having a redirect URL for the Anthos Plugin for Kubectl, you need a redirect URL for Google Cloud console. The redirect URL for Google Cloud console is:

https://console.cloud.google.com/kubernetes/oidc

When you configure the Google OpenID provider, specify https://console.cloud.google.com/kubernetes/oidc as one of your redirect URLs.

Configuring your consent screen

In this section, you configure Google's Oauth consent screen. When a developer in your organization initiates authentication to a user cluster, they are taken to this consent screen. At that time, they prove their identity to Google and give Google permission to create a token that provides identifying information to the OAuth client. In the context of this topic, the OAuth client is either the Anthos Plugin for Kubectl or Google Cloud console.

1. Go to the OAuth consent screen page in Google Cloud console.

   Configure the OAuth consent screen

2. Select Internal, and click Create.

3. For Application type, select Internal.

4. For Application name, enter a name of your choice. Suggestion: GKE on-prem.

5. Under Authorized domains, add google.com.

6. Fill in additional fields as you see fit.

7. Click Save.

Registering a client application with Google

In this section, you register GKE on-prem with Google, so that Google can act as the OpenID provider for developers in your organization. As part of the registration, you must supply the two redirect URLs that you created previously.

1. Go to the Credentials page in Google Cloud console.

   Go to the Credentials page

2. Click Create credentials, and select OAuth client ID.

3. For Application type, select Web application.

4. For Name, enter a name of your choice.

5. Under Authorized redirect URIs, add your two redirect URLs. Recall that you created a redirect URL for the Anthos Plugin for Kubectl and a redirect URL for Google Cloud console.

6. Click Create.

7. You are given a client ID and a client secret. Save these for later use.

Populating the oidc specification in GKE on-prem configuration file

This section is for cluster administrators.

Before you create a user cluster, you generate a GKE on-prem configuration file using gkectl create-config. The configuration includes the following oidc specification. You populate oidc with values specific to your provider:

oidc:
  issuerurl:
  kubectlredirecturl:
  clientid:
  clientsecret:
  username:
  usernameprefix:
  group:
  groupprefix:
  scopes:
  extraparams:
  usehttpproxy:
  capath:

• issuerurl: Set this to "https://accounts.google.com". Client applications, like the Anthos Plugin for Kubectl and Google Cloud console, send authorization requests to this URL. The Kubernetes API server uses this URL to discover public keys for verifying tokens.
• kubectlredirecturl: The redirect URL that you configured previously for the Anthos Plugin for Kubectl.
• clientid: ID for the client application that makes authentication requests to the OpenID provider. Both the Anthos Plugin for Kubectl and Google Cloud console use this ID. You were given this ID when you registered your client application with Google.
• clientsecret: Secret for the client application. Both the Anthos Plugin for Kubectl and Google Cloud console use this secret. You were given this ID when you registered your client application with Google.
• username: Set this to "email".
• usernameprefix: Optional. Prefix prepended to username claims to prevent clashes with existing names. If you do not provide this field, and username is a value other than email, the prefix defaults to issuerurl#. You can use the value - to disable all prefixing.
• group: Leave this blank.
• groupprefix: Leave this blank.
• scopes: Set this to "email"
• extraparams: Set this to "prompt=consent,access_type=offline".
• usehttpproxy: Set this to "false".
• capath: Leave this blank.

Creating an RBAC policy for your user cluster

This section is for cluster administrators.

After you create a cluster, use Kubernetes role-based access control (RBAC) to grant access to authenticated cluster users. To grant access to resources in a particular namespace, create a Role and a RoleBinding. To grant access to resources across an entire cluster, create a ClusterRole and a ClusterRoleBinding.

When you use Google as your OpenID provider, you must grant access to individual users. It does not work to grant access to groups. This is because the token that the Google OpenID provider returns does not have any information about the the groups that an individual user belongs to.

For example, suppose you want [e-mail] to be able to view all Secret objects across the cluster.

Here's a manifest for a ClusterRole named secret-viewer. A person who is granted this role can get, watch, and list any Secret in the cluster.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secret-viewer
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "watch", "list"]

Here is a manifest for a ClusterRoleBinding named people-who-view-secrets. The binding grants the secret-viewer role to a user named [e-mail].

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: people-who-view-secrets
subjects:
- kind: User
  name: [e-mail]
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-viewer
  apiGroup: rbac.authorization.k8s.io

To create the ClusterRole, save the manifest to a file named secret-viewer-cluster-role.yaml, and enter this command:

kubectl --kubeconfig [USER_CLUSTER_KUBECONFIG] apply -f secret-viewer-cluster-role.yaml

where [USER_CLUSTR_KUBECONFIG] is the kubeconfig file for your user cluster.

To create the ClusterRoleBinding, save the manifest to a file named secret-viewer-cluster-role-binding.yaml, and enter this command:

kubectl --kubeconfig [USER_CLUSTER_KUBECONFIG] apply -f secret-viewer-cluster-role-binding.yaml

Creating your first authentication configuration file

This section is for cluster administrators.

After you create a user cluster, create an authentication configuration file for your cluster:

gkectl create-login-config --kubeconfig [USER_CLUSTER_KUBECONFIG]

where [USER_CLUSTER_KUBECONFIG] is the path of your user cluster's kubeconfig file. When you created your user cluster, gkectl create cluster generated a kubeconfig file for the cluster.

The preceding command creates a file named kubectl-anthos-config.yaml in the current directory.

Adding additional clusters to your authentication configuration file

This section is for cluster administrators.

You can store the authentication configuration for several clusters in a single file. Then you can distribute that one file to the developers who should have access to all of those clusters.

Start with an existing authentication configuration file. Then use the following command to merge that file with the configuration for an additional cluster:

gkectl create-login-config --kubeconfig [USER_CLUSTER_KUBECONFIG] \
    --merge-from [IN_AUTH_CONFIG_FILE] --output [OUT_AUTH_CONFIG_FILE]

where

• [USER_CLUSTER_KUBECONFIG] is the kubeconfig file of the additional cluster.

• [IN_AUTH_CONFIG_FILE] is the path of the existing authentication configuration file.

• [OUT_AUTH_CONFIG_FILE] is the path where you want to save the file that holds the merged configuration. This can be the same as [IN_AUTH_CONFIG_FILE].

Note: In a single authentication configuration file, the individual cluster names must be unique.

Distributing the authentication configuration file

This section is for cluster administrators.

The authentication configuration file that you distribute to your developers must be namedkubectl-anthos-config.yaml. You can distribute the authentication configuration file in a number of ways:

• Manually give the file to each developer.

• Host the file at a URL, so developers can download it.

• Push the file to each developer's machine.

Regardless of your distribution method, the authentication configuration file must be placed at this location on each developer's machine:

Linux

$HOME/.config/google/anthos/kubectl-anthos-config.yaml

macOS

$HOME/Library/Preferences/google/anthos/kubectl-anthos-config.yaml

Windows

%APPDATA%\google\anthos\kubectl-anthos-config.yaml

Placing the authentication configuration file

This section is for developers.

The authentication configuration file contains the details of all clusters you can authenticate to. Do not modify the contents of this file.

If your cluster administrator has provided you with an authentication configuration file, place the file in the correct directory. If your cluster administer has already placed the configuration on your machine, you can skip this section.

Copy your authentication configuration file to its default location:

Linux

mkdir -p  $HOME/.config/google/anthos/
cp [AUTH_CONFIG_FILE] $HOME/.config/google/anthos/kubectl-anthos-config.yaml

where [AUTH_CONFIG_FILE] is the name of your authentication configuration file.

macOS

mkdir -p  $HOME/Library/Preferences/google/anthos/
cp [AUTH_CONFIG_FILE] $HOME/Library/Preferences/google/anthos/kubectl-anthos-config.yaml

where [AUTH_CONFIG_FILE] is the name of your authentication configuration file.

Windows

md "%APPDATA%\google\anthos"
copy [AUTH_CONFIG_FILE] "%APPDATA%\google\anthos\kubectl-anthos-config.yaml"

where [AUTH_CONFIG_FILE] is the name of your authentication configuration file.

Note: If you do not want to set up a default configuration, you can manually pass the path of the authentication configuration file to the plugin commands by using the login-config flag. For example, see Using the Anthos Plugin for Kubectl to authenticate.

Getting the Anthos Plugin for Kubectl

This section is for cluster administrators.

The Anthos Plugin for Kubectl is included in your admin workstation at:

Linux

/usr/bin/kubectl_anthos/v1.0beta/linux_amd64/kubectl-anthos

macOS

/usr/bin/kubectl_anthos/v1.0beta/darwin_amd64/kubectl-anthos

Windows

/usr/bin/kubectl_anthos/v1.0beta/windows_amd64/kubectl-anthos

You can distribute the plugin to your developers, or you can have them download the plugin directly.

Downloading the Anthos Plugin for Kubectl

This section is for cluster administrators and developers.

Each developer needs to have the Anthos Plugin for Kubectl on their own machine. Developers can download the plugin individually, or the cluster administrator can download the plugin and then distribute it to the developers.

Note for developers: Ask your cluster administrator what version of the plugin you should use.

Download the Anthos Plugin for Kubectl:

Linux

gsutil cp gs://gke-on-prem-release/kubectl-anthos/v1.0beta/linux_amd64/kubectl-anthos ./
chmod +x kubectl-anthos

macOS

gsutil cp gs://gke-on-prem-release/kubectl-anthos/v1.0beta/darwin_amd64/kubectl-anthos ./
chmod +x kubectl-anthos

Windows

gsutil cp gs://gke-on-prem-release/kubectl-anthos/v1.0beta/windows_amd64/kubectl-anthos ./
rename kubectl-anthos kubectl-anthos.exe.

Installing the Anthos Plugin for Kubectl

This section is for developers.

Your cluster administrator might provide you with the Anthos Plugin for Kubectl. Or your cluster administrator might tell you to download the plugin yourself.

To install the Anthos Plugin for Kubectl, move the executable file to any location on your PATH. The executable file must be named kubectl-anthos. To learn more, see the Installing kubectl plugins.

Verify that the Anthos Plugin for Kubectl is installed:

kubectl plugin list
kubectl anthos version

Note: Some Windows users might have to run the command as kubectl-anthos.exe version instead of kubectl anthos version.

Note: Extending kubectl with plugins is supported in kubectl v1.12 and later.

Listing available clusters

This section is for developers.

If your authentication configuration file is at the default path, enter this command to list the clusters that you can authenticate to:

kubectl anthos listclusters

If your authentication configuration file is not at the default path, enter this command to list clusters:

kubectl anthos listclusters --loginconfig [AUTH_CONFIG_FILE]

where [AUTH_CONFIG_FILE] is the path to your authentication configuration file.

Using the Anthos Plugin for Kubectl to authenticate

This section is for developers.

Log in to a user cluster:

kubectl anthos login --cluster [CLUSTER_NAME] --user [USER_NAME] \
    --loginconfig [AUTH_CONFIG_FILE] --kubeconfig [USER_CLUSTER_KUBECONFIG]

where:

• [CLUSTER_NAME] is the name of your user cluster. This name must be selected from the list provided by the kubectl anthos listclusters command.

• [USER_NAME] an optional parameter that specifies the username for the credentials stored in the kubeconfig file. The default value is [CLUSTER_NAME]-anthos-default-user.

• [AUTH_CONFIG_FILE] is the path of your authentication configuration file. If your authentication configuration file is in the default location, you can omit this parameter.

• [USER_CLUSTER_KUBECONFIG] is the path of your user cluster's kubeconfig file. If a kubeconfig file does not exist, the command generates a new kubeconfig file from the authentication configuration file, and adds the cluster details and tokens to the kubeconfig file.

kubectl anthos login launches a browser where you can enter your credentials.

The kubeconfig file provided now contains an ID token that kubectl can use to authenticate to the Kubernetes API server on the user cluster.

Note: Some Windows users might have to run the command as kubectl-anthos.exe login instead of kubectl anthos login.

The kubectl anthos login command has an optional --dry-run flag. If you include the --dry-run flag, the command prints the kubectl commands that would add tokens to the kubeconfig file, but does not actually save the tokens in the kubeconfig file.

Verify that authentication was successful by entering a kubectl command. For example:

kubectl get nodes --kubeconfig [USER_CLUSTER_KUBECONFIG]

Using OIDC with Google Cloud console

This section is for developers.

1. Verify that your cluster is configured for OIDC.

2. Verify that your cluster has been registered with Google Cloud, either automatically during cluster creation or manually.

3. Visit the Kubernetes clusters page in Google Cloud console.

   Visit the Kubernetes clusters page

4. In the list of clusters, locate your GKE On-Prem cluster, and click Login.

5. Select Authenticate with the Identity Provider configured for the cluster, and click LOGIN.

   You will be redirected to your identity provider, where you might need to log in or consent to Google Cloud console accessing your account. Then you will be redirected back to the Kubernetes clusters page in Google Cloud console.

Custom configuration

By default, the Anthos Plugin for Kubectl expects the authentication configuration file to be at a certain location. If you do not want to put the authentication configuration file at the default location, you can manually pass the path of the authentication configuration file to the plugin commands by using the --login-config flag. For example, see Using the Anthos Plugin for Kubectl to authenticate.

Troubleshooting OIDC in GKE on-prem

Invalid configuration

If Google Cloud console cannot read the OIDC configuration from your cluster, the LOGIN button will be disabled.

Invalid provider configuration

If your identity provider configuration is invalid, you will see an error screen from your identity provider after you click LOGIN. Follow the provider-specific instructions to correctly configure the provider or your cluster.

Invalid permissions

If you complete the authentication flow, but still don't see the details of the cluster, make sure you granted the correct RBAC permissions to the account that you used with OIDC. Note that this might be a different account from the one you use to access Google Cloud console.

Error: missing 'RefreshToken' field in 'OAuth2Token' in credentials struct

You might get this error if the authorization server prompts for consent, but the required authentication parameter wasn't provided. Provide the prompt=consent parameter to GKE on-prem configuration file's oidc: extraparams field, and regenerate the client authentication file with the --extra-params prompt=consent flag.

What's next

• Read the overview of authenticating with OpenID Connect in GKE on-prem.

• Learn more about OAuth 2.0.

• Learn more about OpenID Connect.

• Learn more about scopes and claims.

• Learn about custom claims in ID tokens.