Authenticating with OIDC and A DFS

Alpha

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Caution: Be sure to plan your installation ahead of time. Your clusters' configurations, as declared in the GKE On-Prem configuration file, become immutable after you create the clusters. You can't change most aspects of your clusters after they are created, including networking and authentication. The only configurable exceptions are:

• Resizing user clusters
• Creating additional user clusters
• Upgrading clusters
• Creating and managing node pools

This page shows how to use OpenID Connect (OIDC) with Active Directory Federation Services (AD FS) to configure authentication for GKE On-Prem user clusters.

For an overview of the authentication flow, see Authentication. For information about using OpenID providers other than AD FS, see Authenticating with OpenID Connect.

Overview

GKE On-Prem supports OpenID Connect (OIDC) as one of the authentication mechanisms for interacting with a user cluster's Kubernetes API server. With OIDC, you can manage access to Kubernetes clusters by using the standard procedures in your organization for creating, enabling, and disabling employee accounts.

There are two ways an employee can use the OIDC authentication flow:

• An employee can use kubectl to initiate an OIDC flow. To make this flow automatic, GKE On-Prem provides the Kubectl Plugin for OIDC, a kubectl plugin.

• An employee can use Google Cloud console to initiate an OIDC authentication flow.

In this exercise, you configure both options: kubectl and Google Cloud console. You use a set of AD FS management wizards to configure your AD FS server, and your AD employee database.

Before you begin

This topic assumes you are familiar with OAuth 2.0 and OpenID Connect. This topic assumes you are familiar with OpenID scopes and claims.

This topic applies to enterprises that have the following infrastructure:

• The enterprise uses Active Directory (AD) for its employee database.
• The enterprise runs an Active Directory Federation Services (AD FS) server.
• The AD FS server acts as an OpenID provider.

Downloading the Kubectl Plugin for OIDC

This section is for administrators and employees who want to use the Kubectl Plugin for OIDC.

Download the plugin and set access permissions:

Linux

gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/linux_amd64/kubectl-oidc .
chmod +x kubectl-oidc

Windows

gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/windows_amd64/kubectl-oidc .

macOS

gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/darwin_amd64/kubectl-oidc .
chmod +x kubectl-oidc

Installing the plugin

Install the plugin by moving the executable file to any location on your PATH. The executable file must be named kubectl-oidc. To learn more, see Installing kubectl plugins.

Creating a redirect URL for the Kubectl Plugin for OIDC

This section is for administrators.

As part of establishing a relationship with your AD FS server, you must specify a redirect URL that the AD FS server can use to return ID tokens to Kubectl Plugin for OIDC. The Kubectl Plugin for OIDC runs on each employee's local machine and listens on a port of your choice. Choose a port number greater than 1024 that is suitable for this purpose. Then, the redirect URL is:

http://localhost:[PORT]/callback

where [PORT] is your port number.

When you configure your AD FS server, specify http://localhost:[PORT]/callback as one of your redirect URLs.

Configuring a redirect URL for Google Cloud console

This section is for administrators.

In addition to having a redirect URL for the Kubectl Plugin for OIDC, you need a redirect URL for Google Cloud console. The redirect URL for Google Cloud console is:

https://console.cloud.google.com/kubernetes/oidc

When you configure your AD FS server, specify https://console.cloud.google.com/kubernetes/oidc as one of your redirect URLs.

Configuring AD FS

The following sections explain how to configure AD FS for GKE On-Prem.

Setting the redirect URLs

1. Open the AD FS management pane.

2. Select Application Groups > Actions > Add an Application Group.

3. Select Server Application. Enter a name and description of your choice. Click Next.

4. Enter your two redirect URLs. You are given a client ID. This is how the AD FS server identifies the Kubectl Plugin for OIDC and Google Cloud console. Save the client ID for later.

5. Select Generate a shared secret. The Kubectl Plugin for OIDC and Google Cloud console use this secret to authenticate to the AD FS server. Save the secret for later.

Configuring security groups (optional)

1. In AD FS management, select Relying party trusts > Add a new relying party trust.

2. Select Claims aware, and click Start.

3. Select Enter data about relying party manually.

4. Enter a display name.

5. Skip the next two steps.

6. Enter a Relying party trust identifier. Suggestion: token-groups-claim.

7. For Access control policy, select Permit everyone. This means that all employees share their security group information with the Kubectl Plugin for OIDC and Google Cloud console.

8. Click Finish.

Mapping LDAP attributes to claim names

1. In AD FS management, select Relying party trusts > Edit claim issuance policy.

2. Select Send LDAP Attributes as Claims, and click Next.

3. For Claim rule name, enter groups.

4. For Attribute store, select Active Directory.

5. In the table, for LDAP Attribute, select:

   • AD FS version 5.0 and later: Token-Groups Qualified by Domain name
   • AD FS versions before 5.0: Token Groups - Qualified Names

6. For Outgoing Claim Type, select:

   • AD FS version 5.0 and later: Group
   • AD FS versions before 5.0: groups

7. Click Finish, and click Apply.

Registering the Kubectl Plugin for OIDC and Google Cloud console with AD FS

Open a PowerShell window in Administrator mode, and enter this command:

Grant-AD FSApplicationPermission `
    -ClientRoleIdentifier "[CLIENT_ID]" `
    -ServerRoleIdentifier [SERVER_ROLE_IDENTIFIER] `
    -ScopeName "allatclaims", "openid"

where:

• [CLIENT_ID] is the client ID that you obtained previously.

• [SERVER_ROLE_IDENTIFIER] is the claim identifier you entered previously. Recall that the suggested identifier was token-groups-claim.

Populating the oidc specification in the cluster configuration file

This section is for employees who want to create a cluster that is configured to use OIDC.

Before you create a user cluster, you generate a GKE On-Prem configuration file using gkectl create-config. The configuration includes the following oidc specification. You populate oidc with values specific to your provider:

oidc:
  issuerurl:
  clientid:
  clientsecret:
  username:
  usernameprefix:
  group:
  groupprefix:
  scopes:
  extraparams:
  usehttpproxy:
  capath:

• issuerurl: Required. URL of your OpenID provider, such as https://example.com/adfs. Client applications, like the Kubectl Plugin for OIDC and Google Cloud console, send authorization requests to this URL. The Kubernetes API server uses this URL to discover public keys for verifying tokens. Must use HTTPS.
• clientid: Required. ID for the client application that makes authentication requests to the OpenID provider. Both the Kubectl Plugin for OIDC and Google Cloud console use this ID.
• clientsecret: Optional. Secret for the client application. Both the Kubectl Plugin for OIDC and Google Cloud console use this secret.
• username: Optional. JWT claim to use as the username. The default is sub, which is expected to be a unique identifier of the end user. You can choose other claims, such as email or name, depending on the OpenID provider. However, claims other than email are prefixed with the issuer URL to prevent naming clashes with other plugins.
• usernameprefix: Optional. Prefix prepended to username claims to prevent clashes with existing names. If you do not provide this field, and username is a value other than email, the prefix defaults to issuerurl#. You can use the value - to disable all prefixing.
• group: Optional. JWT claim that the provider will use to return your security groups.
• groupprefix: Optional. Prefix prepended to group claims to prevent clashes with existing names. For example, given a group foobar and a prefix gid-, gid-foobar.
• scopes: Optional. Additional scopes to send to the OpenID provider as a comma-delimited list.
• extraparams: Optional. Additional key-value parameters to send to the OpenID provider as a comma-delimited list.
  • For a list of authentication parameters, see Authentication URI parameters
  • For a list of Microsoft Azure's authentication parameters, see Send the sign-in request.
  • If you are authorizing a group, pass in resource=token-groups-claim.
  • If your authorization server prompts for consent, pass in prompt=consent.
• usehttpproxy: Optional. Specifies whether to deploy a reverse proxy in the cluster to allow Connect Agent access to the on-premises OIDC provider for authenticating users. Value must be a string: "true" or "false".
• capath: Optional. Path to the certificate for the certificate authority (CA) that issued your identity provider's web certificate. This value might not be necessary. For example, if your identity provider's certificate was issued by a well-known public CA, then you would not need to provide a value here.

Example: Authorizing users and groups

Many providers encode user-identifying properties, such as email and user IDs, in a token. However, these properties have implicit risks for authentication policies:

• User IDs can make policies difficult to read and audit.
• Emails can create both an availability risk (if a user changes their primary email) and potentially a security risk (if an email can be re-assigned).

Therefore, it's a best practice to use group policies, as a group ID can be both persistent and easier to audit.

Suppose your provider creates identity tokens that include the following fields:

{
  'iss': 'https://server.example.com'
  'sub': 'u98523-4509823'
  'groupList: ['[e-mail]', '[e-mail]']
  ...
}

Given this token format, you'd populate your configuration file's oidc specification like so:

issueruri: 'https://server.example.com'
username: 'sub'
usernameprefix: 'uid-'
group: 'groupList'
groupprefix: 'gid-'
extraparams: 'resource=token-groups-claim'
...

After you've created the user cluster, you could then use Kubernetes role-based access control (RBAC) to grant privileged access to the authenticated users. For example, you could create a ClusterRole that grants its users read-only access to the cluster's Secrets, and create a ClusterRoleBinding resource to bind the role to the authenticated group:

ClusterRole

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  # The resource type for which access is granted
  resources: ["secrets"]
  # The permissions granted by the ClusterRole
  verbs: ["get", "watch", "list"]

ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-secrets-admins
subjects:
  # Allows anyone in the "us-east1-cluster-admins" group to
  # read Secrets in any namespace within this cluster.
- kind: Group
  name: gid-us-east1-cluster-admins # Name is case sensitive
  apiGroup: rbac.authorization.k8s.io
  # Allows this specific user to read Secrets in any
  # namespace within this cluster
- kind: User
  name: uid-u98523-4509823
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

Saving the certificate of the CA of the Kubernetes API server

This section is for employees who have created a user cluster and now want to use the Kubectl Plugin for OIDC.

Your user cluster has a Kubernetes API server. And your user cluster's kubeconfig file stores the certificate of the CA that issued a certificate to the Kubernetes API server. The CA's certificate is the base-64 encoded value of the certificate-authority-data field. You need to decode this value and store it in a local file, like server-ca-cert:

cat [USER_CLUSTER_KUBECONFIG]  | grep certificate-authority-data | awk '{ print $2}' | base64 --decode > server-ca-cert

Generating the client authentication configuration file

This section is for employees who have created a user cluster and now want to use the Kubectl Plugin for OIDC.

To generate a client authentication configuration file, enter the following command:

Linux

kubectl oidc client-config \
--issuer-uri [ISSUER_URI] \
--redirect-uri [REDIRECT_URL] \
--client-id [CLIENT_ID] \
--client-secret [CLIENT_SECRET] \
--scopes "[CUSTOM_SCOPES]" \
--cluster-name [USER_CLUSTER_NAME] \
--server [CLUSTER_URL] \
--server-ca-file [SERVER_CA_CERT] \
--issuer-ca-file [PROVIDER_CA_CERT] \
--extra-params [KEY]=[VALUE], ... # e.g. --extra-params "resource=token-groups-claim"
> client-config.yaml

where:

• [ISSUER_URI] is your issuer URI.
• [REDIRECT_URL] is the redirect URL for the Kubectl Plugin for OIDC
• [CLIENT_ID] is the client ID for the Kubectl Plugin for OIDC.
• [CLIENT_SECRET] is the client secret for the Kubectl Plugin for OIDC.
• [USER_CLUSTER_NAME] is your user cluster's name.
• [CLUSTER_URL] is the URL of the user cluster's Kubernetes API server.
• [SERVER_CA_FILE] is the path to the certificate of the CA that issued a certificate to the Kubernetes API server. This is the certificate file that you created in the previous section.
• [PROVIDER_CA_CERT] is the path to the certificate of the CA that signed the OpenID provider's certificate. This is the same as the value of oidc:cacert in your cluster configuration file.
• [CUSTOM_SCOPES] is the comma-separated list of your custom scopes for security groups. This is the same as the value of oidc:scopes in your cluster configuration file.
• --extra-params [KEY]=[VALUE], ... is a list of comma-delimited key-value pairs to be included in authorization requests to the OpenID provider.
  • For a list of authentication parameters, see Authentication URI parameters
  • If you are authorizing a group, pass in resource=token-groups-claim.
  • If your authorization server prompts for consent, pass in prompt=consent.

PowerShell

kubectl oidc client-config `
--issuer-uri [ISSUER_URI] `
--redirect-uri [REDIRECT_URL] `
--client-id [CLIENT_ID] `
--client-secret [CLIENT_SECRET] `
--scopes "[CUSTOM_SCOPES]" `
--cluster-name [USER_CLUSTER_NAME] `
--server [CLUSTER_URL] `
--server-ca-file [SERVER_CA_CERT] `
--issuer-ca-file [PROVIDER_CA_CERT] `
--extra-params [KEY]=[VALUE]
> client-config.yaml

where:

• [ISSUER_URI] is your issuer URI.
• [REDIRECT_URL] is the redirect URL for the Kubectl Plugin for OIDC
• [CLIENT_ID] is the client ID for the Kubectl Plugin for OIDC.
• [CLIENT_SECRET] is the client secret for the Kubectl Plugin for OIDC.
• [USER_CLUSTER_NAME] is your user cluster's name.
• [CLUSTER_URL] is the URL of the user cluster's Kubernetes API server.
• [SERVER_CA_FILE] is the path to the certificate of the CA that issued a certificate to the Kubernetes API server. This is the certificate file that you created in the previous section.
• [PROVIDER_CA_CERT] is the path to the certificate of the CA that signed the OpenID provider's certificate. This is the same as the value of oidc:cacert in your cluster configuration file.
• [CUSTOM_SCOPES] is the comma-separated list of your custom scopes for security groups. This is the same as the value of oidc:scopes in your cluster configuration file.
• --extra-params [KEY]=[VALUE], ... is a list of comma-delimited key-value pairs to be included in authorization requests to the OpenID provider.
  • For a list of Google's authentication parameters, see Authentication URI parameters
  • For a list of Microsoft Azure's authentication parameters, see Send the sign-in request.
  • If you are authorizing a group, pass in resource=token-groups-claim.
  • If your authorization server prompts for consent, pass in prompt=consent.

This command produces a client authentication configuration file called client-config.yaml. Do not manually edit this file.

Note: Microsoft recommends that you have applied security update KB4019472 if you are using allatclaims scope.

Authenticating against a user cluster using the Kubectl Plugin for OIDC

This section is for employees who have created a user cluster and now want to use the Kubectl Plugin for OIDC.

1. Initialize the plugin using the client-config.yaml file:

   kubectl oidc login --clientconfig-file=client-config.yaml --user [NAME] \
       --kubeconfig [KUBECONFIG_OUTPUT_PATH]

   where:

   • [NAME] is your username.
   • [KUBECONFIG_OUTPUT_PATH] is the path to the kubeconfig file where the Kubectl Plugin for OIDC will store credentials.

   kubectl oidc login launches a browser where you can enter your credentials.

   The kubeconfig file provided now contains an ID token that kubectl can use to authenticate to the Kubernetes API server on the user cluster.

   Note: Windows users might have to run the command as kubectl-oidc.exelogin instead of kubectl oidc login.

2. Verify that the authentication was successful by running any kubectlcommand. For example:

   kubectl get nodes --kubeconfig [KUBECONFIG_OUTPUT_PATH]

Using OIDC with Google Cloud console

This section is for employees who have created a user cluster and now want to use Google Cloud console to authenticate against the cluster.

1. Verify that your cluster is configured for OIDC.

2. Verify that your cluster has been registered with Google Cloud, either automatically during cluster creation or manually.

3. Visit the Kubernetes clusters page in Google Cloud console.

   Visit the Kubernetes clusters page

4. In the list of clusters, locate your GKE On-Prem cluster, and click Login.

5. Select Authenticate with the Identity Provider configured for the cluster, and click LOGIN.

   You will be redirected to your identity provider, where you might need to log in or consent to Google Cloud console accessing your account. Then you will be redirected back to the Kubernetes clusters page in Google Cloud console.

Summary

Your enterprise runs an AD FS server that acts as your OpenID provider. Your OpenID provider knows about your two client applications: the Kubectl Plugin for OIDC and Google Cloud console. Your OpenID provider knows that your client applications can request the openid and allatclaims scopes.

In AD FS version before 5.0, Token-Groups Qualified Names LDAP attribute in your AD database is mapped to the groups claim in your OpenID provider. In 5.0 and later, the attribute is Token-Groups Qualified by Domain name. The provider returns tokens that include the employee's ID, the issuer ID, the openid claim and groups claim. The groups (Group in 5.0) claim lists the security groups that an employee belongs to.

Troubleshooting OIDC in GKE On-Prem

Invalid configuration

If Google Cloud console cannot read the OIDC configuration from your cluster, the LOGIN button will be disabled.

Invalid provider configuration

If your identity provider configuration is invalid, you will see an error screen from your identity provider after you click LOGIN. Follow the provider-specific instructions to correctly configure the provider or your cluster.

Invalid permissions

If you complete the authentication flow, but still don't see the details of the cluster, make sure you granted the correct RBAC permissions to the account that you used with OIDC. Note that this might a different account from the one you use to access Google Cloud console.

Error: missing 'RefreshToken' field in 'OAuth2Token' in credentials struct

You might get this error if the authorization server prompts for consent, but the required authentication parameter wasn't provided. Provide the prompt=consent parameter to GKE On-Prem configuration file's oidc: extraparams field, and regenerate the client authentication file with the --extra-params prompt=consent flag.

What's next

• Read the overview of authenticating with OpenID Connect in GKE On-Prem.

• Learn more about OAuth 2.0.

• Learn more about OpenID Connect.

• Learn more about scopes and claims.

• Learn about custom claims in ID tokens.