Anthos clusters on Azure is available for customers with an existing support relationship with Google Cloud. Contact your account representative for access.

Configure OpenID Connect (OIDC)

Learn how to configure Anthos clusters on Azure to use OpenID Connect (OIDC) for authentication to user clusters.

For an overview of the Anthos clusters on Azure authentication flow, see Security and authentication.

Anthos clusters on Azure supports OIDC as one of the authentication mechanisms for interacting with a user cluster's Kubernetes API server. With OIDC, you can manage access to Kubernetes clusters by using the standard procedures in your organization for creating, enabling, and disabling user accounts.

Before you begin

  • This documentation assumes that you are familiar with the following Kubernetes and OpenID concepts:

  • Cloud SDK and the gcloud command-line tool must be installed on each developer's local machine.

  • Headless systems (for example, a remote shell) are unsupported. To authenticate, you must open a browser on the local machine running the gcloud tool. The browser then prompts you to authorize your user account.

  • To authenticate through the Google Cloud Console, each cluster that you want to configure for OIDC authentication must be registered with Google Cloud.

Personas

This page refers to three personas. Each persona has a different set of responsibilities and permissions. One person might have multiple personas. The personas are the following:

  • Organization administrator: This person chooses an OpenID provider and registers client applications with the provider.

  • Cluster administrator: This person creates one or more user clusters and creates authentication configuration files for developers who use the clusters.

  • Developer: This person runs workloads on one or more clusters and uses OIDC to authenticate.

Register with the OpenID provider

This section is for organization administrators. You only need to perform this once for any given OIDC provider.

You can use any OpenID provider of your choice. For a list of certified providers, see OpenID Certification.

Anthos clusters on Azure supports OIDC login from the gcloud command-line tool or the Google Cloud Console.

Before your developers can use either of the clients for OIDC login, you need to register these these two applications with the OpenID provider.

General instructions

On the high-level, registration includes these steps:

  • Learn your provider's issuer URI. The gcloud tool or the Cloud Console sends authentication requests to this URI.

  • Create a single client ID that your provider uses to identify both the gcloud tool and the Cloud Console.

  • Create a single client secret that both the gcloud tool and the Cloud Console use to authenticate to the OpenID provider.

  • Configure your client with the following redirect URIs:

    • For gcloud tool, use http://localhost:PORT/callback, where PORT is of your choice and should be greater than 1024.

    • For Cloud Console, use https://console.cloud.google.com/kubernetes/oidc. This is the standard redirect URI used for OIDC login when using the Cloud Console.

  • Optional and may vary depending on the OIDC providers and your need:

    • Create a custom scope that the gcloud tool or the Cloud Console can use to request the user's username or group information.

    • Create a custom claim name that the provider uses to return the user's username or group information.

Examples

Following are example instructions for registering with Google Accounts, and Azure Active Directory.

For other OIDC providers or other advanced use cases, consult the official documentations of your selected OIDC provider.

Google

To use Google Accounts as an identity provider, you first create a consent screen, and then register an application.

  1. Create a consent screen.

    In this section, you configure Google's OAuth consent screen. When a developer in your organization authenticates to a user cluster, they are taken to this consent screen. At that time, they prove their identity to Google and give Google permission to create a token that provides identifying information to the OAuth client.

    1. In the Google Cloud Console, go to the OAuth consent screen page.

      Go to OAuth consent screen

    2. Select Internal, and click Create.

    3. For Application name, enter a name of your choice. For example, Anthos on Azure.

    4. Under Authorized domains, add google.com.

    5. Fill in additional fields as you see fit.

    6. Click Save.

  2. Registering a client application with Google

    1. In the Google Cloud Console, go to the Credentials page.

      Go to Credentials

    2. Click Create credentials, and select OAuth client ID.

    3. For Application type, select Web application.

    4. For Name, enter a name of your choice.

    5. Under Authorized redirect URIs, add your the following redirect URIs.

      • For the gcloud tool: http://localhost:PORT/callback, where PORT is of your choice and should be greater than 1024.

      • For the Cloud Console: https://console.cloud.google.com/kubernetes/oidc.

    6. Click Create.

    7. You are given a client ID and a client secret. Save these for later use.

Now your client registration is complete. You should have the following info for the next step:

  • Issuer URI: https://accounts.google.com.

  • Client ID: You were given this in the last step. You can also retrieve this from the client page on Google Cloud Console.

  • Client Secret: You were given this in the last step. You can also retrieve this from the client page on Google Cloud Console.

Now proceed to Enable OIDC on your cluster.

Azure

To register an OAuth client with Azure, complete the steps at the following links:

  1. If you haven't, Set up a tenant on Azure Active Directory.

  2. Register an application with the Microsoft identity platform.

  3. Open the App registrations page on the Azure Portal and select your application by name.

  4. Create a Client Secret.

    1. Click Add a certificate or secret under Essentials. A list of certificates and a list of secrets appears.

    2. Click New client secret. Name your secret and click Add.

    3. Save the Value* of the secret in a secure location. You will not be able to retrieve it after you close or refresh the page.

  5. Add redirect URIs.

    1. Return to the application page.

    2. Select Add a Redirect URI under Essentials. The Authentication page appears.

    3. Choose Add a platform, and panel named Configure platforms appears on the right.

    4. Choose Web. Under Redirect URIs, enter http://localhost:PORT/callback for the gcloud tool login flow. Pick a PORT greater than 1024. Click the Configure button.

    5. Click the Add URI button to add another URI, https://console.cloud.google.com/kubernetes/oidc, for the Cloud Console login.

    6. Click the Save button on the top.

Now your client registration is complete. You should have the following info for the next step:

  • Issuer URI: https://sts.windows.net/TENANT_ID/. The tenant ID is displayed as the Directory (tenant) ID on the Application page on Azure portal. Be sure to include the trailing / when you use this in the configuration.

  • Client ID: The client ID is displayed as the Application (client) ID on the Application page on Azure portal.

  • Client Secret: You were given this in the last step. You won't be able to retrieve this if you close the page upon secret creation. Make sure to save the value in a secure location (or generate a new secret if you lose track of the previous one).

Now proceed to Enable OIDC on your cluster.

Enable OIDC on your cluster

This section is for cluster administrators. You need to perform this for each cluster and each OpenID provider.

  1. Make sure you have registered with your selected OpenID provider.

  2. Configure the authentication details in the ClientConfig Kubernetes customer resource.

    For every cluster, there's a ClientConfig resource named default in the kube-public namespace. This default customer resource can be configured with the client registration info.

    To edit your default ClientConfig, make sure you can connect to your cluster via kubectl, and run the following command:

    kubectl --kubeconfig=KUBECONFIG_PATH edit ClientConfigs default -n kube-public
    

    Replace KUBECONFIG_PATH with the path to your cluster's kubeconfig file—for example $HOME/.kube/config.

    A text editor loads your cluster's ClientConfig resource. Add the spec.authentication.oidc object as instructed below. Do not modify any default data that has already been written. For concrete examples with Google or Azure AD, see Appendix: Example ClientConfig.

    apiVersion: authentication.gke.io/v2alpha1
    kind: ClientConfig
    metadata:
     name: default
     namespace: kube-public
    spec:
     authentication:
     - name: CONFIG_NAME
       oidc:
           clientID: CLIENT_ID
           clientSecret: CLIENT_SECRET
           issuerURI:  ISSUER_URI
           kubectlRedirectURI:  KUBECTL_REDIRECT_URI
           cloudConsoleRedirectURI: https://console.cloud.google.com/kubernetes/oidc
           scopes:  SCOPES
           userClaim:  USER_CLAIM
           # Optional fields
           userPrefix:  USER_PREFIX
           groupsClaim:  GROUPS_CLAIM
           groupPrefix:  GROUPS_PREFIX
           certificateAuthorityData: CERTIFICATE_STRING
           extraParams:  EXTRA_PARAMS
    
    # Rest of the resource is managed by Google. DO NOT MODIFY.
    ...
    

    Replace the following:

    • CONFIG_NAME: A unique name for this authentication method.

    • CLIENT_ID: ID for the OIDC client application that makes authentication requests to the OpenID provider.

    • CLIENT_SECRET: shared secret between the OIDC client application and OIDC provider.

    • ISSUER_URI: URI where authorization requests are sent to your OpenID. The Kubernetes API server uses this URI to discover public keys for verifying tokens. The URI must use HTTPS. For example:

      • https://accounts.google.com for Google Accounts

      • https://sts.windows.net/TENANT_ID/ for Azure AD

    • KUBECTL_REDIRECT_URI: the URI to redirect users going through the OAuth flow for your gcloud tool client. For example, http://localhost:PORT/callback.

    • SCOPES: comma-separated list of identifiers used to specify what access privileges are being requested. The openid and email scopes are required. Microsoft Azure and Okta require an extra offline_access scope.

    • USER_CLAIM: name of the JWT claim in the OIDC ID Token that holds the username. If this is missing from the ID Token, authentication will fail.

      • If you do not have a claim, use the value "".

      • You can choose other claims, such as email, name, or unique_name. This is specific to your OpenID provider and your user's account info.

    • USER_PREFIX: optional prefix prepended to username claims to prevent clashes with existing names.

    • GROUPS_CLAIM: optional name of the JWT claim in the OIDC ID Token that holds the user's group information. This claim in the ID token should contain a comma-separated list of strings that corresponds to Kubernetes RBAC groups.

    • GROUPS_PREFIX: optional prefix prepended to group claims to prevent clashes with existing names. For example, If you have two groups named foobar add a prefix gid-, the resulting group name is gid-foobar.

    • CERTIFICATE_STRING: optional base64-encoded PEM-encoded certificate for the certificate authority of the OIDC provider.

      • Leave this empty or unspecified if you don't need one.

      • To create the string, encode the certificate, including headers, into base64. Include the resulting string in certificateAuthorityData as a single line. For example: certificateAuthorityData: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tC...k1JSUN2RENDQWFT==.

    • EXTRA_PARAMS: optional comma-separated list of key-value pairs that will be query-encoded and sent with the authentication endpoint request to the OpenID provider.

      • If your authorization server prompts for consent:

        • For Microsoft Azure and Okta, pass in prompt=consent.

        • For Google Accounts, pass in prompt=consent,access_type=offline.

      • If you are authorizing a group, add resource=token-groups-claim.

  3. After you complete your ClientConfig, save the file. The kubectl command-line tool updates the ClientConfig on your cluster.Syntax errors will be caught and you will be prompt to re-edit the config to fix them.

Create and distribute a login config

This section is for cluster administrators.

Follow the steps below to create and distribute a required configuration file to your developers so they can authenticate to your cluster with gcloud tool using OIDC.

Create a login config

  1. To get started, make sure that the following prerequisites are met:

    • Update your gcloud tool installation by running the following commands:

      gcloud components update
      gcloud components install anthos-auth
      
    • Verify that the gcloud tool was installed successfully by running the following command:

      gcloud anthos auth version
      

      If the components are installed, the output includes details about the required arguments and available options.

  2. Create the login config. You need to do this for each cluster and each OpenID provider.

    After you create a cluster, you need to generate a configuration file for the cluster by using gcloud anthos create-login-config.

    To create the configuration, run the gcloud anthos command. For KUBECONFIG_PATH, use the path to your user cluster's kubeconfig file. On Linux and macOS, by default this file is at ~/.kube/config.

    gcloud anthos create-login-config --kubeconfig KUBECONFIG_PATH
    

    The command extracts the authentication information from the ClientConfig Kubernetes resource that you configured earlier, and downloads it into a kubectl-anthos-config.yaml file in your current directory. Your developer uses this file for OIDC authentication with gcloud tool. You should not modify this file manually.

    The command creates the file if it does not already exist. Otherwise, it examines the content of the existing file. If there's no login config for the given cluster, the login info is appended to the file; otherwise, the command fails and the file is kept unmodified.

Distribute the login config

You need to perform this for each developer that needs to log into the given cluster with the given OIDC provider.

Distribute the config file to developers that need to authenticate to your clusters. You can distribute the config by:

  • Placing the file in the default directory on developer workstations.
  • Securely distributing the file to developers.

The developer should store the configuration file in a well-known location on their workstations. For different OS types, the location may vary:

  • Linux: $HOME/.config/google/anthos/kubectl-anthos-config.yaml, where $HOME is the user's home directory.

  • macOS: $HOME/Library/Preferences/google/anthos/kubectl-anthos-config.yaml, where $HOME is the user's home directory.

  • Windows: %APPDATA%/google/anthos/kubectl-anthos-config.yaml, where %APPDATA% is the user's application data directory.

After the login config has been distributed, your developers are ready to use gcloud tool to access the cluster.

Authenticate to your cluster

This section is for developers or cluster administrators.

You can use OIDC to authenticate to your cluster in the following ways:

  • Use the gcloud command-line tool
  • Use Connect on the Google Cloud Console

Use the gcloud tool to authenticate

  1. To get started, make sure that the following prerequisites are met:

    • Your cluster administrator must enable OIDC on your cluster.
    • Obtain a login config.
    • Update your gcloud tool installation by running the following commands:

      gcloud components update
      gcloud components install anthos-auth
      
    • Verify that the gcloud tool was installed successfully by running the following command, which should respond with details about the required arguments and available options:

      gcloud anthos auth version
      
  2. Authenticate to your cluster by running the gcloud anthos auth login command.

    If you placed the login config in the default location and configured the cluster name, you can use gcloud anthos auth login with no options. You can also configure the cluster, user, and other authentication details with optional parameters.

    Default

    Find the cluster resource name in the following format and use it for cluster login:

     "projects/PROJECT_NUMBER/locations/GOOGLE_CLOUD_LOCATION/azureClusters/CLUSTER_ID",
    

    The default command reads the login config from the kubectl-anthos-config.yaml file in your current directory, and appends the kubeconfig file (with user credentials from OIDC) into $HOME/.kube/config:

    CLUSTER_NAME=$(gcloud alpha container azure clusters describe CLUSTER_ID \
       --location GOOGLE_CLOUD_LOCATION \
       --format "value(name)")
    gcloud anthos auth login --cluster $CLUSTER_NAME
    

    Optional parameters

    Find the cluster resource name in the following format and use it for cluster login:

     "projects/PROJECT_NUMBER/locations/GOOGLE_CLOUD_LOCATION/azureClusters/CLUSTER_ID"),
    

    The gcloud anthos auth login command supports the following optional parameters:

     CLUSTER_NAME=$(gcloud alpha container azure clusters describe CLUSTER_ID \
         --location GOOGLE_CLOUD_LOCATION \
         --format "value(name)")
     gcloud anthos auth login \
         --cluster $CLUSTER_NAME \
         --user USERNAME \
         --login-config ANTHOS_CONFIG_YAML \
         --login-config-cert LOGIN_CONFIG_CERT_PEM \
         --kubeconfig=OUTPUT_KUBECONFIG \
         --dry-run
    

    The parameters are described in the following table.

    Parameter Description
    cluster The name of the cluster to authenticate to.
    Defaults to the cluster in kubectl-anthos-config.yaml.
    user Username for credentials in kubeconfig.
    Defaults to {cluster-name}-anthos-default-user.
    login-config Either the path to the configuration file generated by the cluster admin for the developer or a URL hosting the file.
    Defaults to kubectl-anthos-config.yaml.
    login-config-cert If using a URL for login-config, the path to the CA certificate file for making HTTPS connections.
    kubeconfig Output path to the kubeconfig file that contains tokens.
    Defaults to $HOME/.kube/config.
    dry-run Lets you test your command-line options without changing your configuration or cluster.
  3. The gcloud anthos login command launches a browser that asks the user to log in with their enterprise credentials, performs the OIDC credential exchange, and acquires the relevant tokens. The gcloud tool then writes the tokens to a kubeconfig file. kubectl uses this file to authenticate to the user cluster.

  4. To verify that the authentication was successful, run any kubectl command with your kubeconfig file (if you used the preceding default command, you can skip this or pass in $HOME/.kube/config):

    kubectl get nodes --kubeconfig OUTPUT_KUBECONFIG
    

Use the Cloud Console to authenticate

  1. To get started, make sure that the following prerequisites are met:

  2. Authenticate with the Google Cloud Console by initiating the authentication flow from the Kubernetes clusters page:

    Console

    1. In the Cloud Console, go to the Kubernetes clusters page.

      Go to Kubernetes clusters

    2. Locate your Anthos clusters on Azure in the list, and then click Login.

    3. Select Authenticate with the Identity Provider configured for the cluster, and then click Login.

      You are redirected to your identity provider, where you might need to log in or consent to the Cloud Console accessing your account. You are then redirected back to the Kubernetes clusters page on the Cloud Console.

Advanced topics

Authorize users and groups

This section describes how to simplify OIDC authentication with Kubernetes RBAC.

Many providers encode user-identifying properties, such as email and user IDs, in a token. However, these properties have implicit risks for authentication policies:

  • User IDs can make policies difficult to read and audit.
  • Using email addresses can create both an availability risk (if a user changes their primary email) and a security risk (if an email can be re-assigned).

Instead of assigning user IDs, we recommend group policies, which can be both persistent and easier to audit.

Suppose your provider creates identity tokens that include the following fields:

{
  'iss': 'https://server.example.com'
  'sub': 'u98523-4509823'
  'groupList': ['developers@example.corp', 'us-east1-cluster-admins@example.corp']
  ...
}

Given this token format, you'd populate your configuration file's oidc specification like so:

issuerURI: 'https://server.example.com'
username: 'sub'
usernamePrefix: 'uid-'
group: 'groupList'
groupPrefix: 'gid-'
extraParams: 'resource=token-groups-claim'
...

After you've created your user cluster, you use Kubernetes role-based access control (RBAC) to grant privileged access to the authenticated users.

In the following example, you create a ClusterRole that grants its users read- only access to the cluster's Secrets, and create a ClusterRoleBinding resource to bind the role to the authenticated group.

  1. Define a ClusterRole. Copy the following YAML into a file named secret-reader-role.yaml.

    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: secret-reader
    rules:
    - apiGroups: [""]
      # The resource type for which access is granted
      resources: ["secrets"]
      # The permissions granted by the ClusterRole
      verbs: ["get", "watch", "list"]
    
  2. Define a ClusterRoleBinding. Copy the following YAML into a file named secret-reader-admins.yaml.

    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: read-secrets-admins
    subjects:
      # Allows anyone in the "us-east1-cluster-admins" group to
      # read Secrets in any namespace within this cluster.
    - kind: Group
      name: gid-us-east1-cluster-admins # Name is case sensitive
      apiGroup: rbac.authorization.k8s.io
      # Allows this specific user to read Secrets in any
      # namespace within this cluster
    - kind: User
      name: uid-u98523-4509823
      apiGroup: rbac.authorization.k8s.io
    roleRef:
      kind: ClusterRole
      name: secret-reader
      apiGroup: rbac.authorization.k8s.io
    
  3. Apply secret-reader-role.yaml and secret-reader-admins.yaml to your cluster with kubectl.

      kubectl apply -f secret-reader-role.yaml && \
      kubectl apply -f secret-reader-admins.yaml
    

    Users granted access in read-secrets-admins now have access to read Secrets in your cluster.

Multiple OIDC configs

You can configure multiple OIDC providers for a single cluster. This can be useful if you want to configure the cluster to allow your developers to authenticate with multiple OIDC providers. This works best when your developers Authenticate with gcloud tool.

When you configure the OIDC providers using kubectl edit ClientConfig -n kube-public default, you can add multiple OIDC configs under the spec.authentication list. Below is an example:

apiVersion: authentication.gke.io/v2alpha1
kind: ClientConfig
metadata:
  name: default
  namespace: kube-public
spec:
  # You can add multiple OIDC configs under authentication.
  authentication:
  - name: oidc-config-1
    oidc:
      issuerURI: ISSUER_1
      ...
  - name: oidc-config-2
    oidc:
      issuerURI: ISSUER_2
      ...
  ...

If you apply the ClientConfig like this and create and distribute the login config to the developers, they will be able to decide which OIDC provider to use at login time, like this:

$ gcloud anthos auth login --cluster <your-cluster-name>
Please select your preferred authentication option for cluster
 [1] oidc-config-1
 [2] oidc-config-2
 [3] cancel
Please enter your numeric choice:

Troublehooting

Redirect URI mismatch

You might run into errors if your configured redirect URIs in the ClienctConfig resource are not registered to your OpenID provider. Work with your Organization admin and Cluster admin to make sure that the URIs match each other.

Example error messages:

  • Google: The redirect URI in the request, <URL configured in ClientConfig>, does not match the ones authorized for the OAuth client. To update the authorized redirect URIs, visit: ...

  • Azure: The reply URL specified in the request does not match the reply URLs configured for the application.

Console: Failed to load OIDC config from the cluster

This could be caused by an unsuccessful cluster registration. For debugging help, see Connect troubleshooting.

Appendix: Example ClientConfig

Following is an example ClientConfig. You can refer to this example when you execute kubectl edit ClientConfig -n kube-public default.

Google

...
spec:
  authentication:
  - name: oidc-google
    oidc:
      clientID: CLIENT_ID
      clientSecret: CLIENT_SECRET
      cloudConsoleRedirectURI: https://console.cloud.google.com/kubernetes/oidc
      extraParams: prompt=consent,access_type=offline
      issuerURI: https://accounts.google.com
      kubectlRedirectURI: http://localhost:PORT/callback
      scopes: openid,email
      userClaim: email
...

Azure

...
spec:
  authentication:
  - name: oidc-ad
    oidc:
      clientID: CLIENT_ID
      clientSecret: CLIENT_SECRET
      cloudConsoleRedirectURI: https://console.cloud.google.com/kubernetes/oidc
      extraParams: prompt=consent
      issuerURI: https://sts.windows.net/TENANT_ID/
      kubectlRedirectURI: http://localhost:PORT/callback
      scopes: openid,email,offlie_access
      userClaim: email
...

What's next