This page describes how to install Config Connector on a Google Kubernetes Engine (GKE) cluster using the Config Connector add-on.
If you are using GKE on Google Cloud, we recommend you install Config Connector using this add-on. For details on other installation options, see Choosing an installation type.
Before you begin
Before you start, make sure you have performed the following tasks:
- Ensure that you have enabled the Google Kubernetes Engine API. Enable Google Kubernetes Engine API
- Ensure that you have installed the Google Cloud CLI.
- Set up default Google Cloud CLI settings for your project by using one of the following methods:
- Use
gcloud init
, if you want to be walked through setting project defaults. - Use
gcloud config
, to individually set your project ID, zone, and region. -
Run
gcloud init
and follow the directions:gcloud init
If you are using SSH on a remote server, use the
--console-only
flag to prevent the command from launching a browser:gcloud init --console-only
- Follow the instructions to authorize the gcloud CLI to use your Google Cloud account.
- Create a new configuration or select an existing one.
- Choose a Google Cloud project.
- Choose a default Compute Engine zone.
- Choose a default Compute Engine region.
- Set your default project ID:
gcloud config set project PROJECT_ID
- Set your default Compute Engine region (for example,
us-central1
):gcloud config set compute/region COMPUTE_REGION
- Set your default Compute Engine zone (for example,
us-central1-c
):gcloud config set compute/zone COMPUTE_ZONE
- Update
gcloud
to the latest version:gcloud components update
gcloud init
gcloud config
By setting default locations, you can avoid errors in gcloud CLI like the
following: One of [--zone, --region] must be supplied: Please specify location
.
- Select or create a Google Cloud project to install Config Connector in.
Installing the Config Connector add-on
You use the Config Connector add-on by creating a new GKE cluster, or enabling it on an existing cluster. After installing the Config Connector add-on, you configure your Config Connector installation with your Google service accounts and your namespaces.
Requirements
The Config Connector add-on has the following requirements:
You must use a GKE version of:
- 1.15.11-gke.5 and later
- 1.16.8-gke.8 and later
- 1.17.4-gke.5 and later
You must enable a Workload Identity pool and Kubernetes Engine Monitoring on the clusters where you enable Config Connector.
Setting up a GKE cluster
You can use the Config Connector add-on on a new or existing cluster.
Creating a new cluster with the Config Connector add-on enabled
You can create a GKE cluster using the gcloud CLI or the Google Cloud console.
gcloud
To create a cluster with the Google Cloud CLI run the following command:
gcloud container clusters create CLUSTER_NAME \
--release-channel CHANNEL \
--addons ConfigConnector \
--workload-pool=PROJECT_ID.svc.id.goog \
--logging=SYSTEM \
--monitoring=SYSTEM
Replace the following:
- CLUSTER_NAME with the name of your GKE cluster.
- CHANNEL with a
GKE release channel,
rapid
andregular
are supported. - PROJECT_ID with your Google Cloud project ID.
Cloud console
To create a cluster with the Google Cloud console, perform the following steps:
Visit the Google Kubernetes Engine menu in Cloud console.
Click Create. The Create a Kubernetes cluster page appears.
Specify a Name for your cluster.
Choose a supported Master version.
Configure the rest of your cluster as you want.
From the navigation pane, under Cluster, click Security.
Select the Enable Workload Identity checkbox.
From the navigation pane on the left side, under Cluster, click Features.
Select the Enable Config Connector checkbox.
Click Create.
After you've created the cluster, move on to Creating an identity.
Enabling the Config Connector add-on on an existing cluster
You can enable the Config Connector add-on on an existing GKE cluster
with gcloud
or the Google Cloud console.
Prerequisites
Enabling the Config Connector add-on on an existing cluster has the following prerequisites:
- You need a cluster that meets the requirements for the Config Connector add-on.
Set up Workload Identity on the cluster where you want to install Config Connector.
To enable Workload Identity for a node pool, use the gcloud
command-line
tool:
gcloud container node-pools update NODE_POOL \
--workload-metadata=GKE_METADATA \
--cluster CLUSTER_NAME
Replace the following:
- NODE_POOL with your node pool's name
- CLUSTER_NAME with your cluster's name
Enabling the Config Connector add-on
You can enable the Config Connector add-on in an existing GKE cluster with the Google Cloud CLI or the Google Cloud console.
gcloud
To enable the Config Connector add-on in an existing GKE cluster use the Google Cloud CLI:
gcloud container clusters update CLUSTER_NAME \
--update-addons ConfigConnector=ENABLED
Replace CLUSTER_NAME with the name of your GKE cluster.
Cloud console
Visit the Google Kubernetes Engine menu in Cloud console.
Select the cluster that you want to install Config Connector on. The Cluster Details page appears.
Under the Features section, locate the Config Connector row and click
Edit.Select the Enable Config Connector checkbox and click Save Changes to update your cluster.
Creating an identity
Config Connector creates and manages Google Cloud resources by authenticating with a Identity and Access Management (IAM) service account and using GKE's Workload Identity to bind IAM service accounts with Kubernetes service accounts.
To create the identity, complete the following steps:
-
Create an IAM service account. If you want to use an existing service account, you
can use that account and skip this step.
To create the service account, use the following command:gcloud iam service-accounts create SERVICE_ACCOUNT_NAME
ReplaceSERVICE_ACCOUNT_NAME
with a name for your service account. -
Give the IAM service account elevated permissions on your project:
gcloud projects add-iam-policy-binding PROJECT_ID \ --member="serviceAccount:SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/editor"
Replace the following:PROJECT_ID
with your project ID.SERVICE_ACCOUNT_NAME
with your service account's name.
-
Create an IAM policy binding between the IAM
service account and the predefined Kubernetes service account that
Config Connector runs:
gcloud iam service-accounts add-iam-policy-binding \ SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com \ --member="serviceAccount:PROJECT_ID.svc.id.goog[cnrm-system/cnrm-controller-manager]" \ --role="roles/iam.workloadIdentityUser"
Replace the following:SERVICE_ACCOUNT_NAME
with your service account's name.PROJECT_ID
with your project ID.
To learn more about creating service accounts, see Creating and managing service accounts.
Configuring Config Connector
To complete the installation, create a configuration file for
the ConfigConnector
CustomResource,
then apply it using the kubectl apply
command. The Config Connector Operator installs
Google Cloud Resource CRDs and Config Connector components in your cluster.
To configure the operator, complete the following steps:
-
Copy the following YAML file into a file named
configconnector.yaml
:# configconnector.yaml apiVersion: core.cnrm.cloud.google.com/v1beta1 kind: ConfigConnector metadata: # the name is restricted to ensure that there is only one # ConfigConnector resource installed in your cluster name: configconnector.core.cnrm.cloud.google.com spec: mode: cluster googleServiceAccount: "SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com"
Replace the following:SERVICE_ACCOUNT_NAME
with your service account's name.PROJECT_ID
with your project ID.
-
Apply the configuration to your cluster with
kubectl apply
:kubectl apply -f configconnector.yaml
Specifying where to create your resources
Config Connector can organize resources by project, folder, or organization, which is the same way you would organize resources with Google Cloud.
Before creating resources with Config Connector, you must configure where to create your resources. To determine where to create the resource, Config Connector uses an annotation on either the resource configuration or an existing Namespace. For more information, see Organizing resources.
If you do not have a Namespace for this purpose, create one withkubectl
.
kubectl create namespace NAMESPACE
Replace NAMESPACE
with your namespace name. For example
config-connector
.
Select a tab to choose where you want Config Connector to create resources.
Project
To create resources in a certain project, run the following command:
kubectl annotate namespace \ NAMESPACE cnrm.cloud.google.com/project-id=PROJECT_ID
Replace the following:
NAMESPACE
with your namespace name.PROJECT_ID
with your Google Cloud project ID.
Folder
To create resources in a certain folder, run the following command:
kubectl annotate namespace \ NAMESPACE cnrm.cloud.google.com/folder-id=FOLDER_ID
Replace the following:
NAMESPACE
with your namespace name.FOLDER_ID
with your Google Cloud folder ID.
Organization
To create resources in a certain organization, run the following command:
kubectl annotate namespace \ NAMESPACE cnrm.cloud.google.com/organization-id=ORGANIZATION_ID
Replace the following:
NAMESPACE
with your namespace name.ORGANIZATION_ID
with your Google Cloud organization ID.
When you annotate your namespace, Config Connector creates resources in the corresponding project, folder or organization. To learn more about how Config Connector uses Kubernetes namespaces, see Kubernetes Namespaces and Google Cloud projects.
Verifying your installation
Config Connector runs all of its components in a namespace named cnrm-system
.
You can verify the Pods are ready by running the following command:
kubectl wait -n cnrm-system \
--for=condition=Ready pod --all
If Config Connector is installed correctly, the output is similar to the following:
pod/cnrm-controller-manager-0 condition met
Upgrading Config Connector
Upgrades to your cluster's Config Connector add-on are managed by Google Cloud.
If you want to routinely have Config Connector upgraded to the latest version, enroll your clusters in the Rapid channel. If you want even faster upgrades, consider using Config Controller.
Google Cloud maps specific GKE minor versions to specific Config Connector versions. When your cluster is upgraded to a new GKE minor version, your cluster's Config Connector add-on is automatically upgraded.
Every 1-2 weeks, GKE minor versions that are available on the Rapid channel only (that is, GKE minor versions that are not available on the Regular or Stable channels) are upgraded to the latest Config Connector version. Clusters on those GKE minor versions are automatically upgraded accordingly. To learn about how to determine which versions are available on which release channels, see Viewing the default and available versions for release channels.
Google Cloud does not upgrade GKE minor versions that are already available on the Regular or Stable channels to newer Config Connector versions for stability reasons, except in emergency situations where a widespread issue has been detected.
Uninstalling Config Connector
To uninstall Config Connector, complete the following steps:
Use
kubectl delete
to remove the Config Connector CRDs along with controller components:kubectl delete ConfigConnector configconnector.core.cnrm.cloud.google.com --wait=true
Disable the Config Connector add-on in your cluster using the gcloud CLI or the Google Cloud console:
gcloud
To disable the Config Connector add-on with
gcloud
, run the following command:gcloud container clusters update CLUSTER_NAME --update-addons ConfigConnector=DISABLED
Replace CLUSTER_NAME with the name of the cluster that has Config Connector add-on installed.
Cloud console
To disable the Config Connector add-on from the Google Cloud console, perform the following steps.
Go to the Google Kubernetes Engine Clusters page on the Google Cloud console and select the cluster that you want to update.
Click Edit. The Edit clusters screen appears.
Click Add-ons.
Select Config Connector and choose Disabled.
Click Save to update your cluster.
Troubleshooting
The following sections provide you with troubleshooting tips for your Config Connector installation.
Troubleshooting Config Connector add-on installations
If you can't enable the Config Connector add-on successfully, the following
error message appears: Node version 1.15.x-gke.x s unsupported
. To solve this
error, verify that the version of the GKE cluster meets the
requirements.
To get all valid versions for your clusters, run the following command:
gcloud container get-server-config --format "yaml(validMasterVersions)" \
--zone ZONE
Replace ZONE with the Compute Engine zone.
Pick a version from the list that meets the requirements. The error message also appears if Workload Identity or Kubernetes Engine Monitoring are disabled. Ensure these features are enabled to fix the error.
Troubleshooting permissions for resource reconciliations
If Config Connector can't reconcile resources successfully and the logs contain
the error message The caller does not have permission, forbidden.
,
then Workload Identity might not be enabled on your GKE cluster
and/or node pool.
To investigate, complete the following steps:
- Save the following Pod configuration as
wi-test.yaml
:apiVersion: v1 kind: Pod metadata: name: workload-identity-test namespace: cnrm-system spec: containers: - image: google/cloud-sdk:slim name: workload-identity-test command: ["sleep","infinity"] serviceAccountName: cnrm-controller-manager
- Create the Pod in your GKE cluster:
kubectl apply -f wi-test.yaml
- Open an interactive session in the Pod:
kubectl exec -it workload-identity-test \ --namespace cnrm-system \ -- /bin/bash
- List your identity:
gcloud auth list
-
Verify that the identity listed matches the Google service account bound to your resources.
If you see the Compute Engine default service account instead, then that means that Workload Identity is not enabled on your GKE cluster and/or node pool.
- Exit the interactive session, then delete the Pod from your GKE cluster:
kubectl delete pod workload-identity-test \ --namespace cnrm-system
What's next
- Get started with Config Connector.
- Learn how to install Config Connector for other cloud providers.
- Learn about advanced installation options for Config Connector.