Installing, upgrading, and uninstalling Config Connector

This topic describes how to install Config Connector on your cluster.

Before you begin

Before you start, make sure you have performed the following tasks:

Set up default gcloud settings using one of the following methods:

  • Using gcloud init, if you want to be walked through setting defaults.
  • Using gcloud config, to individually set your project ID, zone, and region.

Using gcloud init

  1. Run gcloud init and follow the directions:

    gcloud init

    If you are using SSH on a remote server, use the --console-only flag to prevent the command from launching a browser:

    gcloud init --console-only
  2. Follow the instructions to authorize gcloud to use your Google Cloud account.
  3. Create a new configuration or select an existing one.
  4. Choose a Google Cloud project.
  5. Choose a default Compute Engine zone.

Using gcloud config

  • Set your default project ID:
    gcloud config set project project-id
  • If you are working with zonal clusters, set your default compute zone:
    gcloud config set compute/zone compute-zone
  • If you are working with regional clusters, set your default compute region:
    gcloud config set compute/region compute-region
  • Update gcloud to the latest version:
    gcloud components update
  • Select a Google Cloud project where you will install Config Connector.

Choosing your installation type

You can install Config Connector in one of the following ways:

  • Using the Config Connector add-on with Workload Identity (Beta)
  • Manually using a GKE Workload Identity,
  • Manually using a Google Cloud Identity and Service Account, or
  • Manually using Namespaced mode, to support managing multiple projects, each with their own Google Cloud identities.

For more information on these installation types, see Choosing an installation type.

Select a tab below to choose your installation type.

Add-on

You use the Config Connector add-on by creating a new cluster or enabling it on an existing cluster.

Requirements

The Config Connector add-on requires a GKE version of:

  • 1.15.11-gke.5 and higher
  • 1.16.8-gke.8 and higher
  • 1.17.4-gke.5 and higher

Additionally, your cluster must have Workload Identity and Kubernetes Engine Monitoring enabled.

Creating a new cluster with the Config Connector add-on enabled

You can create a GKE cluster with gcloud or the Google Cloud Console. These instructions create a new cluster with a Workload Identity pool and Kubernetes Engine Monitoring.

After installing the Config Connector add-on, you configure your Google Service Accounts and Namespaces.

gcloud

To create a cluster with gcloud, run the following command:

gcloud beta container clusters create [CLUSTER_NAME] \
--cluster-version [CLUSTER_VERSION] --addons ConfigConnector \
--workload-pool=[PROJECT_ID].svc.id.goog --enable-stackdriver-kubernetes

Replace the following:

  • [CLUSTER_NAME] with the name of your GKE cluster.
  • [CLUSTER_VERSION] with a supported cluster version.
  • [PROJECT_ID] with your Google Cloud project ID.

Cloud Console

To create a cluster with the Google Cloud Console perform the following steps:

  1. Visit the Google Kubernetes Engine menu in Cloud Console.

    Visit the Google Kubernetes Engine menu

  2. Click the Add cluster button, then click Create Cluster. The Create a Kubernetes cluster screen appears.

  3. Specify a Name for your cluster.

  4. Choose a supported Master version.

  5. Configure your cluster as desired.

  6. From the navigation pane, under Cluster, click Security.

  7. Select the Enable Workload Identity checkbox.

  8. From the navigation pane on the left side, under Cluster, click Features.

  9. Select the Enable Config Connector checkbox.

  10. Click Create.

Once you have completed this task, continue to Configuring the Config Connector add-on.

Enabling the Config Connector add-on on an existing cluster

You can enable the Config Connector add-on on a GKE cluster with gcloud or the Google Cloud Console.

Before you begin
  • Have a cluster that meets the requirements for the Config Connector add-on.
  • Set up Workload Identity on the cluster where you will install Config Connector.
Enabling the Config Connector add-on.

You can enable the Config Connector add-on in an existing GKE cluster with gcloud or the Google Cloud Console.

After installing the Config Connector add-on, you configure your Config Connector installation with your Google Service Accounts and your Namespaces.

gcloud

To enable the Config Connector add-on in an existing GKE cluster with gcloud, run the following command:

gcloud beta container clusters update [CLUSTER_NAME] --update-addons ConfigConnector=ENABLED

Replace [CLUSTER_NAME] with the name of your GKE cluster.

Cloud Console

  1. Visit the Google Kubernetes Engine menu in Cloud Console.

    Visit the Google Kubernetes Engine menu

  2. Select the cluster where you want to install Config Connector. The Cluster Details screen appears.

  3. Click Edit.

  4. Click Add-ons. The menu expands.

  5. Under Config Connector, choose Enabled.

  6. Click Save to update your cluster.

Once you have completed this task, continue to Configuring the Config Connector add-on.

Confirm Config Connector Operator installation

The Config Connector add-on installs a Kubernetes Operator that configures Config Connector. To check the status of your Config Connector Operator installation, run the following kubectl command:

gcloud

kubectl wait pod/configconnector-operator-0 -n configconnector-operator-system --for=condition=Initialized

When the Config Connector Operator is running, the command returns that there is exactly one Pod with a STATUS of Running.

NAME                                        READY   STATUS    RESTARTS   AGE
configconnector-operator-0   1/1          Running    0                    2m

Cloud Console

To check the status of the Config Connector Operator from the Google Cloud Console, visit the Google Kubernetes Engine Workloads menu.

Visit the Workloads menu

If the Config Connector Operator is installed correctly, a workload named configconnector-operator is displayed with a Status of OK.

Configuring Config Connector

To complete installing the Config Connector add-on, you create a configuration file for the ConfigConnector CustomResource, then apply it using the kubectl apply command.

  1. Copy the following YAML into a file named configconnector.yaml.

    # configconnector.yaml
    apiVersion: core.cnrm.cloud.google.com/v1beta1
    kind: ConfigConnector
    metadata:
      # the name is restricted to ensure that there is only ConfigConnector instance installed in your cluster
      name: configconnector.core.cnrm.cloud.google.com
    spec:
      # the default Google Service Account used by ConfigConnector to authenticate Google Cloud APIs
      googleServiceAccount: "[GSA]@[PROJECT_ID].iam.gserviceaccount.com"
    

    Replace the following:

    • [GSA] with your Cloud IAM Service Account name
    • [PROJECT_ID] with the Project hosting Config Connector
  2. Apply the configuration to your cluster with kubectl apply:

    kubectl apply -f configconnector.yaml
    

The Config Connector Operator installs Google Cloud Resource CRDs and Config Connector components in your cluster.

To view available Google Cloud resources in Config Connector, run the following command:

kubectl get crds --selector cnrm.cloud.google.com/managed-by-kcc=true

Configure Config Connector to watch your namespaces

For the rest of this section, the Google Cloud Project where you install Config Connector is known as the host project, or [HOST_PROJECT_ID]. The other projects where you manage resources are known as the managed projects, or [MANAGED_PROJECT_ID]. These could be the same project if you only intend to use Config Connector to create Google Cloud resources in the same project as your cluster.

Create a Namespace

You can skip this step if you already have a Namespace configured to match the GSA that will create Google Cloud resources.

Use kubectl to create a new Namespace by running the following command:

kubectl create namespace [NAMESPACE]

Replace [NAMESPACE] with the Namespace you are configuring.

Create a service account

Next, create a Cloud IAM service account to bind your Google Service Account (GSA) and Kubernetes Service Account (KSA).

  1. Configure gcloud to use your host project.

    gcloud config set project [HOST_PROJECT_ID]
    

    Replace [HOST_PROJECT_ID] with your host project ID.

  2. Create a service account. If you have an existing service account, you can use it instead of creating a new service account. Use gcloud to create the service account by running the following command:

    gcloud iam service-accounts create [NAMESPACE_GSA]
    

    Replace [NAMESPACE_GSA] with the GSA bound to your Namespace.

  3. Give the Cloud IAM Service Account elevated permissions on your managed project.

    gcloud config set project [MANAGED_PROJECT_ID]
    gcloud projects add-iam-policy-binding [MANAGED_PROJECT_ID] \
    --member="serviceAccount:[NAMESPACE_GSA]@[HOST_PROJECT_ID].iam.gserviceaccount.com" \
    --role="roles/owner"
    

    Replace the following:

    • [MANAGED_PROJECT_ID] with your managed project's ID
    • [NAMESPACE_GSA] with the GSA bound to your Namespace
    • [HOST_PROJECT_ID] with your host project's ID
Create a ConfigConnectorContext

To create Google Cloud resources, you need to configure Config Connector to watch your Namespace by adding a ConfigConnectorContext CustomResource object in the desired Namespace.

  1. Copy the following YAML into a file named configconnectorcontext.yaml.

    # configconnectorcontext.yaml
    
    apiVersion: core.cnrm.cloud.google.com/v1beta1
    kind: ConfigConnectorContext
    metadata:
      # you can only have one ConfigConnectorContext per Namespace
      name: configconnectorcontext.core.cnrm.cloud.google.com
      namespace: [NAMESPACE]
    spec:
      # The Google Service Account used to authenticate Google Cloud APIs in this Namespace
      googleServiceAccount: "[NAMESPACE_GSA]@[HOST_PROJECT_ID].iam.gserviceaccount.com"
    

    Replace the following:

    • [NAMESPACE] with your Namespace
    • [NAMESPACE_GSA] with the GSA bound to your Namespace
    • [HOST_PROJECT_ID] with your host project's ID
  2. Apply the file to your cluster with kubectl.

    kubectl apply -f configconnectorcontext.yaml
    
  3. Verify that the Config Connector Operator created a Kubernetes service account for your namespace with kubectl by running the following command:

    kubectl get serviceaccount/cnrm-controller-manager-[NAMESPACE] -n cnrm-system
    

    Replace [NAMESPACE] with your Namespace name.

  4. Verify that the Config Connector controller Pod is running for your namespace with kubectl by running the following command:

    kubectl wait -n cnrm-system \
    --for=condition=Initialized pod \
    cnrm-controller-manager-[NAMESPACE]-0
    

    Replace [NAMESPACE] with your namespace name.

Bind your ConfigConnector Kubernetes Service Account to Google Service Account

Create a Cloud Identity and Access Management policy binding between the Cloud IAM Service Account and the Config Connector Kubernetes service account. You bind the service accounts by running the following gcloud command:

gcloud config set project [HOST_PROJECT_ID]
gcloud iam service-accounts add-iam-policy-binding \
[NAMESPACE_GSA]@[HOST_PROJECT_ID].iam.gserviceaccount.com \
--member="serviceAccount:[HOST_PROJECT_ID].svc.id.goog[cnrm-system/cnrm-controller-manager-[NAMESPACE]]" \
--role="roles/iam.workloadIdentityUser"

Replace the following:

  • [HOST_PROJECT_ID] with your host project's ID
  • [NAMESPACE_GSA] with the GSA bound to your Namespace
  • [NAMESPACE] with your Namespace

Workload Identity

Prerequisites

Before you install Config Connector to a cluster, perform the following steps.

  • Create or identify a GKE cluster where Config Connector has not yet been installed.
  • Set up Workload Identity on the cluster where you will install Config Connector. If you want to use the same Google Service Account (GSA) and Kubernetes Service Account (KSA) you created when installing Workload Identity, have their IDs available.

  • Configure kubectl to connect to your cluster.

Create an Identity

Setting up the identity includes:

  • Creating an Cloud IAM Service Account.
  • Creating a binding between the Cloud IAM Service Account and Config Connector's Kubernetes service account.
  1. Create the cnrm-system Service Account with gcloud:

    gcloud iam service-accounts create cnrm-system
  2. Give the IAM Service Account elevated permissions on your project. Replace [PROJECT_ID] with your project ID.

    gcloud projects add-iam-policy-binding [PROJECT_ID] \
    --member="serviceAccount:cnrm-system@[PROJECT_ID].iam.gserviceaccount.com" \
    --role="roles/owner"
    
  3. Create a Cloud IAM policy binding between the IAM Service Account and the predefined Kubernetes service account run by Config Connector. Replace [PROJECT_ID] with your project ID.

    gcloud iam service-accounts add-iam-policy-binding \
    cnrm-system@[PROJECT_ID].iam.gserviceaccount.com \
    --member="serviceAccount:[PROJECT_ID].svc.id.goog[cnrm-system/cnrm-controller-manager]" \
    --role="roles/iam.workloadIdentityUser"
    

Deploying Config Connector

  1. Download the latest installation bundle tar file:

     gsutil cp gs://cnrm/latest/release-bundle.tar.gz release-bundle.tar.gz
  2. Extract the tar file.

    tar zxvf release-bundle.tar.gz
  3. Provide your project ID in the controller's installation manifest. Before running the following command, replace [PROJECT_ID] with your project ID.

    sed -i.bak 's/${PROJECT_ID?}/[PROJECT_ID]/' install-bundle-workload-identity/0-cnrm-system.yaml
  4. Apply the manifests to your cluster.

    kubectl apply -f install-bundle-workload-identity/

GCP Identity

Prerequisites

Before you can install Config Connector, perform the following steps:

  1. Create a GKE cluster.
  2. Configure kubectl to connect to your cluster.

Creating a ClusterRoleBinding

Config Connector needs permission to create Kubernetes Roles before it can create resources.

Verify that you can create Roles by running the following command.

kubectl auth can-i create roles

If the output is yes, continue to Creating an identity.

If the output is no, create a ClusterRoleBinding in your cluster. This allows you to create Roles. Replace [ACCOUNT_EMAIL] with the email associated with your Google Cloud account.

kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin \
--user [ACCOUNT_EMAIL]

The outputs should contain the phrase cluster-admin-binding created. If it does not, contact your Google Cloud account or GKE cluster administrator about permissions.

Creating an identity

A Config Connector cluster needs a Google Cloud identity to communicate with other resources. To set up the identity, you create an Cloud Identity and Access Management (Cloud IAM) Service Account and Service Account Key. After that, you import the Key's credentials as a Secret in each cluster that runs Config Connector.

Creating a service account
  1. Create the cnrm-system Service Account with gcloud:

    gcloud iam service-accounts create cnrm-system
  2. Give the IAM Service Account elevated permissions on your project. Replace [HOST_PROJECT_ID] with your project ID.

    gcloud projects add-iam-policy-binding [HOST_PROJECT_ID] \
    --member serviceAccount:cnrm-system@[HOST_PROJECT_ID].iam.gserviceaccount.com \
    --role roles/owner
  3. Create a Service Account Key and export its credentials to a file named key.json. Replace [HOST_PROJECT_ID] with your project ID and run the following:

    gcloud iam service-accounts keys create --iam-account \
    cnrm-system@[HOST_PROJECT_ID].iam.gserviceaccount.com key.json

Applying credentials to your cluster

Apply the credentials to each cluster where you will run Config Connector.

  1. Create the cnrm-system namespace.

    kubectl create namespace cnrm-system
  2. Import the key's credentials as a Secret.

    kubectl create secret generic gcp-key --from-file key.json --namespace cnrm-system
  3. Remove the credentials from your system.

    rm key.json

Installing Config Connector

To manually install Config Connector, download the installation tar file and extract it, then apply the contents to your cluster.

  1. Download the latest installation bundle tar file:

    gsutil cp gs://cnrm/latest/release-bundle.tar.gz release-bundle.tar.gz
  2. Extract the tar file:

    tar zxvf release-bundle.tar.gz
  3. Apply the manifests to your cluster:

    kubectl apply -f install-bundle-gcp-identity/

Namespaced mode

Namespaced mode is an extension of the Workload Identity installation. It allows you to manage multiple projects with multiple Google service accounts bound to your your Config Connector cluster. For more information, see Choosing an installation type.

Prerequisites

For the rest of this section, the Google Cloud project where you install Config Connector is known as the host project, or [HOST_PROJECT_ID]. The other projects where you manage resources are known as the managed projects, or [MANAGED_PROJECT_ID].

Before you install Config Connector to a cluster, perform the following steps.

Install Config Connector

  1. Download the latest installation bundle tar file:

    gsutil cp gs://cnrm/latest/release-bundle.tar.gz release-bundle.tar.gz
  2. Extract the tar file.

    tar zxvf release-bundle.tar.gz
  3. Install the common Config Connector components on your cluster.

    kubectl apply -f install-bundle-namespaced/0-cnrm-system.yaml \
     -f install-bundle-namespaced/crds.yaml

Set up your namespaces

Next, set up a namespace and a service account for each managed project.

  1. Create a service account for the Namespace in the host project. Replace [HOST_PROJECT_ID] with your host project ID and [NAMESPACE] with your Namespace name.

    gcloud iam service-accounts create cnrm-system-[NAMESPACE] --project [HOST_PROJECT_ID]
  2. Give the Cloud IAM Service Account elevated permissions on the managed project. Replace:

    • [NAMESPACE] with your Namespace name,
    • [MANAGED_PROJECT_ID] with your managed project's ID, and
    • [HOST_PROJECT_ID] with your host project ID.
    gcloud projects add-iam-policy-binding [MANAGED_PROJECT_ID] \
    --member="serviceAccount:cnrm-system-[NAMESPACE]@[HOST_PROJECT_ID].iam.gserviceaccount.com" \
    --role="roles/owner"
    
  3. Create a Cloud IAM policy binding between the IAM Service Account and the Kubernetes service account created by Config Connector. Replace [HOST_PROJECT_ID] with your host project ID and [NAMESPACE] with your Namespace name.

    gcloud iam service-accounts add-iam-policy-binding cnrm-system-[NAMESPACE]@[HOST_PROJECT_ID].iam.gserviceaccount.com \
    --member="serviceAccount:[HOST_PROJECT_ID].svc.id.goog[cnrm-system/cnrm-controller-manager-[NAMESPACE]]" \
    --role="roles/iam.workloadIdentityUser" \
    --project [HOST_PROJECT_ID]
  4. If the Namespace doesn't exist, create it with kubectl. Replace [NAMESPACE] with your Namespace name.

    kubectl create namespace [NAMESPACE]
  5. Set up the Config Connector namespace components for the Namespace. Replace [HOST_PROJECT_ID] with your host project ID and [NAMESPACE] with your Namespace name.

    sed -e 's/${PROJECT_ID?}/[HOST_PROJECT_ID]/' \
    -e 's/${NAMESPACE?}/[NAMESPACE]/' \
    install-bundle-namespaced/per-namespace-components.yaml > install-bundle-namespaced/per-namespace-components-[NAMESPACE].yaml
  6. Install the Config Connector namespace components for your Namespace. Replace [NAMESPACE] with your Namespace name.

    kubectl apply -f install-bundle-namespaced/per-namespace-components-[NAMESPACE].yaml
  7. Set the project ID annotation for the Namespace. Config Connector uses this to map resources in this Namespace to the project ID. Replace [MANAGED_PROJECT_ID] with your managed project's ID and [NAMESPACE] with your Namespace name.

    kubectl annotate namespace [NAMESPACE] cnrm.cloud.google.com/project-id=[MANAGED_PROJECT_ID]

You are now ready to create your resources with Config Connector.

Specify where to create your resources

Config Connector can organize resources by project, folder, or organization, which is the same way you would organize resources with Google Cloud.

Before creating resources with Config Connector, you must configure where to create your resources. To determine where to create the resource, Config Connector uses an annotation on either the resource configuration or the namespace. For more information, see Organizing resources.

You can create resources in a project, folder, or organization. Select a tab to choose where you want Config Connector to create resources.

Project

To create resources in a certain project, run the following command, replacing [NAMESPACE_NAME] with your Namespace name and [PROJECT_ID] with your Google Cloud project ID:

kubectl annotate namespace \
[NAMESPACE_NAME] cnrm.cloud.google.com/project-id=[PROJECT_ID]

Folder

To create resources in a certain folder, run the following command, replacing [NAMESPACE_NAME] with your Namespace name and [FOLDER_ID] with your Google Cloud folder ID:

kubectl annotate namespace \
[NAMESPACE_NAME] cnrm.cloud.google.com/folder-id=[FOLDER_ID]

Organization

To create resources in a certain organization, run the following command, replacing [NAMESPACE_NAME] with your Namespace name and [ORGANIZATION_ID] with your Google Cloud organization ID:

kubectl annotate namespace \
[NAMESPACE_NAME] cnrm.cloud.google.com/organization-id=[ORGANIZATION_ID]

When you annotate your Namespace, Config Connector creates resources in the corresponding project, folder or organization. To learn more about how Config Connector uses Kubernetes Namespaces, see Kubernetes Namespaces and Google Cloud projects.

Verify Your Installation

Config Connector runs all its components in a namespace named cnrm-system. You can verify the pods are ready by executing the following command:

kubectl wait -n cnrm-system \
 --for=condition=Ready pod --all

If Config Connector is installed correctly, the output is similar to the following:

pod/cnrm-controller-manager-0 condition met

Setting your default namespace

You can simplify commands and avoid adding `--namespace` to each `kubectl` command by changing the default context's Namespace. To do so, run the following command, replacing [NAMESPACE_NAME] with your Namespace name:
kubectl config set-context --current --namespace [NAMESPACE_NAME]

Troubleshooting

CRD instances can be created, but nothing seems to happen. There are no events.

Check the logs from the Config Connector controller:

kubectl logs cnrm-controller-manager-0 --namespace=cnrm-system

Troubleshooting Google Cloud identity installations

If the pod with name like cnrm-controller-manager-X does not have status Running, try reinstalling the Google Cloud Service Account credentials by first deleting the existing secret (kubectl --namespace=cnrm-system delete secret gcp-key) and then following the instructions to create the Secret. Then, delete the controller pod to trigger a restart:

kubectl delete pod --namespace=cnrm-system cnrm-controller-manager-0

Uninstalling Config Connector

Uninstall add-on

You use kubectl delete to remove Google Cloud Resource CRDs along with controller components.

kubectl delete ConfigConnector configconnector.core.cnrm.cloud.google.com --wait=true

Disable ConfigConnector Addon

You can disable ConfigConnector Addon in your cluster from gcloud or the Google Cloud Console:

gcloud

To disable the Config Connector add-on with gcloud, run the following command:

gcloud beta container clusters update [CLUSTER_NAME] --update-addons ConfigConnector=DISABLED

Replace [CLUSTER_NAME] with the name of the cluster that has Config Connector add-on installed.

Cloud Console

To disable the Config Connector add-on from the Google Cloud Console, perform the following steps.

  1. Go to the Google Kubernetes Engine Clusters page on the Google Cloud Console and select the cluster that you want to update.

    Visit the Google Kubernetes Engine menu

  2. Click Edit. The Edit clusters screen appears.

  3. Click Add-ons.

  4. Select Config Connector and choose Disabled.

  5. Click Save to update your cluster.

Manual uninstall

If you originally installed Config Connector without the add-on, remove all Config Connector artifacts from your cluster:

kubectl delete sts,po -n cnrm-system -l cnrm.cloud.google.com/component=cnrm-controller-manager --wait=true
kubectl delete crds -l cnrm.cloud.google.com/system=true --wait=true
kubectl delete sts,deploy,po,svc,roles,clusterroles,clusterrolebindings --all-namespaces -l cnrm.cloud.google.com/system=true --wait=true
kubectl delete validatingwebhookconfiguration abandon-on-uninstall.cnrm.cloud.google.com --ignore-not-found --wait=true
kubectl delete validatingwebhookconfiguration validating-webhook.cnrm.cloud.google.com --ignore-not-found --wait=true
kubectl delete mutatingwebhookconfiguration mutating-webhook.cnrm.cloud.google.com --ignore-not-found --wait=true
kubectl delete namespace cnrm-system --ignore-not-found --wait=true

Upgrading Config Connector

Add-on upgrade

Your Config Connector add-on is upgraded to a new minor release with the GKE Cluster master. The resources in your cluster are preserved.

Manual upgrade

Upgrading Config Connector will preserve the state of the Google Cloud resources in your cluster while upgrading the system components.

To upgrade Config Connector:

  1. Remove the system components:
    kubectl delete sts,deploy,po,svc,roles,clusterroles,clusterrolebindings --all-namespaces -l cnrm.cloud.google.com/system=true --wait=true
    kubectl delete validatingwebhookconfiguration abandon-on-uninstall.cnrm.cloud.google.com --ignore-not-found --wait=true
    kubectl delete validatingwebhookconfiguration validating-webhook.cnrm.cloud.google.com --ignore-not-found --wait=true
    kubectl delete mutatingwebhookconfiguration mutating-webhook.cnrm.cloud.google.com --ignore-not-found --wait=true
  2. Run the manual install steps

What's next

Get started with Config Connector.