This topic provides you with two additional installation options for installing Config Connector on your Google Kubernetes Engine (GKE) cluster:
- Manual installation: Installing with the manual method provides you with faster updates than using the add-on. This method also provides you with more configuration options. For example, you can increase the CPU limit of the Config Connector Operator.
- Namespaced mode: This method is an extension of the Config Connector installation. Namespaced mode supports managing multiple projects, each with their own Google Cloud identities.
For more information on these installation types, see Choosing an installation type.
Manually installing the Config Connector Operator
The following sections show you how to manually install the Config Connector Operator.
Before you begin
Before you manually install Config Connector Operator, complete the following steps:
- Create or identify a GKE cluster where Config Connector has not yet been installed and that has Workload Identity and Kubernetes Engine Monitoring enabled.
- Configure
kubectl
to connect to your cluster.
Installing the Config Connector Operator
Config Connector uses a Kubernetes Operator to keep its installation up to date. To install this Operator, complete the following steps:
Download the latest Config Connector Operator tar file:
gsutil cp gs://configconnector-operator/latest/release-bundle.tar.gz release-bundle.tar.gz
Extract the tar file:
tar zxvf release-bundle.tar.gz
Install the Config Connector operator on your cluster:
kubectl apply -f operator-system/configconnector-operator.yaml
Creating an identity
Config Connector creates and manages Google Cloud resources by authenticating with a Identity and Access Management (IAM) service account and using GKE's Workload Identity to bind IAM service accounts with Kubernetes service accounts.
To create the identity, complete the following steps:
-
Create an IAM service account. If you want to use an existing service account, you
can use that account and skip this step.
To create the service account, use the following command:gcloud iam service-accounts create SERVICE_ACCOUNT_NAME
ReplaceSERVICE_ACCOUNT_NAME
with a name for your service account. -
Give the IAM service account elevated permissions on your project:
gcloud projects add-iam-policy-binding PROJECT_ID \ --member="serviceAccount:SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/owner"
Replace the following:PROJECT_ID
with your project ID.SERVICE_ACCOUNT_NAME
with your service account's name.
-
Create an IAM policy binding between the IAM
service account and the predefined Kubernetes service account that
Config Connector runs:
gcloud iam service-accounts add-iam-policy-binding \ SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com \ --member="serviceAccount:PROJECT_ID.svc.id.goog[cnrm-system/cnrm-controller-manager]" \ --role="roles/iam.workloadIdentityUser"
Replace the following:SERVICE_ACCOUNT_NAME
with your service account's name.PROJECT_ID
with your project ID.
To learn more about creating service accounts, see Creating and managing service accounts.
Configuring Config Connector
To complete the installation, you create a configuration file for
the ConfigConnector
CustomResource,
then apply it using the kubectl apply
command. The Config Connector Operator installs
Google Cloud Resource CRDs and Config Connector components in your cluster.
To configure the operator, complete the following steps:
-
Copy the following YAML file into a file named
configconnector.yaml
:# configconnector.yaml apiVersion: core.cnrm.cloud.google.com/v1beta1 kind: ConfigConnector metadata: # the name is restricted to ensure that there is only one # ConfigConnector resource installed in your cluster name: configconnector.core.cnrm.cloud.google.com spec: mode: cluster googleServiceAccount: "SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com"
Replace the following:SERVICE_ACCOUNT_NAME
with your service account's name.PROJECT_ID
with your project ID.
-
Apply the configuration to your cluster with
kubectl apply
:kubectl apply -f configconnector.yaml
Specifying where to create your resources
Config Connector can organize resources by project, folder, or organization, which is the same way you would organize resources with Google Cloud.
Before creating resources with Config Connector, you must configure where to create your resources. To determine where to create the resource, Config Connector uses an annotation on either the resource configuration or an existing Namespace. For more information, see Organizing resources.
If you do not have a Namespace for this purpose, create one withkubectl
.
kubectl create namespace NAMESPACE
Replace NAMESPACE
with your namespace name. For example
config-connector
.
Select a tab to choose where you want Config Connector to create resources.
Project
To create resources in a certain project, run the following command:
kubectl annotate namespace \ NAMESPACE cnrm.cloud.google.com/project-id=PROJECT_ID
Replace the following:
NAMESPACE
with your namespace name.PROJECT_ID
with your Google Cloud project ID.
Folder
To create resources in a certain folder, run the following command:
kubectl annotate namespace \ NAMESPACE cnrm.cloud.google.com/folder-id=FOLDER_ID
Replace the following:
NAMESPACE
with your namespace name.FOLDER_ID
with your Google Cloud folder ID.
Organization
To create resources in a certain organization, run the following command:
kubectl annotate namespace \ NAMESPACE cnrm.cloud.google.com/organization-id=ORGANIZATION_ID
Replace the following:
NAMESPACE
with your namespace name.ORGANIZATION_ID
with your Google Cloud organization ID.
When you annotate your namespace, Config Connector creates resources in the corresponding project, folder or organization. To learn more about how Config Connector uses Kubernetes namespaces, see Kubernetes Namespaces and Google Cloud projects.
Verifying your installation
Config Connector runs all of its components in a namespace named cnrm-system
.
You can verify the Pods are ready by running the following command:
kubectl wait -n cnrm-system \
--for=condition=Ready pod --all
If Config Connector is installed correctly, the output is similar to the following:
pod/cnrm-controller-manager-0 condition met
Upgrading Config Connector
To download and install the latest version of Config Connector operator:
gsutil cp gs://configconnector-operator/latest/release-bundle.tar.gz release-bundle.tar.gz
tar zxvf release-bundle.tar.gz
kubectl apply -f operator-system/configconnector-operator.yaml
Uninstalling Config Connector
Use kubectl delete
to remove the Config Connector CRDs along with
controller components:
kubectl delete ConfigConnector configconnector.core.cnrm.cloud.google.com \
--wait=true
To uninstall the Config Connector operator, run the following command:
kubectl delete -f operator-system/configconnector-operator.yaml --wait=true
Installing Config Connector using namespaced mode
The following sections show you how to enable namespaced mode.
Before you begin
Before configuring Config Connector to run in namespaced mode, ensure that you have enabled the Config Connector GKE add-on or manually installed Config Connector Operator.
Configure Config Connector to run in namespaced mode
To enable namespaced mode, complete the following steps::
Copy the following YAML manifest into a file named
configconnector.yaml
:apiVersion: core.cnrm.cloud.google.com/v1beta1 kind: ConfigConnector metadata: # the name is restricted to ensure that there is only ConfigConnector resource installed in your cluster name: configconnector.core.cnrm.cloud.google.com spec: mode: namespaced
Apply the configuration to your cluster with
kubectl apply
:kubectl apply -f configconnector.yaml
Configure Config Connector to watch your namespaces
In the following sections, the Google Cloud project where you install Config Connector is known as the host project, or HOST_PROJECT_ID. The other projects where you manage resources are known as the managed projects, or MANAGED_PROJECT_ID. These could be the same project if you only intend to use Config Connector to create Google Cloud resources in the same project as your cluster.
Creating a namespace
You can skip this step if you already have a namespace to use to organize Google Cloud resources.
Use kubectl
to create a new namespace by running the following command:
kubectl create namespace NAMESPACE
Replace NAMESPACE with a name for the namespace.
Creating an Identity
Create an Identity and Access Management (IAM) service account and create a binding between the IAM Service Account and Config Connector's Kubernetes service account:
Create an IAM service account. If you have an existing service account, you can use it instead of creating a new service account. Use
gcloud
to create the service account by running the following command:gcloud iam service-accounts create NAMESPACE_GSA --project HOST_PROJECT_ID
Replace the following:
- NAMESPACE_GSA with the name of the Google service account (GSA) bound to your namespace.
- HOST_PROJECT_ID with your host project's ID.
To learn more about creating service accounts, see Creating and managing service accounts.
Give the IAM service account elevated permissions on your managed project.
gcloud projects add-iam-policy-binding MANAGED_PROJECT_ID \ --member="serviceAccount:NAMESPACE_GSA@HOST_PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/owner"
Replace the following:
- MANAGED_PROJECT_ID with your managed project's ID.
- NAMESPACE_GSA with the name of the Google service account bound to your namespace.
- HOST_PROJECT_ID with your host project's ID.
Create an IAM policy binding between the IAM service account and the Config Connector Kubernetes service account. You bind the service accounts by running the following
gcloud
command:gcloud iam service-accounts add-iam-policy-binding \ NAMESPACE_GSA@HOST_PROJECT_ID.iam.gserviceaccount.com \ --member="serviceAccount:HOST_PROJECT_ID.svc.id.goog[cnrm-system/cnrm-controller-manager-NAMESPACE]" \ --role="roles/iam.workloadIdentityUser" \ --project HOST_PROJECT_ID
Replace the following:
- HOST_PROJECT_ID with your host project's ID.
- NAMESPACE_GSA with the name of the Google service account bound to your namespace.
- NAMESPACE with your namespace.
Give the IAM service account permissions to publish Prometheus metrics to Google Cloud's operations suite on your host project.
gcloud projects add-iam-policy-binding HOST_PROJECT_ID \ --member="serviceAccount:NAMESPACE_GSA@HOST_PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/monitoring.metricWriter"
Replace the following:
- HOST_PROJECT_ID with your host project's ID.
- NAMESPACE_GSA with the name of the Google service account bound to your namespace.
Creating a ConfigConnectorContext
To create Google Cloud resources, you need to configure Config Connector
to watch your namespace by adding a ConfigConnectorContext
object
in the namespace you want to use.
To create a ConfigConnectorContext
, complete the following steps:
Copy the following YAML manifest into a file named
configconnectorcontext.yaml
:apiVersion: core.cnrm.cloud.google.com/v1beta1 kind: ConfigConnectorContext metadata: # you can only have one ConfigConnectorContext per namespace name: configconnectorcontext.core.cnrm.cloud.google.com namespace: NAMESPACE spec: googleServiceAccount: "NAMESPACE_GSA@HOST_PROJECT_ID.iam.gserviceaccount.com"
Replace the following:
- NAMESPACE with the name of your namespace.
- NAMESPACE_GSA with the name of the Google service account bound to your namespace.
- HOST_PROJECT_ID with your host project's ID.
Apply the file to your cluster with
kubectl
:kubectl apply -f configconnectorcontext.yaml
Verify that the Config Connector Operator created a Kubernetes service account for your namespace with
kubectl
by running the following command:kubectl get serviceaccount/cnrm-controller-manager-NAMESPACE -n cnrm-system
Replace NAMESPACE with the name of your namespace.
Verify that the Config Connector controller Pod is running for your namespace with
kubectl
by running the following command:kubectl wait -n cnrm-system \ --for=condition=Ready pod \ -l cnrm.cloud.google.com/component=cnrm-controller-manager \ -l cnrm.cloud.google.com/scoped-namespace=NAMESPACE
Replace NAMESPACE with the name of your namespace.
If the Config Connector controller is running, the output is similar to:
cnrm-controller-manager-abcdefghijk-0 condition met.
Upgrading from non-operator installations
Config Connector version 1.33.0 and higher only supports installation with the GKE add-on, or the operator.
To upgrade to the operator (and retain all Config Connector resources), you must remove all Config Connector system components except the CRDs, and then install the operator.
Run the following commands to remove Config Connector system non-CRD components:
kubectl delete sts,deploy,po,svc,roles,clusterroles,clusterrolebindings --all-namespaces -l cnrm.cloud.google.com/system=true --wait=true kubectl delete validatingwebhookconfiguration abandon-on-uninstall.cnrm.cloud.google.com --ignore-not-found --wait=true kubectl delete validatingwebhookconfiguration validating-webhook.cnrm.cloud.google.com --ignore-not-found --wait=true kubectl delete mutatingwebhookconfiguration mutating-webhook.cnrm.cloud.google.com --ignore-not-found --wait=true
Install Config Connector with the GKE add-on, or the operator.
Switching from the add-on to a manual installation
When installed as an add-on, the version of Config Connector is directly tied to the GKE version that is installed.
The manual installation allows for faster updates at the cost of manual upgrades.
To switch while safely keeping all resources:
Disable the addon without deleting any
ConfigConnector
orConfigConnectorContext
objects:gcloud container clusters update CLUSTER_NAME --update-addons ConfigConnector=DISABLED
Replace
CLUSTER_NAME
with the name of the cluster that you installed Config Connector on.Follow the instructions to install the manual operator of the desired version.
Troubleshooting
The following section provides you with troubleshooting tips for your Config Connector installation.
Troubleshooting permissions for resource reconciliations
If Google Cloud can't reconcile resources successfully, the logs contain the following error
message: The caller does not have permission, forbidden.
To investigate this error, complete the following steps:
- Open a shell with the cloud-sdk image:
kubectl run -it \ --image google/cloud-sdk:slim \ --serviceaccount cnrm-controller-manager \ --namespace cnrm-system \ workload-identity-test
- List your identity:
gcloud auth list
-
Verify that the identity listed matches the Google service account bound to your resources.
If you see the Compute Engine default service account instead, then that means that Workload Identity is not enabled on your GKE cluster and/or node pool.
- Verify that the Google service account has the needed roles to manage the Google Cloud resources.
What's next
- Get started with Config Connector.
- Learn how to install Config Connector as a GKE add-on.
- Learn how to install Config Connector on other Kubernetes distributions.