Installing Config Sync

Config Sync lets cluster operators manage Kubernetes resources by using files, called configs, stored in Git repositories and this page shows you how to enable and configure Config Sync.

Config Sync is available for Anthos and Google Kubernetes Engine (GKE) users.

Before you begin

Before you install Config Sync, make sure that you have prepared your environment, clusters, and roles and enabled Anthos Config Management.

Preparing your local environment

Prepare your local environment by completing the following tasks:

  1. Create, or have access to, a Git repository that can contain the configs that Config Sync syncs to.
  2. Install and initialize the Cloud SDK, which provides the gcloud, and nomos commands. If you use Cloud Shell, the Cloud SDK comes pre-installed.

Cluster requirements

Create, or have access to, a cluster that meets the following requirements:

  • Is on an Anthos supported platform and version.
  • Uses a Kubernetes version 1.14.x or later.
  • Is not an Autopilot cluster.
  • Has a node pool that uses a machine type with at least 4 vCPU.
  • Has Workload Identity enabled. Workload Identity is the recommended way to access Google Cloud services from applications running within Google Kubernetes Engine (GKE) due to its improved security properties and manageability.
  • If you want to use a private GKE cluster, the Cloud NAT is created to permit egress from private GKE nodes. Alternatively, you can enable Private Google Access to connect to the set of external IP addresses used by Google APIs and services.

  • If you want to use Google service account when granting Config Sync access to your Git repository, then access scopes for the nodes in the cluster must include read only scope for Cloud Source Repositories.

    You can add this scope by including cloud-source-repos-ro in the -- scopes list specified at cluster creation time, or by using the cloud- platform scope at cluster creation time. For example:

    gcloud container clusters create CLUSTER_NAME --scopes=cloud-platform
    

Preparing your cluster

After you have created a suitable cluster, complete the following steps:

  1. If you are using Anthos Config Management for the first time, enable Anthos Config Management.

  2. Register your cluster to a fleet. Your project's fleet provides a unified way to view and manage your clusters and their workloads as part of Anthos, including clusters outside Google Cloud. Anthos charges apply only to your registered clusters.

Preparing permissions

Once your cluster is created, grant yourself the GKE Hub Admin role that you need to Config Sync in the Google Cloud Console. The Hub Admin role provides full access to a fleet.

To grant the Hub Admin Role, complete the following steps:

Console

  1. In the Cloud Console, go to the IAM page.

    Go to the IAM page

  2. Click Add.

  3. In the New members field, enter an email address. You can add individuals, service accounts, or Google Groups as members.

  4. In the Select a role drop-down list, search for and select GKE Hub Admin.

  5. Click Save.

gcloud

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member=MEMBER \
  --role=roles/gkehub.admin

Replace the following:

  • MEMBER: An identifier for the member, which usually has the following form: member-type:id For example, user:my-user@example.com. For a full list of the values that member can have, see the Policy Binding reference.

  • PROJECT_ID: your project ID

Installing Config Sync

In the following sections, you grant Config Sync access to your repository. After you have granted access, you configure the installation.

Granting Config Sync read-only access to Git

Config Sync needs read-only access to your Git repository so that it can read the configs committed to the repository and apply them to your clusters.

If your repository does not require authentication for read-only access, you can continue to configure Config Sync and use none as your authentication type. For example, if you can browse the repository using a web interface without logging in, or if you can use git clone to create a clone of the repository locally without providing credentials or using saved credentials, then you do not need to authenticate. In this case, you do not need to create a Secret.

However, most users need to create credentials because read access to their repository is restricted. If credentials are required, they are stored in the git-creds Secret on each enrolled cluster (unless you are using a Google service account).

Config Sync supports the following mechanisms for authentication:

  • SSH key pair
  • cookiefile
  • Token
  • Google Service Account (Google Source Repositories only)

The mechanism that you choose depends on what your repository supports. If all mechanisms are available, we recommend using an SSH key pair. GitHub, Cloud Source Repositories, and Bitbucket all support using an SSH key pair. If your organization hosts your repository and you don't know which authentication methods are supported, contact your administrator.

SSH key pair

An SSH key pair consists of two files, a public key and a private key. The public key typically has a .pub extension.

To use an SSH key pair, complete the following steps:

  1. Create an SSH key pair to allow Config Sync to authenticate to your Git repository. This step is necessary if you need to authenticate to the repository to clone it or read from it. Skip this step if a security administrator provides you with a key pair. You can use a single key pair for all clusters, or a key pair per cluster, depending on your security and compliance requirements.

    The following command creates a 4096-bit RSA key. Lower values are not recommended:

    ssh-keygen -t rsa -b 4096 \
    -C "GIT_REPOSITORY_USERNAME" \
    -N '' \
    -f /path/to/KEYPAIR_FILENAME
    

    Replace the following:

    • GIT_REPOSITORY_USERNAME: the username that you want Config Sync to use to authenticate to the repository
    • /path/to/KEYPAIR_FILENAME: a path to the key pair

    If you are using a third-party Git repository host such as GitHub, or you want to use a service account with Cloud Source Repositories, we recommend that you use a separate account.

  2. Configure your repository to recognize the newly created public key. Refer to the documentation for your Git hosting provider. Instructions for some popular Git hosting providers are included for convenience:

  3. Add the private key to a new Secret in the cluster:

    kubectl create ns config-management-system && \
    kubectl create secret generic git-creds \
     --namespace=config-management-system \
     --from-file=ssh=/path/to/KEYPAIR_PRIVATE_KEY_FILENAME
    

    Replace /path/to/KEYPAIR_PRIVATE_KEY_FILENAME with the name of the private key (the one without the .pub suffix).

  4. Delete the private key from the local disk or otherwise protect it.

cookiefile

The process for acquiring a cookiefile depends on the configuration of your repository. For an example, see Generate static credentials in the Cloud Source Repositories documentation. The credentials are usually stored in the .gitcookies file in your home directory, or they might be provided to you by a security administrator.

To use a cookiefile, complete the following steps:

  1. After you create and obtain the cookiefile, add it to a new Secret in the cluster.

    kubectl create ns config-management-system && \
    kubectl create secret generic git-creds \
     --namespace=config-management-system \
     --from-file=cookie_file=/path/to/COOKIEFILE
    

    Replace /path/to/COOKIEFILE with the appropriate path and filename.

  2. Protect the contents of the cookiefile if you still need it locally. Otherwise, delete it.

Token

If your organization does not permit the use of SSH keys, you might prefer to use a token. With Config Sync, you can use GitHub's personal access tokens (PATs), GiLab's PATs or deploy keys, or Bitbucket's app password as your token.

To create a Secret using your token, complete the following steps:

  1. Create a token using GitHub, GitLab, or Bitbucket:

  2. After you create and obtain the token, add it to a new Secret in the cluster:

    kubectl create ns config-management-system && \
    kubectl create secret generic git-creds \
      --namespace="config-management-system" \
      --from-literal=username=USERNAME \
      --from-literal=token=TOKEN
    

    Replace the following:

    • USERNAME: the username that you want to use.
    • TOKEN: the token that you created in the previous step.
  3. Protect the token if you still need it locally. Otherwise, delete it.

Google service account

If your repository is in Cloud Source Repositories, you can give Config Sync access to a repository in the same project as your managed cluster by using a Google service account.

To use a repository in Cloud Source Repositories as your Config Sync repository, complete the following steps:

  1. List all repositories:

    gcloud source repos list
    
  2. From the output, copy the URL from the repository that you want to use. For example:

    REPO_NAME  PROJECT_ID  URL
    my-repo    my-project  https://source.developers.google.com/p/my-project/r/my-repo-csr
    
  3. When you configure Config Sync in the following section, add the URL to your configuration. If you are configuring Config Sync using the Google Cloud Console, add the value that you copied in the URL field. If you are configuring Config Sync using the gcloud command-line tool, add the value that you copied to the repo field of your ConfigManagement object.

  4. When you configure Config Sync, select the appropriate authentication type.

    If you are configuring Config Sync using the Google Cloud Console, select Google Cloud Repository as your Authentication type when you configure Config Sync.

    If you are configuring Config Sync using the gcloud command-line tool, select gcpServiceAccount or gcenode as your secretType:

    • gcpServiceAccount: If Workload Identity is enabled on your cluster add gcpServiceAccount as your authentication type.

      If necessary, create a service account that you can use when you configure Config Sync. Ensure that the service account has read access to Cloud Source Repositories by granting it the source.reader role.

      When you select gcpServiceAccount as your method of authentication, you also need to add a gcpServiceAccountEmail when you configure Config Sync. In addition to adding the gcpServiceAccountEmail, you need to create a role binding after configuring Config Sync. For information on these steps, see the Using Cloud Source Repositories with Workload Identity section.

    • gcenode: If Workload Identity is not enabled on your cluster, add gcenode to use the Compute Engine default service account. By default, the Compute Engine default service account, PROJECT_ID-compute@developer.gserviceaccount.com, has source.reader access to the repository for the same project. However, if your Cloud Source Repositories is located in a project different from your cluster's project, you have to grant default Compute Engine service account from the cluster's project, the source.reader in the Cloud Source Repositories's project.

      If needed, you can add the source.reader` role with the following command:

      gcloud projects add-iam-policy-binding PROJECT_ID \
        --member serviceAccount:PROJECT_NUMBER-compute@developer.gserviceaccount.com \
        --role roles/source.reader
      

      Replace the following:

      • PROJECT_ID: your organization's project ID
      • PROJECT_NUMBER: your organization's project number

      You cannot modify access scopes after you create a node pool. However, you can create a new node pool with the proper access scope while using the same cluster. The default gke-default scope does not include cloud-source-repos-ro.

Configuring Config Sync

The Google Cloud Console guides you through the installation process and automates many of the steps. Console commands are slightly different for Anthos and GKE customers. Click on the relevant tab to view the appropriate instructions.You can also use the gcloud command-line tool to complete the installation.

Console - Anthos

Before continuing, make sure you've registered your clusters to a fleet.

If you use Anthos, complete the following steps to configure Config Sync:

  1. In the Cloud Console, go to the Anthos Config Management page.

    Go to Anthos Config Management

  2. Select your registered clusters and click Configure.

  3. In the Git Repository Authentication for ACM section, select one of the following options:

    • None: Use no authentication
    • SSH: Use a SSH key pair
    • Cookiefile: Use a cookiefile
    • Token: Use a token
    • Google Cloud Repository: Use a Google service account to access a Cloud Source Repositories. Only select this option if Workload Identity is not enabled in your cluster.
  4. If you have not done so already, follow the instructions in the Google Cloud Console to grant Config Sync read-only access to Git.

  5. Click Continue.

  6. In the ACM settings for your clusters section, complete the following:

    1. In the Version field, select version 1.7.0 or later for Anthos Config Management, which enables syncing from multiple repositories by default.
    2. Select the Enable Config Sync checkbox.
      1. In the URL field, add the URL of the Git repository to use as the source of truth. You can enter URLs using either the HTTPS or SSH protocol. For example, https://github.com/GoogleCloudPlatform/anthos-config-management-samples uses the HTTPS protocol. If you don't enter a protocol, the URL is treated as an HTTPS URL. This field is required.
      2. Optional: In the Branch field, add the branch of the repository to sync from. The default is the main (master) branch.
      3. Optional: In the Tag/Commit field, add the Git revision (tag or hash) to check out. The default is HEAD.
      4. Optional: In the Policy directory field, add the path within the repository to the top of the policy hierarchy to sync. The default is the root directory of the repository.
      5. Optional: In the Sync wait field, add the period in seconds between consecutive syncs. The default is 15 seconds.
      6. Optional: In the Git proxy field, enter the URL for the HTTPS proxy to be used when communicating with the Git repository. This is an optional field, and if it's left blank, no proxy is used.
      7. Optional: In the Source format field, choose either hierarchy (default) or unstructured (recommended). We recommend that you select unstructured as this format lets you organize your configs in the way that is most convenient to you.
  7. Click Done. You are taken back to the Anthos Config Management page. After a few minutes, you should see Synced in the status column next to the clusters that you configured. If you see Error, click on the word Error for more information.

Console - GKE

If you use GKE, complete the following steps to configure Config Sync:

Registering your clusters

To use Config Management with GKE, you must first register the clusters. Registering your clusters lets them share a common set of configurations and policies.

To register your clusters, complete the following tasks:

  1. In the Cloud Console, go to the Config Management page.

    Go to Config Management

    This page shows you which clusters are currently registered and configured and lets you begin registering new clusters.

  2. To begin the registration process, click Set up Config Management.

  3. To enable the Config Management API, click Next.

  4. In the Select registered clusters for Config Management page, locate the Unregistered clusters from this project table, and find the cluster that you want to register.

  5. Click Register next to the cluster that you want to register.

    Once the cluster is successfully registered, it appears in the Select registered clusters for Config Management table.

Installing Config Sync

Once you have registered your clusters, you can continue on to install and configure Config Sync.

To install Config Sync, complete the following steps:

  1. To install Config Sync, select the clusters that you want to configure, and select Next. Selecting multiple clusters lets you apply the same configurations to whichever clusters you configure.
  2. In the Config Sync page that appears, select the Anthos Config Management Version that you want to use. The default is the current version.
  3. In the Configurations section, leave the Enable Config Sync checkbox selected.
  4. In the URL field, add the URL of the Git repository to use as the source of truth. You can enter URLs using either the HTTPS or SSH protocol. For example, https://github.com/GoogleCloudPlatform/anthos-config-management-samples uses the HTTPS protocol. If you don't enter a protocol, the URL is treated as an HTTPS URL.
  5. In the Authentication type drop-down list, select the one of the following options:

    • None: Use no authentication
    • SSH: Use a SSH key pair
    • Cookiefile: Use a cookiefile
    • Token: Use a token
    • Google Cloud Repository: Use a Google service account to access a Cloud Source Repositories. Only select this option if Workload Identity is not enabled in your cluster.
  6. Follow the instructions in the Google Cloud Console to grant Config Sync read-only access to Git and click Continue.

  7. Optional: In the Branch field, add the branch of the repository to sync from. The default is the master branch.

  8. Optional: In the Tag/Commit field, add the Git revision (tag or hash) to check out. The default is HEAD.

  9. Optional: In the Policy directory field, add the path within the repository to the top of the policy hierarchy to sync. The default is the root directory of the repository.

  10. Optional: In the Sync wait field, add the period in seconds between consecutive syncs. The default is 15 seconds.

  11. Optional: In the Git proxy field, enter the URL for the HTTPS proxy to be used when communicating with the Git repository. This is an optional field, and if it's left blank, no proxy is used.

  12. Optional: In the Source format field, choose either hierarchy (default) or unstructured (recommended). We recommend that you select unstructured as this format lets you organize your configs in the way that is most convenient to you.

  13. If you don't want to install Policy Controller, click Next and clear the **Enable Policy Controller checkbox.

  14. Click Complete to complete the Config Sync installation.

gcloud

Before continuing, make sure you've registered your clusters to a fleet.

If you use Anthos or GKE, complete the following steps to configure Config Sync with the gcloud command-line tool.

  1. Prepare the configuration by either creating a new apply-spec.yaml manifest or by using an existing manifest. Using an existing manifest lets you configure your cluster with the same settings used by another cluster.

Create new manifest

To configure Config Sync with new settings for your cluster, create a file named apply-spec.yaml and copy the following YAML file into it:

# apply-spec.yaml

applySpecVersion: 1
spec:
  configSync:
    # Set to true to install and enable Config Sync
    enabled: true
    sourceFormat: FORMAT
    syncRepo: REPO
    syncBranch: BRANCH
    secretType: TYPE
    gcpServiceAccountEmail: EMAIL
    policyDir: DIRECTORY

Replace the following:

  • FORMAT: add unstructured to use an unstructured repository or add hierarchy to use a hierarchical repository. These values are case-sensitive. This field is optional and the default value is hierarchy. We recommend that you add unstructured as this format lets you organize your configs in the way that is most convenient to you.
  • REPO: add the URL of the Git repository to use as the source of truth. You can enter URLs using either the HTTPS or SSH protocol. For example, https://github.com/GoogleCloudPlatform/anthos-config-management-samples uses the HTTPS protocol. If you don't enter a protocol, the URL is treated as an HTTPS URL. This field is required.
  • BRANCH: the branch of the repository to sync from. The default is the main (master) branch.
  • TYPE: one of the following secretTypes:

    • none: Use no authentication
    • ssh: Use a SSH key pair
    • cookiefile: Use a cookiefile
    • token: Use a token
    • gcpserviceaccount: Use a Google service account to access a Cloud Source Repositories.
    • gcenode: Use a Google service account to access a Cloud Source Repositories. Only select this option if Workload Identity is not enabled in your cluster.

    For more information on these authentication types, see Granting Config Sync read-only access to Git.

  • EMAIL: If you added gcpserviceaccount as your secretType, add your Google service account email address. For example, acm@PROJECT_ID.iam.gserviceaccount.com.

  • DIRECTORY: the path in the Git repository to the root directory that contains the configuration that you want to sync to. The default is the root directory of the repository.

For a complete list of fields that you can add to the spec field, see gcloud fields.

Use existing manifest

To configure your cluster with the same settings used by another cluster, fetch the settings from a registered cluster:

 gcloud alpha container hub config-management fetch-for-apply \
     --membership=MEMBERSHIP_NAME \
     --project=PROJECT_ID \
     > CONFIG_YAML_PATH

Replace the following:

  • MEMBERSHIP_NAME: the membership name of the registered cluster that has the Config Sync settings you want to use
  • PROJECT_ID: your project ID
  • CONFIG_YAML_PATH: the path to the apply-spec.yaml file

2. Apply the apply-spec.yaml file:

  gcloud beta container hub config-management apply \
      --membership=MEMBERSHIP_NAME \
      --config=CONFIG_YAML_PATH \
      --project=PROJECT_ID

Replace the following:

  • MEMBERSHIP_NAME: the membership name that you chose when you registered your cluster, you can find the name with gcloud container hub memberships list.
  • CONFIG_YAML_PATH: the path to your apply-spec.yaml file
  • PROJECT_ID: your project ID

Using Cloud Source Repositories with Workload Identity

If Workload Identity is enabled on your cluster and you used a Google service account as your authentication type, you need to complete additional steps.

After completing the preceding steps, create an IAM policy binding between the Kubernetes service account and the Google service account. The Kubernetes service account is not created until you configure Config Sync for the first time.

This binding lets the Config Sync Kubernetes service account act as the Google service account:

gcloud iam service-accounts add-iam-policy-binding \
   --role roles/iam.workloadIdentityUser \
   --member "serviceAccount:PROJECT_ID.svc.id.goog[config-management-system/root-reconciler]" \
   GSA_NAME@PROJECT_ID.iam.gserviceaccount.com

Replace the following:

  • PROJECT_ID: your organization's project ID
  • GSA_NAME: the custom Google service account that you want to use to connect to Cloud Source Repositories. Make sure that the Google service account that you select has the source.reader role.

Verifying the installation

After you have installed and configured Config Sync, you can verify that the installation completed successfully.

Console - Anthos

Complete the following steps:

  1. In the Cloud Console, go to the Anthos Config Management page.

    Go to Anthos Config Management

  2. View the Status column. A successful installation has a status of Synced.

For a detailed view of your cluster status:

  • Select the cluster that you want to explore. The Cluster details page appears. On this page, you can see details for your cluster and details for your Config Sync installation.

Console - GKE

Complete the following steps:

  1. In the Cloud Console, go to the Config Management page.

    Go to Config Management

  2. In the cluster table, view the Config sync status column. A successful installation has a status of Synced.

gcloud

Run the following command:

gcloud beta container hub config-management status \
    --project=PROJECT_ID

Replace PROJECT_ID with your project's ID.

A successful installation has a status of SYNCED. If you see an error after running the preceding command, make sure that you created the git-creds Secret. If you have created the Secret, try rerunning the following command:

gcloud beta container hub config-management apply

You can also use the nomos status command to check if the Config Sync is installed successfully. A valid installation with no problems has a status of PENDING or SYNCED. An invalid or incomplete installation has a status of NOT INSTALLED OR NOT CONFIGURED. The output also includes any reported errors.

It's possible to have an invalid configuration that isn't detected right away, such as a missing or invalid git-creds Secret. For troubleshooting steps, see Valid but incorrect ConfigManagement object in the Troubleshooting page.

Resource requests

Summary of total resource requests

The following table lists the combined amount of resource requests for Config Sync, depending on what features you are using.

Feature CPU Memory
Default (multi-repo) mode 710 m + 260 m * (number of RootSync and RepoSync objects) 620 Mi + 210 Mi * (number of RootSync and RepoSync objects)
Hierarchy Controller 220 m 220 Mi
Single repository mode 460 m 310 Mi

Detailed resource requests

The following table lists Kubernetes resource requirements for Config Sync components. For more information, see Managing Resources for Containers in the Kubernetes documentation.

Deployment name CPU request (m) per replica Memory request (Mi) per replica Single or multiple repositories
git-importer 450 300 Single
monitor Default1 Default Single
resource-group-controller-manager 100+default 50+default Multi
admission-webhook2 100 20 Multi
otel-collector 200 400 Multi
reconciler-manager 200 120 Multi
reconciler (one per RootSync and RepoSync) 250+default 200+default Multi
hnc-controller-manager 100+default 150+default Hierarchy Controller
gke-hc-controller-manager 100+default 50+default Hierarchy Controller

1The default resource request uses a CPU request of 10 and a memory request of 10 Mi.

2The admission webhook has two replicas, so when calculating the total resource requests, you need to double the value.

What's next