Install Config Sync
With Config Sync, you can manage Kubernetes resources by using files, called configs, that are stored in a source of truth. Config Sync supports Git repositories, OCI images, and Helm charts as sources of truth. This page shows you how to enable and configure Config Sync so that it syncs from your root repository. Config Sync is available if you use Anthos or Google Kubernetes Engine (GKE).
When you install Config Sync using the Google Cloud console or the
Google Cloud CLI, the RootSync
and RepoSync
APIs are enabled by default. This
provides you with additional features such as syncing from multiple repositories
and syncing Kustomize and Helm configurations.
Before you begin
Before you install Config Sync, prepare your environment, clusters, and roles, and enable Anthos Config Management.
Prepare your local environment
Prepare your local environment by completing the following tasks:
- Create, or make sure you have access to, a source of truth. This is where you add the configs that Config Sync syncs to. To learn more about how to set up your configs and source of truth, see one of the following guides:
- Install and initialize the Google Cloud CLI, which provides the
gcloud
, andnomos
commands. If you use Cloud Shell, the Google Cloud CLI comes pre-installed.
Cluster requirements
Create, or make sure you have access to, a cluster that meets the following requirements:
- Is on an Anthos supported platform and version or is a Google Kubernetes Engine (GKE) cluster.
If you use Autopilot clusters on Anthos Config Management, Autopilot adjusts the container resource requirements to meet the following rules:
- Resource limits are set equal to resource requests.
- Pod vCPU are available in increments of 0.25 vCPU (rounded up).
- The minimum value is 250 milliCPU (mCPU).
- The ratio of memory (in GiB) to vCPU must be in the range 1 to 6.5 vCPU.
Due to these rules, for Autopilot clusters, Config Sync:
- ignores resource limit overrides.
- only applies overrides when there exists one or more resource requests higher than the corresponding adjusted output declared in the annotation, or there exists one or more resource requests lower than the corresponding input declared in the annotation.
(Optional) has Workload Identity enabled if you use GKE clusters. Workload Identity is the recommended way to access Google Cloud services from applications running within GKE due to its improved security properties and manageability. If you enabled Workload Identity and want to enable Cloud Monitoring for Config Sync, you must grant the correct metric writing permissions.
If you want to use a private GKE cluster, configure the Cloud NAT to permit egress from private GKE nodes. For details, see Example GKE setup. Alternatively, you can enable Private Google Access to connect to the set of external IP addresses used by Google APIs and services.
If you want to use a Google service account when you grant Config Sync access to your Git repository, then you must include the read-only scope in access scopes for the nodes in the cluster for Cloud Source Repositories.
You can add the read-only scope by including
cloud-source-repos-ro
in the--scopes
list specified at cluster creation time, or by using thecloud-platform
scope at cluster creation time. For example:gcloud container clusters create CLUSTER_NAME --scopes=cloud-platform
If you have strict VPC Firewall requirements that block any unnecessary traffic, you need to Create firewall rules to permit the following traffic on public GKE clusters:
TCP: Allow ingress and egress on port 53 and 443
UDP: Allow egress on port 53
If you don't include these rules, Config Sync doesn't sync correctly, with
nomos status
reporting the following error:Error: KNV2004: unable to sync repo Error in the git-sync container
You can skip these steps if using a private GKE cluster.
Prepare your cluster
After you have created a suitable cluster, complete the following steps:
If you are using Anthos Config Management for the first time, enable Anthos Config Management.
Grant the required IAM roles to the user registering the cluster.
If you plan to use the Google Cloud CLI to configure Config Sync or use clusters outside Google Cloud, ensure that your GKE clusters or clusters outside of Google Cloud are registered to a fleet now. If you plan to use the Google Cloud console, you can register GKE clusters when you configure Config Sync.
Install Config Sync
In the following sections, you grant Config Sync access to one of the following sources of truth:
After you have granted access, you can configure Config Sync.
Grant access to Git
Config Sync needs read-only access to your Git repository so that it can read the configs committed to the repository and apply them to your clusters.
If your repository does not require authentication for read-only access, you can
continue to configure Config Sync and
use none
as your authentication type. For example, if you can browse the
repository using a web interface without logging in, or if you can use git
clone
to create a clone of the repository locally without providing credentials
or using saved credentials, then you don't need to authenticate. In this case,
you don't need to create a Secret.
However, most users need to create credentials because read access to their
repository is restricted. If credentials are required, they are stored in the
git-creds
Secret on each enrolled cluster (unless you are using a Google
service account). The Secret must be named git-creds
because this is a fixed value.
Config Sync supports the following mechanisms for authentication:
- SSH key pair
cookiefile
- Token
- Google service account (Cloud Source Repositories only)
The mechanism that you choose depends on what your repository supports. Generally, we recommend using an SSH key pair. GitHub and Bitbucket both support using an SSH key pair. However, if you are using a repository in Cloud Source Repositories, we recommend that you use a Google service account instead as the process is simpler. If your organization hosts your repository and you don't know which authentication methods are supported, contact your administrator.
SSH key pair
An SSH key pair consists of two files, a public key and a private key. The
public key typically has a .pub
extension.
To use an SSH key pair, complete the following steps:
Create an SSH key pair to allow Config Sync to authenticate to your Git repository. This step is necessary if you need to authenticate to the repository to clone it or read from it. Skip this step if a security administrator provides you with a key pair. You can use a single key pair for all clusters, or a key pair per cluster, depending on your security and compliance requirements.
The following command creates a 4096-bit RSA key. Lower values are not recommended:
ssh-keygen -t rsa -b 4096 \ -C "GIT_REPOSITORY_USERNAME" \ -N '' \ -f /path/to/KEYPAIR_FILENAME
Replace the following:
GIT_REPOSITORY_USERNAME
: the username that you want Config Sync to use to authenticate to the repository/path/to/KEYPAIR_FILENAME
: a path to the key pair
If you are using a third-party Git repository host such as GitHub, or you want to use a service account with Cloud Source Repositories, we recommend that you use a separate account.
Configure your repository to recognize the newly created public key. Refer to the documentation for your Git hosting provider. Instructions for some popular Git hosting providers are included for convenience:
- Cloud Source Repositories
- Bitbucket
- GitHub. We recommend that you create separate deploy keys to provide read-only access to a single GitHub repository.
- GitLab
Add the private key to a new Secret in the cluster:
kubectl create ns config-management-system && \ kubectl create secret generic git-creds \ --namespace=config-management-system \ --from-file=ssh=/path/to/KEYPAIR_PRIVATE_KEY_FILENAME
Replace
/path/to/KEYPAIR_PRIVATE_KEY_FILENAME
with the name of the private key (the one without the.pub
suffix).Delete the private key from the local disk or otherwise protect it.
When you configure Config Sync and add the URL for your Git repository, use the SSH protocol. If you are using a repository in Cloud Source Repositories, you must use the following format when you enter your URL:
ssh://EMAIL@source.developers.google.com:2022/p/PROJECT_ID/r/REPO_NAME
Replace the following:
EMAIL
: your Google Cloud usernamePROJECT_ID
: the ID of the Google Cloud project where the repository is locatedREPO_NAME
: the name of the repository
cookiefile
The process for acquiring a cookiefile
depends on the configuration of your
repository. For an example, see
Generate static credentials
in the Cloud Source Repositories documentation.
The credentials are usually stored in the .gitcookies
file in your home
directory, or they might be provided to you by a security administrator.
To use a cookiefile
, complete the following steps:
After you create and obtain the
cookiefile
, add it to a new Secret in the cluster.If you don't use an HTTPS proxy, create the Secret with the following command:
kubectl create ns config-management-system && \ kubectl create secret generic git-creds \ --namespace=config-management-system \ --from-file=cookie_file=/path/to/COOKIEFILE
If you need to use an HTTPS proxy, add it to the Secret together with
cookiefile
by running the following command:kubectl create ns config-management-system && \ kubectl create secret generic git-creds \ --namespace=config-management-system \ --from-file=cookie_file=/path/to/COOKIEFILE \ --from-literal=https_proxy=HTTPS_PROXY_URL
Replace the following:
/path/to/COOKIEFILE
: the appropriate path and filenameHTTPS_PROXY_URL
: the URL for the HTTPS proxy that you use when communicating with the Git repository
Protect the contents of the
cookiefile
if you still need it locally. Otherwise, delete it.
Token
If your organization does not permit the use of SSH keys, you might prefer to use a token. With Config Sync, you can use GitHub's personal access tokens (PATs), GiLab's PATs or deploy keys, or Bitbucket's app password as your token.
To create a Secret using your token, complete the following steps:
Create a token using GitHub, GitLab, or Bitbucket:
- GitHub: Create a PAT.
Grant the token the
repo
scope so that it can read from private repositories. Because you bind a PAT to a GitHub account, we also recommend that you create a machine user and bind your PAT to the machine user. - GitLab: Create a PAT or create a deploy token
- Bitbucket: Create an app password.
- GitHub: Create a PAT.
Grant the token the
After you create and obtain the token, add it to a new Secret in the cluster.
If you don't use an HTTPS proxy, create the Secret with the following command:
kubectl create ns config-management-system && \ kubectl create secret generic git-creds \ --namespace="config-management-system" \ --from-literal=username=USERNAME \ --from-literal=token=TOKEN
Replace the following:
USERNAME
: the username that you want to use.TOKEN
: the token that you created in the previous step.
If you need to use an HTTPS proxy, add it to the Secret together with
username
andtoken
by running the following command:kubectl create ns config-management-system && \ kubectl create secret generic git-creds \ --namespace=config-management-system \ --from-literal=username=USERNAME \ --from-literal=token=TOKEN \ --from-literal=https_proxy=HTTPS_PROXY_URL
Replace the following:
USERNAME
: the username that you want to use.TOKEN
: the token that you created in the previous step.HTTPS_PROXY_URL
: the URL for the HTTPS proxy that you use when communicating with the Git repository.
Protect the token if you still need it locally. Otherwise, delete it.
Google service account
If your repository is in Cloud Source Repositories, you can give Config Sync access to a repository in the same project as your managed cluster by using a Google service account.
To use a repository in Cloud Source Repositories as your Config Sync repository, complete the following steps:
Retrieve your Cloud Source Repositories URL:
List all repositories:
gcloud source repos list
From the output, copy the URL from the repository that you want to use. For example:
REPO_NAME PROJECT_ID URL my-repo my-project https://source.developers.google.com/p/my-project/r/my-repo-csr
You need to use this URL when you configure Config Sync in the following section. If you configure Config Sync using the Google Cloud console, you add the URL in the URL field. If you configure Config Sync using the Google Cloud CLI, you add the URL to the
syncRepo
field of your configuration file.
When you configure Config Sync, select the appropriate authentication type. The authentication type that you should pick differs depending on what type of cluster you have and how you have enabled Workload Identity.
Workload Identity enabled: Use this method if you have GKE Workload Identity enabled or if you're using fleet Workload Identity. If you're using fleet Workload Identity, then you can use this authentication method for both GKE and non-GKE clusters.
Config Sync uses fleet Workload Identity by default if the cluster is registered in a fleet. Make sure that fleet Workload Identity is enabled on your registered clusters. For more information, see Register a cluster. If your cluster is in a project different from the fleet host project, you need to bind the Google service account with the Kubernetes service account in the fleet host project.
Workload Identity not enabled: You can only use the method for GKE clusters.
Workload Identity enabled
If necessary, create a service account. Ensure that the service account has read access to Cloud Source Repositories by granting it the
source.reader
role.If you configure Config Sync using the Google Cloud console, select Workload Identity as the Authentication Type and then add your service account email.
If you configure Config Sync using the Google Cloud CLI, add
gcpserviceaccount
as thesecretType
and then add your service account email togcpServiceAccountEmail
.After configuring Config Sync, create an IAM policy binding between the Kubernetes service account and the Google service account. The Kubernetes service account is not created until you configure Config Sync for the first time.
If you are using clusters that are registered to a fleet, you only have to create the policy binding once per fleet. All clusters registered in a fleet share the same Workload Identity pool. With fleet's concept of sameness, if you add the IAM policy to your Kubernetes service account in one cluster, then the Kubernetes service account from the same namespace on other clusters in the same fleet also get the same IAM policy.
This binding lets the Config Sync Kubernetes service account act as the Google service account:
gcloud iam service-accounts add-iam-policy-binding \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:PROJECT_ID.svc.id.goog[config-management-system/KSA_NAME]" \ GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
Replace the following:
PROJECT_ID
: If you're using GKE Workload Identity, add your organization's project ID.If you're using fleet Workload Identity, you can use two different project IDs. In
serviceAccount:PROJECT_ID
, add the project ID of the fleet that your cluster is registered to. InGSA_NAME@PROJECT_ID
, add a project ID for any project that has read access to the repository in Cloud Source Repositories.KSA_NAME
: the Kubernetes service account for the reconciler. For root repositories, if the RootSync name isroot-sync
,KSA_NAME
isroot-reconciler
. Otherwise, it isroot-reconciler-ROOT_SYNC_NAME
.For namespace repositories, if the RepoSync name is
repo-sync
,KSA_NAME
isns-reconciler-NAMESPACE
. Otherwise, it isns-reconciler-NAMESPACE-REPO_SYNC_NAME
.GSA_NAME
: the custom Google service account that you want to use to connect to Cloud Source Repositories. Make sure that the Google service account that you select has thesource.reader
role.
Workload Identity not enabled
If you configure Config Sync using the Google Cloud console, select Google Cloud Repository as the Authentication Type.
If you configure Config Sync using the Google Cloud CLI, add
gcenode
as thesecretType
.Selecting either Google Cloud Repository or
gcenode
lets you use the Compute Engine default service account. By default, the Compute Engine default service account,PROJECT_ID-compute@developer.gserviceaccount.com
, hassource.reader
access to the repository for the same project. However, if your Cloud Source Repositories is located in a project different from your cluster's project, you have to grant default Compute Engine service account from the cluster's project, thesource.reader
in the Cloud Source Repositories's project.You can add the
source.reader
role with the following command:gcloud projects add-iam-policy-binding PROJECT_ID \ --member serviceAccount:PROJECT_NUMBER-compute@developer.gserviceaccount.com \ --role roles/source.reader
Replace the following:
PROJECT_ID
: your organization's project IDPROJECT_NUMBER
: your organization's project number
You cannot modify access scopes after you create a node pool. However, you can create a new node pool with the proper access scope while using the same cluster. The default
gke-default
scope does not includecloud-source-repos-ro
.
Grant Config Sync read-only access to OCI
Config Sync needs read-only access to your OCI image stored in Artifact Registry so that it can read the configs included in the image and apply them to your clusters.
If your image does not require authentication for read-only access, you can
continue to
configure Config Sync
and use none
as your authentication type. For example, if your image is public
and can be accessed by anyone on the internet, then you don't need to authenticate.
However, most users need to create credentials to access restricted images.
For images with restricted read access, you can use a Google service
account for authentication with gcenode
or gcpserviceaccount
as the
authentication type.
gcenode
If your cluster is a GKE cluster and Workload Identity is
not enabled, you can use gcenode
as your authentication type.
Config Sync uses the Compute Engine default service account.
You must grant your Compute Engine default service account
reader access to Artifact Registry.
Save the project number to an environment variable by running the following command:
export PROJECT_NUMBER=$(gcloud projects describe PROJECT_ID \ --format=json | jq -r .projectNumber)
Replace
PROJECT_ID
with your project ID.Grant the Compute Engine service account read permission to Artifact Registry by running the following command:
gcloud projects add-iam-policy-binding PROJECT_ID \ --member=serviceAccount:PROJECT_NUMBER-compute@developer.gserviceaccount.com \ --role=roles/artifactregistry.reader
gcpserviceaccount
If your cluster uses
GKE Workload Identity
or fleet Workload Identity,
you can use gcpserviceaccount
as your authentication type.
If you don't already have a service account, create a service account and grant it the Artifact Registry Reader (
roles/artifactregistry.reader
) IAM role.Create an IAM policy binding between the Kubernetes service account and the Google service account by running the following command:
gcloud iam service-accounts add-iam-policy-binding \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:PROJECT_ID.svc.id.goog[config-management-system/KSA_NAME]" \ GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
Replace the following:
PROJECT_ID
: if you're using GKE Workload Identity, this is your organization's project ID. If you're using fleet Workload Identity, you can use two different project IDs. ForserviceAccount:PROJECT_ID
, add the project ID of the fleet that your cluster is registered to. ForGSA_NAME@PROJECT_ID
, add a project ID for any project that has read access to the repository in Cloud Source Repositories.KSA_NAME
: the Kubernetes service account for the reconciler.- For root repositories, if the
RootSync
name isroot-sync
, addroot-reconciler
. Otherwise, addroot-reconciler-ROOT_SYNC_NAME
. - For namespace repositories, if the
RepoSync
name isrepo-sync
, addns-reconciler-NAMESPACE
. Otherwise, addns-reconciler-NAMESPACE-REPO_SYNC_NAME-REPO_SYNC_NAME_LENGTH
whereREPO_SYNC_NAME_LENGTH
is the number of characters inREPO_SYNC_NAME
.
- For root repositories, if the
GSA_NAME
: the custom Google service account that you want to use to connect to Artifact Registry. The service account must have the Artifact Registry Reader (roles/artifactregistry.reader
) IAM role.
Grant Config Sync read-only access to Helm
Config Sync needs read-only access to your Helm repository so that it can read the Helm charts in your repository and install them in your clusters.
If your repository does not require authentication for read-only access, you can
continue to
configure Config Sync
and use none
as your authentication type. For example, if your Helm repository
is public and can be accessed by anyone on the internet, then you don't need to
authenticate.
However, most users need to create credentials to access private Helm repositories. Config Sync supports the following mechanisms for authentication:
token
gcenode
gcpserviceaccount
token
Create a Secret with a Helm repository username and password:
kubectl create secret generic SECRET_NAME \
--namespace=config-management-system \
--from-literal=username=USERNAME \
--from-literal=password=PASSWORD
Replace the following:
SECRET_NAME
: the name that you want to give your Secret.USERNAME
: the Helm repository username.PASSWORD
: the Helm repository password.
When you Configure the Config Management Operator,
you will use the Secret name you chose for spec.helm.secretRef.name
.
gcenode
If your cluster is a GKE cluster and Workload Identity is
not enabled, you can use gcenode
as your authentication type.
Config Sync uses the Compute Engine default service account.
You must grant your Compute Engine default service account
reader access to Artifact Registry. You might need to grant the storage-ro
access
scope to grant read-only
permission to pull images.
Save the project number to an environment variable:
export PROJECT_NUMBER=$(gcloud projects describe PROJECT_ID --format='value(projectNumber)')
Replace
PROJECT_ID
with your project ID.Grant the Compute Engine service account read permission to Artifact Registry:
gcloud projects add-iam-policy-binding PROJECT_ID \ --member=serviceAccount:PROJECT_NUMBER-compute@developer.gserviceaccount.com \ --role=roles/artifactregistry.reader
gcpserviceaccount
If you store your Helm chart in Artifact Registry and your cluster uses
GKE Workload Identity
or fleet Workload Identity,
you can use gcpserviceaccount
as your authentication type.
If you don't already have a service account, create a service account and grant it the Artifact Registry Reader (
roles/artifactregistry.reader
) IAM role. For more information about Artifact Registry roles and permissions, see Configure roles and permissions for Artifact Registry.Create an IAM policy binding between the Kubernetes service account and the Google service account by running the following command:
gcloud iam service-accounts add-iam-policy-binding \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:PROJECT_ID.svc.id.goog[config-management-system/KSA_NAME]" \ GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
Replace the following:
PROJECT_ID
: if you're using GKE Workload Identity, this is your organization's project ID. If you're using fleet Workload Identity, you can use two different project IDs. ForserviceAccount:PROJECT_ID
, add the project ID of the fleet that your cluster is registered to. ForGSA_NAME@PROJECT_ID
, add a project ID for any project that has read access to the repository in Cloud Source Repositories.KSA_NAME
: the Kubernetes service account for the reconciler.- For root repositories, if the
RootSync
name isroot-sync
, addroot-reconciler
. Otherwise, addroot-reconciler-ROOT_SYNC_NAME
. - For namespace repositories, if the
RepoSync
name isrepo-sync
, addns-reconciler-NAMESPACE
. Otherwise, addns-reconciler-NAMESPACE-REPO_SYNC_NAME-REPO_SYNC_NAME_LENGTH
whereREPO_SYNC_NAME_LENGTH
is the number of characters inREPO_SYNC_NAME
.
- For root repositories, if the
GSA_NAME
: the custom Google service account that you want to use to connect to Artifact Registry. The service account must have the Artifact Registry Reader (roles/artifactregistry.reader
) IAM role.
Configure Config Sync
In this section, you configure the settings for your root repository. If you're syncing to a Git repository, you can use the Google Cloud console to guide you through the installation process and automate some steps.
When you install Config Sync using the Google Cloud console or the
Google Cloud CLI, Config Sync automatically creates a RootSync object named
root-sync
. You can use kubectl
commands to modify root-sync
and add
additional Config Sync configurations. To learn more, see
Configure Config Sync with kubectl
commands.
Console
Install Config Sync
To use Config Management, you must first register your clusters. Registering your clusters lets them share a common set of configurations and policies.
To register your clusters, complete the following tasks:
-
In the Google Cloud console:
If you use Google Kubernetes Engine, go to the GKE Config page under the Config & Policy section.
If you use Anthos, go to the Anthos Config page under the Config & Policy section.
- Click add Install Config Sync.
In the Available Clusters table, select the cluster that you want to register and then click Install Config Sync.
After a few minutes, you should see Installed in the Config sync status column for the cluster that you selected.
Deploy a package
After you have registered your clusters, you can deploy a package from a source of truth to your Config Sync cluster. You can deploy a package to more than one cluster at a time, and add multiple packages to a single cluster.
To deploy a package, complete the following steps:
In the Google Cloud console, go to the Config Sync dashboard.
Click Deploy Package.
In the Select clusters for package deployment table, select the cluster that you want to deploy a package to and then click Continue.
Select either Package hosted on Git or Package hosted on OCI as your source type and then click Continue.
In the Package details section, enter a Package name, which identifies the RootSync or RepoSync object.
In the Source section, complete the following:
For sources hosted in a Git repository, enter the following fields:
- Enter a Package name, which identifies the RootSync or RepoSync object.
- Enter the URL of the Git repository that you're using as a source of truth as the Repository URL.
- Optional: Update the Revision field to check out if you're not using
the default
HEAD
. - Optional: Update the Path field if you don't want to sync from the root repository.
- Optional: Update the Branch field if you're not using the default
main
branch.
For sources hosted in an OCI image, enter the following fields:
- Enter the URL of the OCI image that you're using as a source of truth as the Image.
- Enter the path of the directory to sync from, relative to the root directory, as the Directory.
In the Sync type field, choose either RootSync or RepoSync as the sync type.
RootSync objects are cluster-scoped and RepoSync object are namespace-scoped. For more information about these objects, see RootSync and RepoSync fields.
(Optional): Expand the Advanced settings section to complete the following:
Select an Authentication type:
- None: Use no authentication.
- SSH: Use an SSH key pair.
- Cookiefile: Use a
cookiefile
. - Token: Use a token.
- Google Cloud Repository: Use a Google service account to access a Cloud Source Repositories repository. Only select this option if Workload Identity is not enabled in your cluster.
- Workload Identity: Use a Google service account to access a Cloud Source Repositories repository.
Enter a number in seconds to set the Sync wait time, which determines how long Config Sync waits between syncing from the source of truth.
Enter a Git proxy URL for the HTTPS proxy to be used when communicating with the source of truth.
Choose Hierarchy to change the Source format.
The default value Unstructured is recommended in most cases since it lets you organize your source of truth however you want.
Click Deploy Package.
You are redirected to the Config Sync Packages page. After a few minutes, you should see Synced in the Sync status column for the cluster that you configured.
To update a package, on the Packages tab, expand the package name that you want to edit and then click edit Edit.
gcloud
Before you continue, make sure you've registered your clusters to a fleet.
- Prepare the configuration by either creating a new
apply-spec.yaml
manifest or by using an existing manifest. Using an existing manifest lets you configure your cluster with the same settings used by another cluster.
Create new manifest
To configure Config Sync with new settings for your cluster, create
a file named apply-spec.yaml
and copy the following YAML file into it.
You can set all of the optional spec.configSync
fields that you need
when you create your manifest, and later use
kubectl
commands for configuration.
You can also only set the spec.configSync.enabled
field as true
and
omit the optional fields. You can then later use kubectl
commands to
create additional RootSync objects
or RepoSyncs that you can fully manage using kubectl
commands later.
# apply-spec.yaml
applySpecVersion: 1
spec:
configSync:
# Set to true to install and enable Config Sync
enabled: true
# If you don't have a source of truth yet, omit the
# following fields. You can configure them later.
sourceType: SOURCE_TYPE
sourceFormat: FORMAT
syncRepo: REPO
syncBranch: BRANCH
secretType: SECRET_TYPE
gcpServiceAccountEmail: EMAIL
metricsGcpServiceAccountEmail: METRICS_EMAIL
policyDir: DIRECTORY
preventDrift: PREVENT_DRIFT
Replace the following:
SOURCE_TYPE
: addgit
to sync from a Git repository,oci
to sync from an OCI image, orhelm
to sync from a Helm chart. If no value is specified, the default isgit
.FORMAT
: addunstructured
to use an unstructured repository or addhierarchy
to use a hierarchical repository. These values are case-sensitive. This field is optional and the default value ishierarchy
. We recommend that you addunstructured
, because this format lets you organize your configs in the way that is most convenient to you.REPO
: add the URL of the source of truth. Git and Helm repository URLs use either the HTTPS or SSH protocol. For example,https://github.com/GoogleCloudPlatform/anthos-config-management-samples
. If you plan to use SSH as yoursecretType
, enter your URL with the SSH protocol. This field is required and if you don't enter a protocol, the URL is treated as an HTTPS URL.OCI URLs use the following format:
LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/PACKAGE_NAME
. By default, the image is pulled from thelatest
tag, but you can pull in images byTAG
orDIGEST
instead. SpecifyTAG
orDIGEST
in thePACKAGE_NAME
:- To pull by
TAG
:LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/PACKAGE_NAME:TAG
- To pull by
DIGEST
:LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/PACKAGE_NAME@sha256:DIGEST
- To pull by
BRANCH
: the branch of the repository to sync from. This field is optional and the default ismaster
.SECRET_TYPE
: one of the followingsecretTypes
:git
none
: Use no authentication.ssh
: Use an SSH key pair.cookiefile
: Use acookiefile
.token
: Use a token.gcpserviceaccount
: Use a Google service account to access a Cloud Source Repositories. If you select this authentication type, you need to create an IAM policy binding after you finish configuring Config Sync. For details, see the Google service account tab of the Grant Config Sync read-only access to Git section.gcenode
: Use a Google service account to access a Cloud Source Repositories. Only select this option if Workload Identity is not enabled in your cluster.
For more information on these authentication types, see Granting Config Sync read-only access to Git.
oci
none
: Use no authenticationgcenode
: Use the Compute Engine default service account to access an image in Artifact Registry. Only select this option if Workload Identity is not enabled in your cluster.gcpserviceaccount
: Use a Google service account to access an image.
helm
token
: Use a token.gcenode
: Use the Compute Engine default service account to access an image in Artifact Registry. Only select this option if Workload Identity is not enabled in your cluster.gcpserviceaccount
: Use a Google service account to access an image.
EMAIL
: If you addedgcpserviceaccount
as yoursecretType
, add your Google service account email address. For example,acm@PROJECT_ID.iam.gserviceaccount.com
.METRICS_EMAIL
: the email of the Google Cloud Service Account (GSA) used for exporting Config Sync metrics to Cloud Monitoring. The GSA should have the Monitoring Metric Writer (roles/monitoring.metricWriter
) IAM role. The Kubernetes ServiceAccountdefault
in the namespaceconfig-management-monitoring
should be bound to the GSA.DIRECTORY
: the path of the directory to sync from, relative to the root of the Git repository. All sub-directories of the directory that you specify are included and synced to the cluster. The default value is the root directory of the repository.PREVENT_DRIFT
: If set totrue
, enables the Config Sync admission webhook to prevent drifts by rejecting conflicting changes from being pushed to live clusters. The default setting isfalse
. Config Sync always remediates drifts no matter the value of this field.
For a complete list of fields that you can add to the spec
field,
see gcloud fields.
Use existing manifest
To configure your cluster with the same settings used by another cluster, fetch the settings from a registered cluster:
gcloud alpha container fleet config-management fetch-for-apply \
--membership=MEMBERSHIP_NAME \
--project=PROJECT_ID \
> CONFIG_YAML_PATH
Replace the following:
MEMBERSHIP_NAME
: the membership name of the registered cluster that has the Config Sync settings you want to usePROJECT_ID
: your project IDCONFIG_YAML_PATH
: the path to theapply-spec.yaml
file which contains the settings fetched from the cluster
2. Apply the apply-spec.yaml
file. If you are using an existing manifest,
you should apply the file to the cluster that you want to
configure with the settings that you fetched in the previous command:
gcloud beta container fleet config-management apply \
--membership=MEMBERSHIP_NAME \
--config=CONFIG_YAML_PATH \
--project=PROJECT_ID
Replace the following:
MEMBERSHIP_NAME
: the membership name that you chose when you registered your cluster. You can find the name withgcloud container fleet memberships list
.CONFIG_YAML_PATH
: the path to yourapply-spec.yaml
file.PROJECT_ID
: your project ID.
After you have finished configuring your root repository, you can optionally choose to configure syncing from multiple repositories, including other root repositories and namespace repositories. The namespace repositories are helpful if you want a repository that contains namespace-scoped configs synced to a particular namespace across clusters.
Verify the installation
After you have installed and configured Config Sync, you can verify that the installation completed successfully.
Console
Complete the following steps:
-
In the Google Cloud console:
If you use Google Kubernetes Engine, go to the GKE Config page under the Config & Policy section.
If you use Anthos, go to the Anthos Config page under the Config & Policy section.
- On the Packages tab, check the Sync status column in the cluster table. A successful installation of Config Sync has a status of Installed. A successfully configured source of truth has a status of Synced.
gcloud
Run the following command:
gcloud beta container fleet config-management status \
--project=PROJECT_ID
Replace PROJECT_ID
with your project's ID.
A successful installation has a status of SYNCED
. If you see an error after
running the preceding command, make sure that you created the git-creds
Secret. If you have created the Secret, try rerunning the following command:
gcloud beta container fleet config-management apply
You can also use the
nomos status
command to
check if the Config Sync is installed successfully. A valid installation
with no problems has a status of PENDING
or SYNCED
. An invalid or incomplete
installation has a status of NOT INSTALLED
OR NOT CONFIGURED
. The output
also includes any reported errors.
Upgrade Config Sync
Config Sync is upgraded whenever you upgrade Anthos Config Management. To learn more, see Upgrade Anthos Config Management.
Resource requests
Summary of total resource requests
The following tables list the combined amount of resource requests for each supported version of Config Sync, depending on what features you are using.
1.16
Feature | CPU (m) | Memory (Mi) |
---|---|---|
Config Sync | 330 m + 80 m * (number of RootSync and RepoSync objects)1 | 850 Mi + 600 Mi * (number of RootSync and RepoSync objects) |
Config Sync with the admission webhook enabled | 350 m + 80 m * (number of RootSync and RepoSync objects)1 | 1050 Mi + 600 Mi * (number of RootSync and RepoSync objects) |
Hierarchy Controller | 200 m | 200 Mi |
1 If the RootSync and RepoSync source type is set to
helm
, the CPU request is 120m * (number of RootSync and RepoSync
objects).
1.15
Feature | CPU (m) | Memory (Mi) |
---|---|---|
Config Sync | 330 m + 80 m * (number of RootSync and RepoSync objects)1 | 850 Mi + 600 Mi * (number of RootSync and RepoSync objects) |
Config Sync with the admission webhook enabled | 350 m + 80 m * (number of RootSync and RepoSync objects)1 | 1050 Mi + 600 Mi * (number of RootSync and RepoSync objects) |
Hierarchy Controller | 200 m | 200 Mi |
1 If the RootSync and RepoSync source type is set to
helm
, the CPU request is 120m * (number of RootSync and RepoSync
objects).
1.14
Feature | CPU (m) | Memory (Mi) |
---|---|---|
Config Sync | 330 m + 80 m * (number of RootSync and RepoSync objects)1 | 850 Mi + 600 Mi * (number of RootSync and RepoSync objects) |
Config Sync with the admission webhook enabled | 350 m + 80 m * (number of RootSync and RepoSync objects)1 | 1050 Mi + 600 Mi * (number of RootSync and RepoSync objects) |
Hierarchy Controller | 200 m | 200 Mi |
1 If the RootSync and RepoSync source type is set to
helm
, the CPU request is 120m * (number of RootSync and RepoSync
objects).
Detailed resource requests
The following table lists Kubernetes resource requirements for Config Sync components. For more information, see Managing Resources for Containers in the Kubernetes documentation.
1.16
Deployment name | CPU request (m) per replica | Memory request (Mi) per replica | Single or multiple repositories |
---|---|---|---|
git-importer |
450 | 400 | Single |
monitor |
Default1 | Default | Single |
resource-group-controller-manager |
110 | 300 | Multi |
admission-webhook2 |
10 | 100 | Multi |
otel-collector |
200 | 400 | Multi |
reconciler-manager |
20 | 150 | Multi |
reconciler (one per RootSync and RepoSync) |
70 + default34 | 500 + default4 | Multi |
hnc-controller-manager |
100 | 150 | Hierarchy Controller |
gke-hc-controller-manager |
100 | 50 | Hierarchy Controller |
1 The default resource request uses a CPU request of 10 milliCPU (mCPU) and a memory request of 10 Mi.
2 The admission webhook has two replicas, so when calculating the total resource requests, you need to double the value if you are using the admission webhook. The admission webhook is disabled by default.
3 If the RootSync and RepoSync source type is set to
helm
, the CPU request is 120m * (number of RootSync and RepoSync
objects).
4 If you're using Kustomize rendering, the resource requests are slightly higher. The CPU request is 80 + default. The memory request is 600 + default.
1.15
Deployment name | CPU request (m) per replica | Memory request (Mi) per replica | Single or multiple repositories |
---|---|---|---|
git-importer |
450 | 400 | Single |
monitor |
Default1 | Default | Single |
resource-group-controller-manager |
110 | 300 | Multi |
admission-webhook2 |
10 | 100 | Multi |
otel-collector |
200 | 400 | Multi |
reconciler-manager |
20 | 150 | Multi |
reconciler (one per RootSync and RepoSync) |
80 + default3 | 600 + default | Multi |
hnc-controller-manager |
100 | 150 | Hierarchy Controller |
gke-hc-controller-manager |
100 | 50 | Hierarchy Controller |
1 The default resource request uses a CPU request of 10 milliCPU (mCPU) and a memory request of 10 Mi.
2 The admission webhook has two replicas, so when calculating the total resource requests, you need to double the value if you are using the admission webhook. The admission webhook is disabled by default.
3 If the RootSync and RepoSync source type is set to
helm
, the CPU request is 120m * (number of RootSync and RepoSync
objects).
1.14
Deployment name | CPU request (m) per replica | Memory request (Mi) per replica | Single or multiple repositories |
---|---|---|---|
git-importer |
450 | 400 | Single |
monitor |
Default1 | Default | Single |
resource-group-controller-manager |
110 | 300 | Multi |
admission-webhook2 |
10 | 100 | Multi |
otel-collector |
200 | 400 | Multi |
reconciler-manager |
20 | 150 | Multi |
reconciler (one per RootSync and RepoSync) |
80 + default3 | 600 + default | Multi |
hnc-controller-manager |
100 | 150 | Hierarchy Controller |
gke-hc-controller-manager |
100 | 50 | Hierarchy Controller |
1 The default resource request uses a CPU request of 10 milliCPU (mCPU) and a memory request of 10 Mi.
2 The admission webhook has two replicas, so when calculating the total resource requests, you need to double the value if you are using the admission webhook. The admission webhook is disabled by default.
3 If the RootSync and RepoSync source type is set to
helm
, the CPU request is 120m * (number of RootSync and RepoSync
objects).
Bundled Helm and Kustomize versions
Config Sync leverages the Helm and Kustomize executables to render the configurations under the hood. The following table provides a list of Config Sync versions that support the rendering feature, alongside the bundled Helm and Kustomize versions.
Config Sync versions | Helm version | Kustomize version |
---|---|---|
1.16.0 and later | v3.11.3 | v5.1.1 |
1.15.0 to 1.15.3 | v3.11.3 | v5.0.1 |
1.11.0 to 1.14.3 | v3.6.3 | v4.5.2 |
For information about rendering Helm through Kustomize, see Configure Kubernetes with Kustomize. For information about using the Helm API, see Syncing Helm charts from Artifact Registry.
What's next
- Learn more about the
gcloud
commands for configuring Config Sync with Anthos Config Management. - Discover how to configure syncing from multiple repositories.
- Use the
nomos
command. - Learn more about troubleshooting Config Sync.
- Learn how to uninstall Config Sync.
- Review the Default Config Sync permissions.