Config Sync overview
Config Sync is an open source tool that lets cluster operators and platform administrators deploy consistent configurations and policies. You can deploy these configurations and policies to individual Kubernetes clusters, multiple clusters that can span hybrid and multi-cloud environments, and multiple namespaces within clusters. This process simplifies and automates configuration and policy management at scale. Config Sync also lets development teams independently manage their namespaces within clusters, while still being subject to policy guardrails set by administrators.
Using the same principles as Kubernetes itself, Config Sync continuously reconciles the state of registered clusters with a central set of Kubernetes declarative configuration files called configs. You can store configs in one or more Git repositories, Helm repositories (Preview), or OCI images. This GitOps approach (sometimes referred to as configuration as code) lets you manage and deploy common configurations with a process that is auditable, transactional, reviewable, and version-controlled.
In the following diagram, a platform administrator creates consistent configurations for three different clusters by applying configs to the clusters and namespaces within the cluster:
Config Sync benefits
The following points highlight some of the benefits that Config Sync provides you with:
- Reduce the risk of "shadow ops": If unvetted changes are pushed to live clusters, it can be difficult to understand the differences between documented configuration and your live environment. You can require that all cluster configuration changes are applied using Config Sync, lock down direct access to the Kubernetes API, and trace changes back to their sources of truth in Git.
- Use GitOps best practices: Before any changes are pushed to your live environment, you can require code reviews using any repository management tools that you prefer. Use Config Sync to audit exactly which commit caused a configuration change.
- Reduce downtime due to configuration-related outages: Config Sync lets you use a "revert then investigate" strategy to roll back breaking changes and get your live clusters back into a good working state before fixing the problematic change and applying it as a new commit.
- Use Continuous Integration / Continuous Deployment (CI/CD) pipelines: You can use a CI/CD workflow to render configuration using any tools and formats you prefer, test and validate your changes, and commit them automatically when tests pass. Config Sync then applies the changes and then monitors and remediates configuration drift.
Understanding Config Sync
Config Sync lets you create a common set of configuration and cluster-level policies, such as Policy Controller constraints, and consistently apply them across registered and connected clusters from a single source of truth in Git. Before you can configure a cluster, you need to create a config and a repository.
A config is a Kubernetes configuration declaration written in YAML or JSON.
Config Sync reads and applies the config to one or more clusters to create or
configure a Kubernetes object or resource in those clusters, or to provide
information needed by Config Sync itself. A config can contain any configuration
detail you could apply to a Kubernetes cluster using
You can also declare multiple configs in a single file.
Config Sync reads three types of files:
Files with other suffixes are ignored.
Before you write a config, you should understand the Kubernetes object that you are writing the config for.
The repository (or repo) is the Git repository where configs are stored, including the config for Config Sync itself. When you configure Config Sync, you configure the repository, the branch, and the subdirectory that Config Sync monitors for changes.
Config Sync lets you sync configurations from multiple repositories to the same set of clusters. In Config Sync, you can reference each repository using its Git reference (a Git URL, repository branch, commit or tag, and directory). This configuration decouples the config deployment lifecycle for different teams. It also provides you with more autonomy in choosing where you want to place the repository and how to structure it.
You can create two types of repositories; root repositories and namespace repositories:
Root repositories: Root repositories let you sync cluster-scoped and namespace-scoped configs. Root repositories use admin-level credentials to enforce policies on application namespaces and override local changes that drift from the state that you declared in your configs. Root repositories are typically governed by a central administrator.
You can use two different structures for your root repository:
- Unstructured: The unstructured source format lets you organize the configs in your repository in whatever way is most convenient. This format can be especially useful if you are organizing and/or generating configs using a tool such as Kustomize, kpt, or helm. Unstructured is the recommended format for most users and is the default when configuring Config Sync through the Google Cloud console. For more information, see Using an unstructured repo.
- Hierarchical: The hierarchical, or structured, source format
separates configs into distinct categories for system configuration,
cluster metadata, cluster-level configuration, and namespace
configuration to help you organize the configs. It also supports
hierarchical inheritance of configuration across multiple namespaces
based on the directory structure, which is discussed more in the
section. Hierarchical is the default repository format for
Config Sync if you don't specify a
sourceFormatin your manifest. For more information, see Hierarchical repo overview.
To learn how to configure your root repository, see install Config Sync.
Namespace repositories: Namespace repositories are optional and can contain namespace-scoped configs synced to a particular namespace across clusters. You can delegate the setup and control of a namespace repository to non-administrative users. Namespace repositories must use the unstructured repo format.
The following diagram shows you an overview of how teams might sync their clusters to a single root repository (managed by an admin) and multiple namespace repositories (managed by application operators):
In the preceding diagram, a central administrator manages the centralized infrastructure for the organization and enforces policies on the cluster and on all namespaces in the organization. The application operators, who are responsible for managing live deployments, apply configurations to the applications in the namespaces that they work on.
From Anthos Config Management version 1.12.0 and later, you can configure Config Sync to sync from OCI images. You can store images in either Artifact Registry or Container Registry. To use this feature, you must enable the RootSync and RepoSync APIs.
From Anthos Config Management version 1.13.0 and later, you can store your configs in a Helm repository (Preview). To use this feature, you must enable the RootSync and RepoSync APIs. You can also patch Helm charts with Kustomize instead for different use cases, for example using a combination of Helm charts and Kustomize or rendering multiple public Helm charts in a single reconciler.
Configuring multiple clusters
Config Sync helps you to deploy consistent configuration and policies across multiple clusters that can span hybrid and multi-cloud environments.
Config Sync helps you configure multiple clusters by providing you with the following options:
- Instead of repeatedly running the
kubectl applycommand manually, you can orchestrate deployment of configuration changes to fleets of clusters through Git. For more information, see Safe rollouts with Anthos Config Management.
- Ensure that a change is pushed to each relevant cluster, and only relevant clusters, based on metadata applied to each cluster. For more information, see Configuring only a subset of clusters. Cluster metadata is managed slightly differently for the unstructured and hierarchical repository formats.
We highly recommend that you register your clusters. Registering your clusters facilitates the observation of the current status of all clusters through the Google Cloud console and enables centralized configuration and management of Config Sync across registered clusters. For more information, see Registering a cluster, Configuring Config Sync, and Upgrading Config Sync versions.
Configuring namespaces with Config Sync provides you with the following capabilities:
- You can consistently provision Kubernetes namespaces with namespace-scoped policies, such as RBAC roles, across registered and connected clusters. Namespace-scoped policies make it easier to implement and manage multi-tenancy within your clusters.
- Apply policies to multiple related namespaces, without duplicating configs, and with the ability to override or extend a config for a given namespace or set of namespaces, making it easier to apply consistent policies across tenants.
With unstructured repositories, you can use Hierarchy Controller to propagate policies across namespace hierarchies that you define for your clusters. For more information, see the Hierarchy Controller overview.
With hierarchical repositories, you can use groups of namespaces called abstract namespaces to control which namespaces policies should be propagated to. Abstract namespaces require organizing namespaces hierarchically in your repo's directory tree. For more information, see Configuring namespaces and namespace-scoped objects and the Namespace inheritance overview.
Both unstructured and hierarchical formats also support targeting non-hierarchical sets of namespaces using label selectors.
The nomos command
Config Sync provides an API. The
nomos.exe on Windows) command
consumes the API, checks the status of your installation, and
For more information, see Using the nomos command-line tool.