Config Sync overview

Config Sync lets cluster operators and platform administrators deploy consistent configurations and policies. You can deploy these configurations and policies across individual Kubernetes clusters, multiple clusters that can span hybrid and multi-cloud environments, and multiple namespaces within clusters. This process simplifies and automates configuration and policy management at scale. Config Sync also lets development teams independently manage their namespaces within clusters, while still being subject to policy guardrails set by administrators.

Using the same principles as Kubernetes itself, Config Sync continuously reconciles the state of registered clusters with a central set of Kubernetes declarative configuration files called configs. Configs are stored in one or more Git repositories, keeping them consistent for the configured Kubernetes objects. This GitOps approach (sometimes also referred to as configuration as code) lets you manage and deploy common configurations with a process that is auditable, transactional, reviewable, and version-controlled.

The following diagram shows how a platform administrator can create consistent configurations for three different clusters by applying configs to the clusters and namespaces within the cluster:

A platform administrator deploying multiple configs to their clusters

Config Sync benefits

The following examples highlight some of the benefits that Config Sync provides you with:

  • Reduce the risk of "shadow ops": If unvetted changes are pushed to live clusters, it can be difficult to understand the differences between documented configuration and your live environment. You can require that all cluster configuration changes are propagated using Config Sync, lock down direct access to the Kubernetes API, and trace changes back to their sources of truth in Git.
  • Require code reviews: Before any changes are pushed to your live environment, you can use Config Sync to require code reviews and then audit exactly which commit caused a configuration change.
  • Reduce downtime due to configuration-related outages: Config Sync lets you use a "revert then investigate" strategy to roll back breaking changes and get your live clusters back into a good working state before fixing the problematic change and applying it as a new commit.
  • Use Continuous Integration / Continuous Deployment (CI/CD) pipelines: Use a CI/CD workflow to render configuration using any tools and formats you prefer, test and validate your changes, and commit them automatically when tests pass. Config Sync then applies the changes and subsequently monitors and remediates configuration drift.

Understanding Config Sync

Prerequisites

Config Sync uses namespaces, labels, and annotations as core parts of its implementation. A good understanding of these concepts is helpful before you get started with using the product.

Configuring clusters

Config Sync lets you create a common set of configuration and cluster-level policies, such as Policy Controller constraints, and consistently apply them across registered and connected clusters from a single source of truth in Git. Before you can configure a cluster, you need to create a config and a repository.

Configs

A config is a Kubernetes configuration declaration written in YAML or JSON. Config Sync reads and applies the config to one or more clusters to create or configure a Kubernetes object or resource in those clusters, or to provide information needed by Config Sync itself. A config can contain any configuration detail you could apply to a Kubernetes cluster using kubectl edit or kubectl apply. You can also declare multiple configs in a single file.

Before you write a config, you need to understand the required and optional fields allowed for that Kubernetes object.

For more information, see the Configs overview and Configuring Kubernetes objects, which shows you how to create a config.

Repositories

The repository (or repo) is the Git repository where configs are stored, including the config for Config Sync itself. When you configure Config Sync, you configure the repository, the branch, and the subdirectory that Config Sync monitors for changes.

Config Sync lets you sync configurations from multiple repositories to the same set of clusters. Enabling the ability to sync from multiple repositories gives you access to additional functionality (for example, drift prevention and ignoring object mutations). You can enable this functionality even if you only want to use a root repository and don't want to use namespace repositories. To learn more, see Syncing from multiple repositories.

You can create a root repository and namespace repositories:

  • Root repository: This repository lets you sync cluster-scoped and namespace-scoped configs. The root repository uses admin-level credentials to enforce policies on application namespaces and override local changes that drift from the state that you declared in your configs. A central administrator typically governs the root repository and only one root repository can exist for each cluster.

    You can use two different structures for your root repository:

    • Unstructured: The unstructured source format lets you organize the configs in your repository in whatever way is most convenient. This format can be especially useful if you are organizing and/or generating configs using a tool such as kustomize, kpt, or helm. Unstructured is the recommended format for most users. For more information, see Using an unstructured repo.

    • Hierarchical: The hierarchical, or structured, source format separates configs into distinct categories for system configuration, cluster metadata, cluster-level configuration, and namespace configuration to help you organize the configs. It also supports hierarchical inheritance of configuration across multiple namespaces based on the directory structure, which is discussed more in the Configuring namespaces section. Hierarchical is the default repository format for Config Sync. For more information, see Hierarchical repo overview.

  • Namespace repositories: Namespace repositories are optional and can contain namespace-scoped configs synced to a particular namespace across clusters. You can delegate the setup and control of a namespace repository to non-administrative users. Namespace repositories must use the unstructured repo format. For more information, see Configuring syncing from namespace repositories.

Configuring multiple clusters

Config Sync helps you to deploy consistent configuration and policies across multiple clusters that can span hybrid and multi-cloud environments.

Config Sync helps you configure multiple clusters by providing you with the following options:

  • Instead of repeatedly running the kubectl apply command manually, you can orchestrate deployment of configuration changes to fleets of clusters through Git. For more information, see Safe rollouts with Anthos Config Management.
  • Ensure that a change is pushed to each relevant cluster, and only relevant clusters, based on metadata applied to each cluster. For more information, see Configuring only a subset of clusters. Cluster metadata is managed slightly differently for the unstructured and hierarchical repository formats.

We highly recommend that you register your clusters. Registering your clusters facilitates the observation of the current status of all clusters through the Google Cloud Console and enables centralized configuration and management of Config Sync across registered clusters. For more information, see Registering a cluster, Configuring Config Sync, and Upgrading Config Sync versions.

Configuring namespaces

Configuring namespaces with Config Sync provides you with the following capabilities:

  • You can consistently provision Kubernetes namespaces with namespace-scoped policies, such as RBAC roles, across registered and connected clusters. Namespace-scoped policies make it easier to implement and manage multi-tenancy within your clusters.
  • Apply policies to multiple related namespaces, without duplicating configs, and with the ability to override or extend a config for a given namespace or set of namespaces, making it easier to apply consistent policies across tenants.

These capabilities work differently in unstructured and hierarchical repository formats.

With unstructured repositories, you can use Hierarchy Controller to propagate policies across namespace hierarchies that you define for your clusters. For more information, see the Hierarchy Controller overview.

With hierarchical repositories, you can use groups of namespaces called abstract namespaces to control which namespaces policies should be propagated to. Abstract namespaces require organizing namespaces hierarchically in your repo's directory tree. For more information, see Configuring namespaces and namespace-scoped objects and the Namespace inheritance overview.

Both unstructured and hierarchical formats also support targeting non-hierarchical sets of namespaces using label selectors.

The nomos command

Config Sync provides an API. The nomos and nomos.exe commands consume the API, simplify the process of setting up a repository using the hierarchical format, and validate configs.

For more information, see Using the nomos command.

What's next