Using configs in Git repositories

Config Sync is designed for cluster operators who manage many clusters. You can ensure that your clusters meet business and compliance standards by allowing Config Sync to manage namespaces, Roles, RoleBindings, ResourceQuotas, and other important Kubernetes objects, across your fleet.

Config Sync keeps your enrolled clusters in sync using configs. A config is a YAML or JSON file that is stored in your Git repository and contains the same types of configuration details that you can manually apply to a cluster using the kubectl apply command. You can create a config for any Kubernetes object that can exist in a cluster. This topic covers the following:

  • How to organize configs in a directory.
  • How to initialize a Git repository to store the configs.
  • How to configure Config Sync to read from the configs.

Storing configs in repos

Config Sync uses a Git repository for config storage and version control, and takes actions based on its contents. In Config Sync, this repository is called the repo. There are two different formats for a repo: unstructured and hierarchical.

  • The unstructured source format lets you organize the configs in your repository in whatever way is most convenient. This format can be especially useful if you are organizing and/or generating configs using a tool such as kustomize, kpt, or helm.
  • The hierarchical, or structured, source format separates configs into distinct categories to help you organize the configs. The categories are system configuration, cluster metadata, cluster-level configuration, and namespace configuration. For more information about the hierarchical repo structure, see Structure of the hierarchical repo.

The following table highlights the differences between the hierarchical and unstructured repo formats:

Features Hierarchical repo format Unstructured repo format
Namespace selectors Supported Supported
Cluster selectors Supported Supported
Abstract namespaces Supported Not supported
Repo objects Supported Not supported
HierarchyConfig objects Supported Not supported
Hierarchical namespaces in Hierarchy Controller Not supported Supported
Used as the format for a namespace repository Not supported Supported
Used as the format for a root repository Supported Supported
Used together with Hierarchy Controller Not recommended Supported
The nomos init command Supported Not supported
The nomos hydrate command Supported Not supported
The nomos vet command Supported Supported with the --source-format=unstructured flag
All other nomos commands Supported Supported

Example of an unstructured config layout

The following example demonstrates a way to organize your configs, including Google Cloud resources.

You can put your raw configs into a raw-configs directory, use a script or an automated pipeline to render the raw configs, and store the output in the manifests directory. In the end, you configure Config Sync to sync from your manifests directory.

├── manifests
│   ├── access-control
│   │   ├── ...
│   ├── change-control
│   │   ├── ...
│   ├── git-ops
│   │   └── ...
│   ├── logging-monitoring
│   │   └── ...
│   ├── networking
│   │   └── ...
│   ├── project-factory
│   │   └── ...
│   └── resource-hierarchy
│   │   └── ...
├── raw-configs
│   ├── access-control
│   │   └── ...
│   ├── change-control
│   │   └── ...
│   ├── git-ops
│   │   └── ...
│   ├── logging-monitoring
│   │   └── ...
│   ├── networking
│   │   └── ...
│   ├── project-factory
│   │   └── ...
│   └── resource-hierarchy
│   │   └── ...
└── scripts
    └── render.sh

Initializing a Git repository

How you intitialze a repo depends on its structure.

  • For the unstructured format, you can initialize an empty Git repository and start adding configs in the repo.
  • For the hierarchical format, you can initialize a repo using the nomos init command, or you can create the directory structure manually.

Configuring Config Sync to read from the repo

You configure the location of the repo when you install Config Sync, and you can edit its configuration later in the Operator's configuration file. In addition to the location of the repo, you can specify a Git branch and a subdirectory to watch, if the Git repository has contents other than configs.

After updating the configuration file, you apply it to the cluster using the kubectl apply command. Config Sync does not manage the configuration of the Operator itself.

You can grant people access to a given product team's deployment repo. However, you should be aware that when you are granting a person access to a deployment repo, that person is also granted the same RBAC as the reconciler running for that repo.

To configure authentication and authorization between Config Sync and the repo, see the installation step about configuring the git-creds Secret.

Ignoring object mutations

If you don't want Config Sync to maintain the state of the object an the cluster after it exists, you can add the client.lifecycle.config.k8s.io/mutation: ignore annotation to the object that you want Config Sync to ignore mutations in. To use the annotation, you need to enable multi-repo mode and use a version of 1.7.0 or later.

The following example shows you how to add the annotation to an object:

metadata:
  annotations:
    client.lifecycle.config.k8s.io/mutation: ignore 

You cannot manually modify this annotation on managed objects in the cluster.

What's next