This page explains how to add and organize configurations stored in a source of truth.
About configs
Config Sync is designed for cluster operators who manage many clusters. You can ensure that your clusters meet business and compliance standards by letting Config Sync manage namespaces, Roles, RoleBindings, ResourceQuotas, and other important Kubernetes objects, across your fleet.
When Config Sync manages these resources, it keeps your enrolled clusters in
sync using configs. A config is a YAML or JSON file that is stored in a source
of truth. Config Sync supports Git repositories, OCI images, and Helm charts
as the source of truth. Configs
contain the same type of configuration details that you can
manually apply to a cluster using the kubectl apply
command. You can create a
config for any Kubernetes object that can exist in a cluster. However, some
Kubernetes objects, such as Secrets, contain sensitive information that might be
inappropriate to store in a source of truth. Use your judgment when considering
whether to manage these types of objects using Config Sync.
You can also use Config Sync with Config Connector to sync configs for Google Cloud resources. To learn more about working with Config Connector, see managing Google Cloud resources using Config Connector. You can also simplify the installation of Config Sync and Config Connector by setting up Config Controller.
Limitations
You can't change any immutable field in a config by changing the value in the source of truth. If you need to update an immutable field, manually delete the object in the cluster. Config Sync can then re-create the object with the new field value.
Select how to organize your configs
Config Sync uses a source of truth for config storage and version control. There are two different formats you can choose for your source of truth; unstructured and hierarchical.
The unstructured source format lets you organize configs in whatever way is most convenient. This format can be especially useful if you are organizing or generating configs using a tool such as Kustomize, kpt, or Helm. For an example of how you might organize your configs, see Example format for an unstructured repository.
The hierarchical, or structured, source format separates configs into distinct categories to help you organize the configs. The categories are system configuration, cluster metadata, cluster-level configuration, and namespace configuration. For more information about the hierarchical source format, see Structure of the hierarchical repo.
Unstructured is the recommended format for most users. In addition, when you configure RepoSync objects, you must use the unstructured source format.
Supported features for unstructured and hierarchical formats
The following table highlights the differences between the unstructured and hierarchical formats:
Features | Unstructured format (recommended) | Hierarchical format |
---|---|---|
Used as the format for a RootSync object or the central source of truth | Supported | Supported |
Used as the format for a RepoSync object | Supported | Not supported |
ClusterSelector |
Supported | Supported |
NamespaceSelector |
Supported | Supported |
The nomos hydrate command |
Supported with the --source-format=unstructured flag |
Supported |
The nomos init command |
Not supported | Supported |
The nomos vet command |
Supported with the --source-format=unstructured flag |
Supported |
All other nomos commands |
Supported | Supported |
Abstract namespaces | Not supported | Supported |
Repo objects |
Not supported | Supported |
HierarchyConfig objects |
Not supported | Supported |
When to add configs to the source
If you are creating an unstructured format, you can start adding configs to
it as soon as it's created. If you are creating a hierarchical format, use
the nomos init
command to
initialize the source of truth, or create the directory structure manually.
Empty directories cannot be committed to a Git repository, so before you configure Config Sync, you must create configs and add them into your repository.
After you have created the source of truth and added configs to it, use the
nomos vet
command to verify the
structure of your source of truth and check the syntax and validity of your
configs.
Configure Config Sync to read from the source of truth
After you've created a source of truth and placed your configs into it, you can configure Config Sync to read from the source. After you've completed this step, Config Sync syncs configs from your source of truth to your clusters.
You configure the location of the source of truth when you install Config Sync, and you can edit Config Sync's configuration later. In addition to the location of the source of truth, you can specify a branch or subdirectory to watch, if the source has contents other than configs.
If you are using a hierarchical format, and
installing Config Sync manually with kubectl
,
don't place the Operator's config in the system/
directory, or any of the other reserved directories such as cluster/
or
namespaces/
. Placing the config in one of the reserved directories causes
nomos vet
to fail and logs an error such as
KNV1033: IllegalSystemResourcePlacementError,
KNV1038: IllegalKindInNamespacesError,
or KNV1039: IllegalKindInClusterError.
You can grant people access to a given product team's deployment source of truth. However, when you grant a person access to a deployment source of truth, that person is also granted the same RBAC as the reconciler running for that source of truth.
To configure authentication and authorization between Config Sync and
the source of truth, see the installation step about
configuring the git-creds
Secret.
Ignore object mutations
If you don't want Config Sync to maintain the state of the object in the
cluster after it exists, you can add the
client.lifecycle.config.k8s.io/mutation: ignore
annotation to the object that
you want Config Sync to ignore mutations in.
To use the annotation, you must enable the RootSync and RepoSync APIs.
The following example shows you how to add the annotation to an object:
metadata:
annotations:
client.lifecycle.config.k8s.io/mutation: ignore
You cannot manually modify this annotation on managed objects in the cluster.
What's next
- Learn how to manage namespaces and namespace-scoped objects.
- To learn how to publish an OCI image, see Sync OCI artifacts from Artifact Registry.
- To learn how to sync from a Helm chart, see Sync Helm charts from Artifact Registry.