This page describes how Config Sync applies configs to namespaces in your clusters in a hierarchy, based on the structure of the repo. You can also learn about Configuring namespaces and namespace-scoped objects.
How namespace inheritance works
One of the most powerful aspects of Config Sync is the ability to apply configs to groups of namespaces automatically, in all the clusters where those namespaces exist (or should exist), based on where the configs are located in the repo.
Config Sync introduces a notion of inheritance in the namespaces/
directory of the repo and all its subdirectories. Configs in other directories
in the repo, such as cluster/
, are not subject to inheritance.
In the repo, the namespaces/
directory can
contain two different types of subdirectories:
A namespace directory contains a config for a namespace. The name of the file containing the config is not important, but the config must have
kind: Namespace
. A namespace directory can also contain configs for other kinds of Kubernetes objects. A namespace directory cannot contain subdirectories. A namespace config represents an actual namespace in a cluster.An abstract namespace directory contains namespace directories. It can also contain configs for other Kubernetes objects, but it cannot directly contain a config for a namespace. An abstract namespace directory does not represent an object in a Kubernetes cluster, but its descendant namespace directories do.
If you forget to add a config for a namespace to a namespace directory, or you add a directory to a namespace subdirectory or add a config for a namespace to an abstract namespace directory, the result is error KNV1003: IllegalNamespaceSubdirectoryError
Configs in a namespace directory only apply to that namespace, while configs in an abstract namespace directory are applied to all of that abstract namespace's descendant namespace directories (or those descendant namespaces that match a config's NamespaceSelector, if one is present).
Because inheritance of a config in the namespaces/
directory is based largely
on its location within the directory tree in the repo, you can browse the repo
to understand which configs are being applied to a given namespace in a given
cluster.
The following diagram shows the way configs are inherited within the
namespaces/
directory of the
example repo.
The blue rectangles represent abstract namespace directories, and the orange
rectangles represent actual namespaces in Kubernetes.
For example, open the
viewers-rolebinding.yaml
file in your browser. It grants anyone in the system:serviceaccounts:audit
Group the view
ClusterRole in every namespace, managed by
Config Sync, of every enrolled cluster, because it exists in the
namespaces
directory itself.
Now open the namespaces/online/
directory in your browser. The online
directory is an abstract namespace
directory, because it doesn't have a config for a namespace. Click the
shipping-app-backend
directory. It's also an abstract namespace directory, and
contains two configs (pod-creator-rolebinding.yaml
and quota.yaml
). Each of
its three subdirectories is a namespace directory, because it contains a config
for a namespace. The name of the file is not significant, but this repo uses
that file name for all namespace configs, by convention. Each of those
namespaces inherits the pod-creator-rolebinding.yaml
and quota.yaml
configs
in the shipping-app-backend
abstract namespace directory.
The shipping-dev
namespace directory has an additional config for
the job-creators
RoleBinding, but the other two namespace directories do not
have this config, so they do not have that RoleBinding (unless someone creates
it manually).
Disallowed names in namespaces/
The following are reserved and cannot be used as either namespaces or abstract
namespace directories within the namespaces/
directory of the repo:
config-management-system
Example configs
Namespace configs
This config creates a namespace called audit
.
apiVersion: v1
kind: Namespace
metadata:
name: audit
When you create a namespace config, you can also add a label to the namespace. This can be useful in conjunction with NamespaceSelector.
The following config creates a namespace called shipping-prod
if it doesn't
already exist or it already exists without the configmanagement.gke.io/managed
label, and
assigns it the env: prod
. It also ensures that an annotation named audit
is set to true
for the namespace. If someone manually modifies or removes that
annotation, Config Sync quickly resets it to the value in the config.
apiVersion: v1
kind: Namespace
metadata:
name: shipping-prod
labels:
env: prod
annotations:
audit: "true"
ResourceQuota config
This example creates a ResourceQuota called quota
, which sets a hard limit of
1 Pod, 0.1 CPU (100 milli-CPUs), and 100 MiB of memory.
kind: ResourceQuota
apiVersion: v1
metadata:
name: quota
spec:
hard:
pods: "1"
cpu: "100m"
memory: 100Mi
If you place this config within a directory that applies to a specific
namespace (namespaces/[NAMESPACE_NAME]
), the config
applies only to that namespace. If you place this config within an
abstract namespace directory (namespaces/
) that contains
namespace descendants, a separate ResourceQuota is applied to each descendant
namespace.
Excluding namespaces from inheritance
Namespace selectors can be used to exempt particular namespaces from inheriting a resource in the tree.
The following example allows a properly annotated ResourceQuota
object in the
root /namespaces
directory to be inherited by every namespace except those
labeled quota-exempt: exempt
:
kind: NamespaceSelector
apiVersion: configmanagement.gke.io/v1
metadata:
name: excludes-exempt-namespaces
spec:
selector:
matchExpressions:
- key: quota-exempt
operator: NotIn
values:
- exempt
To learn more about NamesspaceSelectors
in Config Sync, see
Limiting which namespaces a config affects.
Effects of Git operations on namespaces
Git operations that create or delete namespace directories from within the
namespaces/
directory may cause different effects than you initially expect.
This section covers those interactions.
Creating a directory in namespaces/
When a valid namespaces/
hierarchy is committed to the repo,
Config Sync creates namespaces, and then creates Kubernetes objects in
those namespaces for each config that the namespace directory contains or
inherits.
Deleting a directory from namespaces/
Deleting a namespace directory is a destructive operation. The namespace is deleted, along with all its contents, on every cluster managed by Config Sync where the namespace exists.
If you delete an abstract namespace directory containing descendant namespace directories, all of those namespaces and their contents are deleted from every cluster managed by Config Sync.
Renaming a directory in namespaces/
Renaming a namespace directory is a deletion followed by a creation, and thus is also a destructive operation.
Renaming an abstract namespace directory has no externally-visible effect.
Moving a directory in namespaces/
Moving a namespace or an abstract namespace directory within namespaces/
does not delete the namespace or objects within it, except where the namespace
starts or stops inheriting a config from an abstract namespace directory, due
to a change in its hierarchy.
Integration with Hierarchy Controller
Hierarchy Controller has a very similar concept of namespace inheritance to that supported by abstract namespaces, as described in the documentation for Hierarchy Controller. However, they support some additional features such as self-service namespaces and hierarchical resource quotas.
Selecting related namespaces
There are times when you might want to apply policies to sets of namespaces that are related through a common ancestor. Hierarchy Controller supports this by using a concept known as tree labels, and these are also supported by abstract namespaces, even if Hierarchy Controller is not enabled.
Tree labels are Kubernetes labels that have the following format:
<namespace-name>.tree.hnc.x-k8s.io/depth: <depth>
These labels let you write namespace selectors that can, for example, be used as a part of a network policy to allow traffic within a subtree of related namespaces, but disallow traffic outside of that subtree.
We can illustrate this concept by returning to the diagram of the example repo.
As an example, the shipping-prod
namespace has the following tree
labels:
shipping-prod.tree.hnc.x-k8s.io/depth: 0
shipping-app-backend.tree.hnc.x-k8s.io/depth: 1
online.tree.hnc.x-k8s.io/depth: 2
root.tree.hnc.x-k8s.io/depth: 3
You can use kubectl
to inspect these relationships directly on the cluster,
without having access to the Git repository:
# View all descendants of 'root'
kubectl get namespaces -l 'root.tree.hnc.x-k8s.io/depth'
# View any immediate children of 'shipping-app-backend'
kubectl get namespaces -l 'shipping-app-backend.tree.hnc.x-k8s.io/depth=1'
Hierarchy Controller also propagates any tree labels from the abstract
namespaces to any descendant namespaces. For example, if you create a child
namespace of shipping-prod
as follows:
kubectl hns create shipping-prod-v1 -n shipping-prod
In this case, the shipping-prod-v1
namespace would include all the labels
from its parent, plus its own, with the depth suitably adjusted:
shipping-prod-v1.tree.hnc.x-k8s.io/depth: 0
shipping-prod.tree.hnc.x-k8s.io/depth: 1
shipping-app-backend.tree.hnc.x-k8s.io/depth: 2
online.tree.hnc.x-k8s.io/depth: 3
root.tree.hnc.x-k8s.io/depth: 4
What's next
- Learn how to manage namespaces and namespace-scoped objects