Configuring only a subset of clusters

Normally, Config Sync applies a config to each enrolled cluster. If the config is within the namespaces/ subdirectory, Config Sync first creates the namespace within each cluster and then applies all inherited configs to that namespace.

A ClusterSelector is a special type of config that uses Kubernetes labelSelectors. You can use a ClusterSelector to limit which clusters a particular config applies to, based on the cluster's labels. You can also use ClusterSelectors to limit which clusters instantiate a namespace-scoped object.

A ClusterSelector config does not persist in a cluster. Instead, you reference it in another config using an annotation, and that config only applies to clusters that match the ClusterSelector.

Before you can use ClusterSelectors, each cluster must have a unique name. You set the cluster's name when you configure the Config Sync Operator. You can apply labels to each cluster manually, but we recommend using a Cluster config.

Next, you create the ClusterSelector, then reference it in another config.

Adding labels to a cluster

To use ClusterSelectors, each cluster must have a set of labels that can be selected. You can label clusters manually, but it is recommended that you use a Cluster config for each cluster. Cluster configs are stored in the clusterregistry/ directory of the repo.

The following example Cluster config declares that cluster-2 has the environment: prod label.

kind: Cluster
apiVersion: clusterregistry.k8s.io/v1alpha1
metadata:
  name: cluster-2
  labels:
    environment: prod

You can also apply annotations using a Cluster config.

Creating a ClusterSelector

A ClusterSelector selects only clusters with a given label or combination of labels. ClusterSelectors are stored in the top-level clusterregistry/ directory in the repo.

The following ClusterSelector selects only clusters with the environment: prod label.

kind: ClusterSelector
apiVersion: configmanagement.gke.io/v1
metadata:
  name: selector-env-prod
spec:
  selector:
    matchLabels:
      environment: prod

A ClusterSelector has no effect until you reference it in another config.

Referencing a ClusterSelector

To reference a ClusterSelector in another config, set the annotation configmanagement.gke.io/cluster-selector: CLUSTERSELECTOR-NAME.

The following config creates a ClusterRole called namespace-reader that defines a set of permissions for reading namespaces. This ClusterRole is only instantiated on clusters that match the selector-env-prod ClusterSelector.

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: namespace-reader
  annotations:
    configmanagement.gke.io/cluster-selector: selector-env-prod
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "watch", "list"]

Limiting the clusters a namespace-scoped config affects

By default, Config Sync applies configs inherited by a namespace to that namespace in each cluster where it exists. You can use a ClusterSelector to apply the config to only a subset of those clusters.

For example, you could configure clusters differently based on their geographic location or the geographic location of their clients. This could be useful for localization or for legal compliance. The examples in this section apply locale-specific configs to only clusters with the label location: france.

The following Cluster config adds an location: france label to a cluster called cluster-1:

kind: Cluster
apiVersion: clusterregistry.k8s.io/v1alpha1
metadata:
  name: cluster-1
  labels:
    location: france

You could label each relevant cluster manually, but this creates room for error and is not recommended.

Create a ClusterSelector config that references the labels you want to select. The following ClusterConfig selects the location: france label:

kind: ClusterSelector
apiVersion: configmanagement.gke.io/v1
metadata:
  name: selector-location-france
spec:
  selector:
    matchLabels:
      location: france

A ClusterSelector has no effect until you reference it in another config. The following RoleBinding config selects only clusters that match the selector-location-france ClusterSelector. This might be useful, for instance, if a compliance guideline only allowed auditors to audit information on clusters in their designated geographic region.

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: viewers
  annotations:
    configmanagement.gke.io/cluster-selector: selector-location-france
subjects:
- kind: Group
  name: system:serviceaccounts:audit
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: view
  apiGroup: rbac.authorization.k8s.io

If you place this config in an abstract namespace or a namespace directory, such as namespaces/online/shipping-app-backend/, it is applied to the namespaces that inherit it, but only on clusters with the location: france label.

What's next