You're viewing documentation for Anthos Config Management 1.8. You can continue using this version, or use the current version.

Managing existing cluster objects

When Config Sync manages a cluster object, it watches the object and the set of configs in the repo that affect the object, and ensures that they are in sync. This topic describes how to start managing an existing object and how to stop managing an object that is currently managed without deleting the object.

Overview

An object in a cluster is managed by Config Sync if it has the configmanagement.gke.io/managed: enabled annotation, and its configsync.gke.io/resource-id annotation, which tracks the group, kind, namespace, and name information of the object, is correct.

The format of the configsync.gke.io/resource-id annotation is GROUP_KIND_NAMESPACE_NAME for a namespace-scoped object, and GROUP_KIND_NAME for a cluster-scoped object.

The following flow chart describes some situations that cause an object to become managed or unmanaged:

How to manage or unmanage a Kubernetes object using Config Sync

The chart contains three separate flows: beginning to manage an object, stopping managing an object, and deleting a managed object.

  1. I want to manage an object. a. Does the object have a config in the repo?
    • No: Create a config for the object. Config Sync sets the annotation configmanagement.gke.io/managed: enabled, sets the configsync.gke.io/resource-id annotation to match the object, and begins managing the object.
    • Yes: Does the config set the annotation configmanagement.gke.io/managed: disabled?
      • No: The object is managed by default.
      • Yes: Edit the config to remove the configmanagement.gke.io/managed: disabled annotation. When the change is pushed to the source repo, Config Sync notices the change, applies the annotation configmanagement.gke.io/managed: enabled and the annotation configsync.gke.io/resource-id, and applies the config.
  2. I want to stop managing an object but not delete it.
    • Edit the config for the object in the repo, and set the annotation configmanagement.gke.io/managed: disabled. When the config change is detected, Config Sync stops managing the object.
  3. I want to stop managing an object and delete it.
    • Delete the object's config from the repo. When you delete a config for an object that was previously managed, Config Sync deletes the object from all clusters or namespaces the config applies to.

In addition to the configmanagement.gke.io/managed: enabled annotation and the configsync.gke.io/resource-id annotation, Config Sync applies the label app.kubernetes.io/managed-by: configmanagement.gke.io to all objects it manages. This label allows you to easily list all objects by Config Sync.

Why not apply the annotation manually?

Config Sync uses a declarative model to apply configuration changes to your clusters by reading your desired configuration from your repo. If you attempt to apply the annotation manually (either using the kubectl command or the Kubernetes API), Config Sync overrides the manual automatically with the contents of your repo.

Before you begin

The following examples build upon the quickstart. Before you begin the following steps, follow the quickstart and complete all of the steps before Examine your cluster and repo.

List all managed objects

To list all objects managed by Config Sync on a given cluster or namespace, use a label selector like the following:

kubectl get object-type -n namespace -l "app.kubernetes.io/managed-by=configmanagement.gke.io"

To list all objects not managed by Config Sync, use a label selector like this:

kubectl get object-type -n namespace -l "app.kubernetes.io/managed-by!=configmanagement.gke.io"

For example, this command lists RoleBindings in the gamestore namespace that are managed by Config Sync:

kubectl get rolebindings -n gamestore \
    -l "app.kubernetes.io/managed-by=configmanagement.gke.io"

Example output: ```none NAME ROLE AGE configsync.gke.io:ns-reconciler ClusterRole/configsync.gke.io:ns-reconciler 34h gamestore-admin ClusterRole/admin 34h gamestore-webstore-admin ClusterRole/webstore-admin 34h


This command lists RoleBindings in the `kube-system` namespace that are **not**
managed by Config Sync:

<pre class="devsite-click-to-copy">
kubectl get rolebindings -n kube-system \
    -l "app.kubernetes.io/managed-by!=configmanagement.gke.io"
</pre>

Example output:
```none
NAME                                             AGE
fluentd-gcp-scaler-binding                       2d21h
gce:cloud-provider                               2d21h
heapster-binding                                 2d21h
metrics-server-auth-reader                       2d21h
system::leader-locking-kube-controller-manager   2d21h
system::leader-locking-kube-scheduler            2d21h
system:controller:bootstrap-signer               2d21h
system:controller:cloud-provider                 2d21h
system:controller:token-cleaner                  2d21h

Start managing an existing object

You can create a config for an existing Kubernetes object, such as a namespace that already exists in your cluster before you install Config Sync. However, this config is ignored unless the object has the annotation configmanagement.gke.io/managed: enabled and has the correct configsync.gke.io/resource-id annotation. For an existing object, you need to apply the annotation manually.

For namespaces specifically, Config Sync does apply configs that create new objects within an unannotated namespace, and applies the configmanagement.gke.io/managed: enabled and configsync.gke.io/resource-id annotations to those objects. However, Config Sync refuses to modify or remove any unannotated cluster-scoped object from a cluster. This is illustrated in the diagram in Working with configs over time.

The following example demonstrates how to manage an existing Role object. First, you create a Role manually, then begin managing it with Config Sync.

  1. Create the myrole Role in the gamestore namespace:

    kubectl create role -n gamestore myrole --verb=get --resource=pods
  2. View the permissions granted by the myrole Role:

    kubectl describe role -n gamestore myrole
    Name:         myrole
    Labels:       <none>
    Annotations:  <none>
    PolicyRule:
      Resources  Non-Resource URLs  Resource Names  Verbs
      ---------  -----------------  --------------  -----
      pods       []                 []              [get]
    

    The Role only has permission to get Pods.

  3. At this point, the Role exists in the cluster, but Config Sync does not know about it.

    1. In a terminal, go to the local clone of your repo.
    2. Use the following command to create a YAML manifest for myrole and save the manifest to a new file called gamestore-myrole.yaml.

      kubectl get role myrole -n gamestore -o yaml > gamestore-myrole.yaml
      
    3. Edit the gamestore-myrole.yaml file.

      1. Remove all fields under the metadata key except for name and namespace.
      2. Add the list verb after get in the rules.verbs list field.

      Save the changes. The resulting file has the following contents:

      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        name: myrole
        namespace: gamestore
      rules:
      - apiGroups:
        - ""
        resources:
        - pods
        verbs:
        - get
        - list
      
    4. Commit the change to the repo.

    5. Wait a few moments for the Config Management Operator to notice the commit. To verify that the myrole Role is now managed by Config Sync, run kubectl describe again.

      kubectl describe role myrole -n gamestore
      

Notice the annotation configmanagement.gke.io/managed: enabled, which indicates that the object is managed by Config Sync, and the annotation configsync.gke.io/resource-id, which tracks the group, kind, namespace, and name information. Also notice the annotations that show the path and file name in the repo that caused the most recent configuration change to the object, and the Git hash that represents the commit.

Name:         myrole
Labels:       app.kubernetes.io/managed-by=configmanagement.gke.io
              configsync.gke.io/declared-version=v1
Annotations:  config.k8s.io/owning-inventory: config-management-system_root-sync
              configmanagement.gke.io/cluster-name: my-cluster
              configmanagement.gke.io/managed: enabled
              configmanagement.gke.io/source-path: quickstart/multirepo/root/gamestore-myrole.yaml
              configmanagement.gke.io/token: 747b843a7ddbd945c0616034a935cf648b58e7b5
              configsync.gke.io/declared-fields: {"f:rules":{}}
              configsync.gke.io/git-context: {"repo":"https://github.com/GoogleCloudPlatform/anthos-config-management-samples","branch":"main","rev":"HEAD"}
              configsync.gke.io/manager: :root
              configsync.gke.io/resource-id: rbac.authorization.k8s.io_role_gamestore_myrole
PolicyRule:
  Resources  Non-Resource URLs  Resource Names  Verbs
  ---------  -----------------  --------------  -----
  pods       []                 []              [get list]

Stop managing a managed object

This example shows how to stop managing an object that Config Sync is currently managing, such as the myrole Role in Start managing an existing object.

  1. Edit the quickstart/multirepo/root/rolebinding-gamestore-webstore-admin.yaml file in the local clone of your repo, and add an annotations: section that matches the bold text below:

     kind: RoleBinding
     apiVersion: rbac.authorization.k8s.io/v1
     metadata:
       annotations:
         configmanagement.gke.io/managed: disabled
       name: gamestore-webstore-admin
       namespace: gamestore
     subjects:
     - kind: ServiceAccount
       name: ns-reconciler-gamestore
       namespace: config-management-system
     roleRef:
       kind: ClusterRole
       name: webstore-admin
       apiGroup: rbac.authorization.k8s.io
    

    Save the file.

  2. Create a Git commit with your changes, and push the commit to your repo.

  3. Wait a few moments for Config Sync to notice and apply the new commit.

  4. Use the following command to verify the annotations and labels of the gamestore-webstore-admin RoleBinding are both empty. Config Sync does not set the configmanagement.gke.io/managed annotation to disabled on the object.

    kubectl get rolebinding gamestore-webstore-admin -n gamestore -o yaml
    
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      annotations:
      name: gamestore-webstore-admin
      namespace: gamestore
    subjects:
    - kind: ServiceAccount
      name: ns-reconciler-gamestore
      namespace: config-management-system
    roleRef:
      kind: ClusterRole
      name: webstore-admin
      apiGroup: rbac.authorization.k8s.io
    

After verifying that the object is now disabled, you could remove the config from your repository and verify that the now-unmanaged object is not deleted from the namespace. If you want to manage the object again, you must add its config back into your repository. Because of this, you might want to unmanage objects and leave their configs in the repo.

Now that the object is not managed, it is not created or re-created on new or existing clusters, and it is not removed even if it exists. To resume managing an object that you previously stopped managing, see the next example, Resume managing a previously unmanaged object.

Resume managing a previously unmanaged object

This example shows how to resume managing an object you previously removed from management, as in Stop managing an existing object. It assumes that you have not removed the config for the gamestore-webstore-admin RoleBinding.

  1. If you deleted the gamestore-webstore-admin RoleBinding from your repo in the last commit, perform the following steps.

    1. Use git revert to revert the last commit:

      git revert HEAD~1
      

      You are asked to confirm the revert operation.

    2. Push the revert commit to your repo.

      git push
      
  2. Edit the quickstart/multirepo/root/rolebinding-gamestore-webstore-admin.yaml file in the local clone of your repo and remove the configmanagement.gke.io/managed: disabled annotation. Save the file.

  3. Commit and push your change. Config Sync does the following:

    • Notices the change
    • Applies the configmanagement.gke.io/managed: enabled annotation and the configsync.gke.io/resource-id annotation; the object is now managed.
    • Applies the config, as would happen with any managed object.
  4. To verify that the object is now managed, list its annotations:

    kubectl get rolebinding gamestore-webstore-admin -n gamestore -o yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      annotations:
        configmanagement.gke.io/cluster-name: my-cluster
        configmanagement.gke.io/managed: enabled
        configsync.gke.io/resource-id: rbac.authorization.k8s.io_rolebinding_gamestore_gamestore-webstore-admin
    ...
    

Stop managing a namespace

You can stop managing a namespace in the same way as you stop managing any type of object. If you want to stop managing other resources within the namespace, perform the following steps:

  1. Add the configmanagement.gke.io/managed:disabled annotation to the namespace config and all configs in the same namespace.

  2. Commit and push your changes to the repo. Wait for the Operator to sync with the repo.

  3. Delete the unmanaged resources from the repository.

If configs for managed configs exist within an unmanaged namespace directory, the reconciler logs errors, but other configs continue to sync normally.

Prevent deletion for Kubernetes objects

After you remove a Kubernetes object from a Git repository that is managed by Config Sync, this object is also deleted from the cluster when the new commit is synced to the cluster.

If you want to prevent deleting the cluster object when its configuration is removed from the Git repository, you can take the following steps:

  1. Add the annotation client.lifecycle.config.k8s.io/deletion: detach to the object configuration in Git repository.

  2. Commit and push the change in Git repository.

  3. Wait for the change being synced to the cluster.

What's next