You're viewing documentation for Anthos Config Management 1.8. You can continue using this version, or use the current version.

Set up Config Controller

This page shows you how to set up Config Controller. Config Controller is a hosted service to provision and orchestrate Anthos and Google Cloud resources. It offers an API endpoint that can provision, actuate, and orchestrate Google Cloud resources as part of Anthos Config Management. To learn more about Config Controller, see Config Controller overview.

Before you begin

Before setting up Config Controller, you need to complete the following steps:

  1. Install and initialize the Cloud SDK, which provides the gcloud, kubectl, and nomos commands used in these instructions. If you use Cloud Shell, the Cloud SDK comes pre-installed.

    kubectl is not installed by default by the Cloud SDK. To install kubectl, run the following command:

    gcloud components install kubectl
    
  2. Create or allow creating a default network. Config Controller depends on it. To create a default network, run the following command:

    gcloud compute networks create default --subnet-mode=auto
    

Set up Config Controller

Create a Config Controller with the following gcloud tool commands:

  1. Enable the services on your project to use the subsequent gcloud tool commands:

    gcloud services enable krmapihosting.googleapis.com \
        container.googleapis.com \
        cloudresourcemanager.googleapis.com
    
  2. Create your Config Controller. This operation can take over 15 minutes.

    gcloud alpha anthos config controller create CONFIG_CONTROLLER_NAME \
        --location=us-central1
    

    Replace CONFIG_CONTROLLER_NAME with the name that you want to give your controller.

  3. Once your instance is created, it appears in the list of instances. To view the list of instances, run the following command:

    gcloud alpha anthos config controller list --location=us-central1
    
  4. Authenticate with the instance to apply manifests:

    gcloud alpha anthos config controller get-credentials CONFIG_CONTROLLER_NAME \
        --location us-central1
    
  5. Give Config Controller permission to manage Google Cloud resources in the project:

    export PROJECT_ID=$(gcloud config get-value project)
    export SA_EMAIL="$(kubectl get ConfigConnectorContext -n config-control \
        -o jsonpath='{.items[0].spec.googleServiceAccount}' 2> /dev/null)"
    gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
        --member "serviceAccount:${SA_EMAIL}" \
        --role "roles/owner" \
        --project "${PROJECT_ID}"
    

Manage Google Cloud resources with Config Controller

Once you finish setting up Config Controller, you are ready to use Config Controller to manage Google Cloud resources.

In this example, you create Cloud Source Repositories that can be used in the Set up GitOps section.

  1. Create a file named service.yaml and copy the following YAML file into it:

    # service.yaml
    
    apiVersion: serviceusage.cnrm.cloud.google.com/v1beta1
    kind: Service
    metadata:
      name: sourcerepo.googleapis.com
      namespace: config-control
    
  2. Apply the manifest and wait to get Cloud Source Repositories enabled:

    kubectl apply -f service.yaml
    kubectl wait -f service.yaml --for=condition=Ready
    
  3. Create a file name repo.yaml and copy the following YAML file into it:

    # repo.yaml
    
    apiVersion: sourcerepo.cnrm.cloud.google.com/v1beta1
    kind: SourceRepoRepository
    metadata:
      name: REPO_NAME
      namespace: config-control
    

    Replace REPO_NAME with the name of the Cloud Source Repositories that you want to give.

  4. Apply the manifest to create the repository:

    kubectl apply -f repo.yaml
    

Set up GitOps

You can have configurations in your Git repository synced to your Config Controller using Config Sync. In the example in this secton, you use the Cloud Source Repositories you created in the preceding section.

  1. Create a file named gitops-iam.yaml and copy the following YAML file into it:

    # gitops-iam.yaml
    
    apiVersion: iam.cnrm.cloud.google.com/v1beta1
    kind: IAMServiceAccount
    metadata:
      name: config-sync-sa
      namespace: config-control
    spec:
      displayName: ConfigSync
    
    ---
    
    apiVersion: iam.cnrm.cloud.google.com/v1beta1
    kind: IAMPolicyMember
    metadata:
      name: config-sync-wi
      namespace: config-control
    spec:
      member: serviceAccount:PROJECT_ID.svc.id.goog[config-management-system/importer]
      role: roles/iam.workloadIdentityUser
      resourceRef:
        apiVersion: iam.cnrm.cloud.google.com/v1beta1
        kind: IAMServiceAccount
        name: config-sync-sa
    
    ---
    
    apiVersion: iam.cnrm.cloud.google.com/v1beta1
    kind: IAMPolicyMember
    metadata:
      name: allow-configsync-sa-read-csr
      namespace: config-control
    spec:
      member: serviceAccount:config-sync-sa@PROJECT_ID.iam.gserviceaccount.com
      role: roles/source.reader
      resourceRef:
        apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
        kind: Project
        external: projects/PROJECT_ID
    

    Replace PROJECT_ID with the project ID where the Config Controller is running.

  2. Apply the manifest to give Config Sync access to the repository.

    kubectl apply -f gitops-iam.yaml
    
  3. To have configurations in your repository synced to your Config Controller using Config Sync, create a file named config-management.yaml and paste the following text into it:

    # config-management.yaml
    
    apiVersion: configmanagement.gke.io/v1
    kind: ConfigManagement
    metadata:
      name: config-management
    spec:
      enableMultiRepo: false
      clusterName: krmapihost-CONFIG_CONTROLLER_NAME
      git:
        policyDir: DIRECTORY
        gcpServiceAccountEmail: config-sync-sa@PROJECT_ID.iam.gserviceaccount.com
        secretType: gcpserviceaccount
        syncBranch: BRANCH
        syncRepo: REPO
      sourceFormat: unstructured
    

    Replace the following:

    • CONFIG_CONTROLLER_NAME with the name of your Config Controller.
    • DIRECTORY: the path in the Git repository to the root directory that contains the configuration that you want to sync to. The default is the root directory of the repository.
    • PROJECT_ID: the project ID.
    • BRANCH: the branch of the repository to sync from. The default is the main (master) branch.
    • REPO: the URL of the Git repository to use as the source of truth. You can enter URLs using the HTTPS protocol.

    For more detailed information on configuring Config Sync, refer to Granting Config Sync read-only access to Git.

  4. Apply the configuration with the kubectl apply command:

    kubectl apply -f config-management.yaml
    

Verify success

You can verify the initial set up by the following steps:

  1. Check that all controllers were set up successfully on your Config Controller:

    kubectl wait pod --all --all-namespaces --for=condition=Ready
    
  2. Verify the Git repository is synchronized to Config Controller with Config Sync by using the nomos command:

    nomos status --contexts $(kubectl config current-context)
    

Delete your Config Controller

If you decide to stop using Config Controller, you should clean up all resources created. You should first remove resources from the Config Controller before deleting the Config Controller itself.

Deleting your Config Controller without first deleting the provisioned resources leaves the resources in an abandoned state. The resources still exist in Google Cloud (and incur billing charges), but are not managed from declarative configuration.

Once all your resources have been deleted, you can delete your Config Controller using gcloud tool:

gcloud alpha anthos config controller delete --location=us-central1 CONFIG_CONTROLLER_NAME

Troubleshooting

Default network

When creating your Config Controller, you might receive an error about the default network not being available:

Error 400: Project \"PROJECT_ID\" has no network named \"default\"., badRequest\n\n  on main.tf line 35, in resource \"google_container_cluster\" \"acp_cluster\"

This error occurs because Config Controller depends on the default network in Google Cloud. To resolve this you should create a new default network:

gcloud compute networks create default --subnet-mode=auto

Alternatively you can select a different network using the --network flag in gcloud alpha anthos config controller create command.

Control plane IPv4 CIDR

Config Controller creation uses a default subnet of 172.16.0.128/28 for the control plane IPv4 CIDR. If there is a conflict in the IPv4 CIDR block, Config Controller creation fails with this error:

Cloud SSA\n\nError: Error waiting for creating GKE cluster: Invalid value for field PrivateClusterConfig.MasterIpv4CidrBlock: 172.16.0.128/28 conflicts with an existing subnet in one of the peered VPCs.

If you see this error, select a different private IPv4 CIDR and use it using the --master-ipv4-cidr-block flag in gcloud alpha anthos config controller create command.

To find IPv4 CIDR blocks that are already in use:

  1. You find the name of peering by using the following command:

    gcloud compute networks peerings list --network=NETWORK
    

    Replace NETWORK with the name of network you want to look up.

    You get a response like the following example:

    NAME                                     NETWORK  PEER_PROJECT               PEER_NETWORK                            PEER_MTU  IMPORT_CUSTOM_ROUTES  EXPORT_CUSTOM_ROUTES  STATE   STATE_DETAILS
    gke-n210ce17a4dd120e16b6-7ebf-959a-peer  default  gke-prod-us-central1-59d2  gke-n210ce17a4dd120e16b6-7ebf-0c27-net            False                 False                 ACTIVE  [2021-06-08T13:22:07.596-07:00]: Connected.
    
  2. To show the IPv4 CIDR being used by the peering, use the following command:

    gcloud compute networks peerings list-routes PEERING_NAME \
        --direction=INCOMING \
        --network=NETWORK \
        --region=us-central1
    

    Replace the following:

    • PEERING_NAME with the name of peering you want to look up.
    • NETWORK with the name of network you want to look up.

Deployment and Config Sync

The configurations in your Git repository are synced to your Config Controller using Config Sync. You can check for errors in this sync process by using the nomos command:

nomos status  --contexts $(kubectl config current-context)

Config Connector resources

Immutable fields and resources

Some fields on the underlying Google Cloud resources are immutable, such as project IDs or the name of your VPC network. Config Connector blocks edits to such fields and is unable to actuate changes. If you want to edit one of these immutable fields, you must first delete the original resource (through Git) before re-adding it with the new preferred values.

Stuck resources

In some cases, resources might fail to delete correctly (as reported by nomos status). This can be fixed by removing the finalizers on the resource then deleting the resource manually. For example, to delete an IAMPolicyMember that is stuck, run the following command:

kubectl patch IAMPolicyMember logging-sa-iam-permissions -p '{"metadata":{"finalizers":[]}}' --type=merge -n config-control
kubectl delete IAMPolicyMember logging-sa-iam-permissions -n config-control