Troubleshoot Config Controller

This page shows you how to resolve issues with Config Controller.

Default network not available

When creating your Config Controller, you might receive an error about the default network not being available:

Error 400: Project \"PROJECT_ID\" has no network named \"default\"., badRequest\n\n  on main.tf line 35, in resource \"google_container_cluster\" \"acp_cluster\"

This error occurs because Config Controller depends on the default network in Google Cloud. To resolve this you should create a new default network:

gcloud compute networks create default --subnet-mode=auto

Note: You must enable the automatic subnets on the default network for this command to work.

Alternatively you can select a different network using the --network flag in gcloud anthos config controller create command.

Invalid value for MasterIpv4CidrBlock

Config Controller creation uses a default subnet of 172.16.0.128/28 for the control plane IPv4 CIDR. If there is a conflict in the IPv4 CIDR block, Config Controller creation fails with this error:

Cloud SSA\n\nError: Error waiting for creating GKE cluster: Invalid value for field PrivateClusterConfig.MasterIpv4CidrBlock: 172.16.0.128/28 conflicts with an existing subnet in one of the peered VPCs.

If you see this error, select a different private IPv4 CIDR and use it using the --master-ipv4-cidr-block flag in the gcloud config controller create command.

To find IPv4 CIDR blocks that are already in use, complete the following steps:

1. Find the name of the peering:

   gcloud compute networks peerings list --network=NETWORK
   

   Replace NETWORK with the name of the network that you want to look up.

   The output is similar to the following:

   NAME                                     NETWORK  PEER_PROJECT               PEER_NETWORK                            PEER_MTU  IMPORT_CUSTOM_ROUTES  EXPORT_CUSTOM_ROUTES  STATE   STATE_DETAILS
   gke-n210ce17a4dd120e16b6-7ebf-959a-peer  default  gke-prod-us-central1-59d2  gke-n210ce17a4dd120e16b6-7ebf-0c27-net            False                 False                 ACTIVE  [2021-06-08T13:22:07.596-07:00]: Connected.
   

2. Show the IPv4 CIDR being used by the peering:

   gcloud compute networks peerings list-routes PEERING_NAME \
       --direction=INCOMING \
       --network=NETWORK \
       --region=us-central1
   

   Replace the following:

   • PEERING_NAME: the name of peering you want to look up
   • NETWORK: the name of network you want to look up
   Note: You must use us-central1 as your region.

Sync errors

The configurations in your Git repository are synced to your Config Controller using Config Sync. You can check for errors in this sync process by using the nomos status command:

nomos status  --contexts $(kubectl config current-context)

Troubleshoot Config Connector resources

Immutable fields and resources

Some fields on the underlying Google Cloud resources are immutable, such as project IDs or the name of your VPC network. Config Connector blocks edits to such fields and is unable to actuate changes. If you want to edit one of these immutable fields, you must first delete the original resource (through Git) before re-adding it with the new preferred values.

Stuck resources

In some cases, resources might fail to delete correctly (as reported by nomos status). This can be fixed by removing the finalizers on the resource then deleting the resource manually.

For example, to delete an IAMPolicyMember that is stuck, run the following command:

kubectl patch IAMPolicyMember logging-sa-iam-permissions -p '{"metadata":{"finalizers":[]}}' --type=merge -n config-control
kubectl delete IAMPolicyMember logging-sa-iam-permissions -n config-control