Unable to apply changes in cluster due to webhook

Problem

Any Anthos Config Management cluster having a webhook whose sideEffects policy is not set to NoneOnDryRun, may be unable to update resources on the cluster.
If this is the case, the git-importer logs would show errors similar to:
KNV2010: unable to update resource: admission webhook "<webhook-name>" does not support dry run

 

Environment

  • Anthos Config Management enabled cluster with:
    • Anthos Config Management version 1.7 or above.
    • ValidatingWebhook or MutatingWebhook whose sideEffects policy is incorrect.

Solution

Perform one of the following steps depending on your  Anthos Config Management clusters needs:
  • Disable (delete) the webhook.
  • Update the sideEffects policy to NoneOnDryRun.
    • Before updating the sideEffects policy, check if the webhook is dryRun aware. That is if a request is sent with dryRun: true, the webhook will suppress the side effects. This may not always be the case if the webhook makes out-of-band changes (modify other resources based on a request).

Cause

The latest version of Anthos Config Management uses server-side apply using dry-run to verify the new changes that will be made to your resources. If your Webhook fails on requests with dry-run, then Anthos Config Management would not be able to verify and therefore update any resource that call the Webhook as all resources updates in the namespace would flow through your web.