Validating configs

In addition to running nomos vet manually or as a pre-commit hook locally, we recommend validating any configuration changes in your CI/CD pipeline. This guide shows how to validate configs with Cloud Build when using GKE clusters. The same setup works in any other container-based CI/CD system, such as CircleCI, with minimal changes.

Before you begin

To follow this guide, you must first complete the Anthos Config Management quickstart.

Configuring Cloud Build

Enabling the Cloud Build API

gcloud

To enable the Cloud Build API, run the following command:

gcloud services enable cloudbuild.googleapis.com

Console

Enable the Cloud Build API

Granting the Cloud Build service account permission to access your GKE cluster

gcloud

To add the Kubernetes Engine Developer role to the Cloud Build service account, run the following command:

PROJECT_ID=$(gcloud config get-value project)
PROJECT_NUM=$(gcloud projects list --filter="$PROJECT_ID" --format="value(PROJECT_NUMBER)")
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member=serviceAccount:$PROJECT_NUM@cloudbuild.gserviceaccount.com \
  --role=roles/container.developer

Console

  1. Open the IAM page in Cloud Console.

    Go to the IAM page

  2. In the member column find the Cloud Build service account:

    [PROJECT_NUMBER]@cloudbuild.gserviceaccount.com

  3. Click the pencil icon in that row.

  4. Click Add another role, select Kubernetes Engine Developer, and then click Save.

Creating a Cloud Build config

Create a Cloud Build config file and store it under the root directory of the repo containing your config files (for example, my-repo/cloudbuild.yaml):

steps:
- name: 'gcr.io/cloud-builders/kubectl'
  args: ['config', 'current-context']
  volumes:
  - name: 'kube'
    path: '/kube'
  env:
  - 'KUBECONFIG=/kube/config'
  - 'CLOUDSDK_COMPUTE_ZONE=[ZONE]'
  - 'CLOUDSDK_CONTAINER_CLUSTER=[CLUSTER_NAME]'
  - 'CLOUDSDK_CONTAINER_USE_APPLICATION_DEFAULT_CREDENTIALS=true'
- name: 'bash'
  args: ['chmod', '444', '/kube/config']
  volumes:
  - name: 'kube'
    path: '/kube'
- name: 'gcr.io/config-management-release/nomos:stable'
  args: ['nomos', 'vet', '--path', '/workspace/[POLICY_DIR]']
  volumes:
  - name: 'kube'
    path: '/kube'
  env:
  - 'KUBECONFIG=/kube/config'
  timeout: 30s

where ZONE is the zone where your cluster is running, CLUSTER_NAME is the name of your cluster, and POLICY_DIR is the path within the git repository that represents the top level of the repo to sync.

There are three steps in this configuration:

  1. Run kubectl config current-context to generate the kubeconfig file needed to authenticate to the my-cluster GKE cluster. The root user generates this file with restricted permissions.
  2. Run chmod 444 /kube/config to make this file readable in the next step.
  3. Run nomos vet on the Git repo which is automatically cloned in /workspace.

Creating a build trigger

The following example creates a trigger that runs for every commit to the master branch of a Cloud Source Repo. Use the Cloud Build configuration file from the previous step:

  1. Open the Triggers page in Cloud Console.

    Go to the triggers page

  2. Click Connect repository.

  3. Select GitHub (mirrored), and then click Continue.

  4. Select your repo, and then click Connect repository.

  5. Click Add trigger.

  6. Enter or select the corresponding entry in each field described in the following table:

    Field Entry
    Event Push to a branch
    Branch ^master$
    Build configuration file type Cloud Build configuration file (yaml or json)
    Cloud Build configuration file location / cloudbuild.yaml
  7. Click Create to save your build trigger.

Testing the build trigger

You can test the setup by manually running the trigger.

  1. Open the Triggers page in Cloud Console.

    Go to the triggers page

  2. Find the trigger you created, and then click Run trigger.

    The message "Build started on master branch SHOW" appears.

  3. Click SHOW.

    The Cloud Build steps appear green if set up correctly.

Invalid Cloud Build configurations

A trigger cannot run if the Cloud Build configuration file is invalid.

For example, update the Cloud Build config in your repo with the following file. Notice the invalid indentation on 6:

steps:
- name: 'gcr.io/cloud-builders/kubectl'
  args: ['config', 'current-context']
  volumes:
  - name: 'kube'
  path: '/kube'
  env:
  - 'KUBECONFIG=/kube/config'
  - 'CLOUDSDK_COMPUTE_ZONE=[ZONE]'
  - 'CLOUDSDK_CONTAINER_CLUSTER=[CLUSTER_NAME]'
  - 'CLOUDSDK_CONTAINER_USE_APPLICATION_DEFAULT_CREDENTIALS=true'
- name: 'bash'
  args: ['chmod', '444', '/kube/config']
  volumes:
  - name: 'kube'
    path: '/kube'
- name: 'gcr.io/nomos-release/nomos:stable'
  args: ['nomos', 'vet', '--path', '/workspace/[POLICY_DIR]']
  volumes:
  - name: 'kube'
    path: '/kube'
  env:
  - 'KUBECONFIG=/kube/config'
  timeout: 30s

If you manually run the trigger again, you receive the following error message because path: on line 6 is not correctly indented:

Failed to trigger build: failed unmarshalling build config cloudbuild.yaml:
unknown field "path" in cloudbuild_go_proto.BuildStep.

To fix this config, indent path: on line 6 to the same level as name: on line 5. For more information about the structure of a Cloud Build configuration file, see Creating a basic Cloud Build config.

What's next