VPC Service Controls for Managed Anthos Service Mesh

Preview — VPC-SC

This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

Managed Anthos Service Mesh supports a preview of VPC Service Controls (VPC-SC) in the Regular and Stable channels, for GKE clusters with a release greater or equal to 1.22.1-gke.100. For the information about VPC-SC in the Rapid channel, see VPC Service Controls for managed Anthos Service Mesh GA

Before you begin

The VPC-SC org-policy and service perimeter are configured at the organization level. Ensure that you have been granted the proper roles for administering VPC-SC.

Configure the org-policy

If you provision a new managed Anthos Service Mesh control plane in the stable or regular channel, follow the instructions for preview below. Otherwise, follow the instructions for GA here.

1. Organization admins must configure the org-policy as described in this section, otherwise you risk accidentally provisioning control planes that do not enforce VPC-SC. Edit the org-policy for your organization or for individual projects, and set the constraint 'Allowed VPC-SC mode for ASM Managed Control Planes' to 'COMPATIBLE', using the Console or the following gcloud command:

   gcloud resource-manager org-policies allow \
     meshconfig.allowedVpcscModes COMPATIBLE \
     --project=PROJECT_ID

   Where:

   • PROJECT_ID is the ID of the project that you want to update.

   or

   gcloud resource-manager org-policies allow \
     meshconfig.allowedVpcscModes COMPATIBLE \
     --organization=ORGANIZATION_ID

   Where:

   • ORGANIZATION_ID is the ID of the organization that you want to update.

Set up your VPC-SC service perimeter

Create or update your service perimeter:

1. Add your cluster project(s) and fleet project to the service perimeter. Having a service mesh spread across multiple VPC-SC perimeters is not supported.

2. Add restricted services to the service perimeter.

   You must add specific services to the allowed and restricted services lists in the service perimeter, so that your Anthos Service Mesh cluster can access them. Access to these services is also restricted within your cluster's Virtual Private Cloud (VPC) network.

   Not adding these services may cause the Anthos Service Mesh installation to fail or to not function properly. For example, if you don't add the Mesh Configuration API to the service perimeter, the installation will fail and the workloads will not receive their Envoy configuration from the managed control plane.

   Console

   1. Follow the steps in Updating a service perimeter to edit the perimeter.
   2. Click on the Edit VPC Service Perimeter page.
   3. Under Restricted Services, Services to protect, click Add Services.
   4. On the Specify services to restrict dialog, click Filter services, and enter Mesh Configuration API.
   5. Select the service's checkbox.
   6. Click Add Mesh Configuration API.
   7. Repeat steps c - f, to add:
      • Cloud Service Mesh Certificate Authority API
      • GKE Hub API
      • Cloud IAM API
      • Cloud Monitoring API
      • Cloud Trace API
      • Cloud Monitoring API
      • Google Cloud Resource Manager API
      • Cloud Run API
      • Google Compute Engine API
      • Google Container Registry API
      • Artifact Registry API
      • Google Cloud Storage API
   8. Click Save.

   gcloud

   To update the list of restricted services, use the update command and specify the services to add as a comma-delimited list:

   gcloud access-context-manager perimeters update PERIMETER_NAME \
     --add-restricted-services=meshconfig.googleapis.com,meshca.googleapis.com,gkehub.googleapis.com,iam.googleapis.com,monitoring.googleapis.com,cloudtrace.googleapis.com,monitoring.googleapis.com,cloudresourcemanager.googleapis.com,run.googleapis.com,compute.googleapis.com,containerregistry.googleapis.com,artifactregistry.googleapis.com,storage.googleapis.com \
     --policy=POLICY_NAME

   Where:

   • PERIMETER_NAME is the name of the service perimeter that you want to update.

   • POLICY_NAME is the numeric name of your organization's access policy. For example, 330193482019.

3. Click on VPC Accessible services and set it to 'All restricted services', so that services restricted at the step above are still accessible from within the VPC-SC perimeter.

4. Unless you are installing Anthos Service Mesh from an in-perimeter network, add an ingress rule to allow the identity running the asmcli command access to the service perimeter.

   For more information, see Updating a service perimeter.

Install the managed Anthos Service Mesh in a VPC-SC perimeter

This section does not require organization admin privileges, but it requires compliance with the VPC-SC policy. You must use the additional flag --use_vpcsc during the installation, otherwise VPC Security Controls will not be fully enforced on the control plane. If you configured the meshconfig.allowedVpcscModes policy as advised, attempts to install without the --use-vpcsc flag will fail.

Follow the steps in Configure managed Anthos Service Mesh page. Then, verify the control plane has been successfully provisioned and there are no VPC-SC related errors.

Troubleshooting

Cannot create cluster with the latest GKE 1.22 image

There is a known issue preventing the creation of a cluster with the latest 1.22 image in a VPC-SC restricted environment. The workaround is to create this cluster first with the default GKE channel image, and then upgrade the image:

gcloud container clusters create CLUSTER \
  --region REGION \
  --release-channel=rapid \
  --workload-pool=PROJECT_ID.svc.id.goog \
  --project PROJECT_ID

gcloud container clusters upgrade CLUSTER \
  --region REGION \
  --master --cluster-version 1.22 \
  --project PROJECT_ID

Containers are not able to download their images.

This may happen if the images are located outside of the service perimeter. Either move the images to a bucket located inside the perimeter, or update the perimeter to add an Egress rule. Typically, the Egress rule may allow selected identities to access the Container Registry API, Artifact Registry API, and Cloud Storage API.

The Status field of the ControlPlaneRevision CRD displays VPC-SC errors

This may happen if you did not use the '--use_vpcsc' flag during installation, in which case, it is safe to re-run.

Otherwise, run this command to get more info about the error:

gcloud logging read --project=PROJECT_ID \
'protoPayload.metadata.@type=type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata'

Where:

• PROJECT_ID is the project ID of the project encountering errors.