Setting up Container Registry or Artifact Registry for GKE private clusters

This page describes how to configure DNS entries for using Container Registry or Artifact Registry with a Google Kubernetes Engine private cluster and VPC Service Controls.

These steps are only required if your GKE private cluster uses Artifact Registry or Container Registry for container storage, and you haven't already configured routing of the pkg.dev or gcr.io registry domains to restricted.googleapis.com.

While private clusters in service perimeters can use Container Registry or Artifact Registry without the restricted VIP, you should not do so. If the restricted VIP is not used, data can be exfiltrated from a supported service to an unsupported one.

Before you begin

Before you create a service perimeter, set up a new private cluster or identify the existing private clusters that you want to protect.

Also, you must permit egress to 199.36.153.4/30 on port 443. Normally, a VPC network has an implied rule that allows all egress traffic to any destination. However, if you have a rule denying such traffic, you must create an egress firewall rule to allow TCP traffic on port 443 to 199.36.153.4/30.

Configuring DNS

To support GKE private clusters that use Container Registry or Artifact Registry inside a service perimeter, you first need to configure your DNS server so requests to registry addresses resolve to restricted.googleapis.com, the restricted VIP. You can do so using Cloud DNS private DNS zones.

Create a managed private zone.
```
gcloud beta dns managed-zones create ZONE_NAME \
    --visibility=private \
    --networks=https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/networks/NETWORK \
    --description=DESCRIPTION \
    --dns-name=REGISTRY_DOMAIN \
    --project=PROJECT_ID
```
Where:
- ZONE_NAME is a name for the zone you are creating. For example, registry. This name will be used in each of the following steps.
- PROJECT_ID is the ID of the project that hosts your GKE private cluster.
- NETWORK is an optional list of names of the cluster network that you want to redirect requests from.
- DESCRIPTION is a human-readable description of the managed zone.
- REGISTRY_DOMAIN is the domain for your registry:
  - pkg.dev for Artifact Registry
  - gcr.io for Container Registry
Start a transaction.
```
gcloud dns record-sets transaction start \
  --zone=ZONE_NAME \
  --project=PROJECT_ID
```
Where:
- ZONE_NAME is the name of the zone you created in the first step.
- PROJECT_ID is the ID of the project that hosts your GKE private cluster.
Add a CNAME record for your registry.
```
gcloud dns record-sets transaction add \
  --name=*.REGISTRY_DOMAIN. \
  --type=CNAME REGISTRY_DOMAIN. \
  --zone=ZONE_NAME \
  --ttl=300 \
  --project=PROJECT_ID
```
Where:
- ZONE_NAME is the name of the zone you created in the first step.
- PROJECT_ID is the ID of the project that hosts your GKE private cluster.
- REGISTRY_DOMAIN is the domain for your registry:
  - pkg.dev for Artifact Registry
  - gcr.io for Container Registry
Add an A record for the restricted VIP.
```
gcloud dns record-sets transaction add \
  --name=REGISTRY_DOMAIN. \
  --type=A 199.36.153.4 199.36.153.5 199.36.153.6 199.36.153.7 \
  --zone=ZONE_NAME \
  --ttl=300 \
  --project=PROJECT_ID
```
Where:
- ZONE_NAME is the name of the zone you created in the first step.
- PROJECT_ID is the ID of the project that hosts your GKE private cluster.
- REGISTRY_DOMAIN is the domain for your registry:
  - pkg.dev for Artifact Registry
  - gcr.io for Container Registry
Execute the transaction.
```
gcloud dns record-sets transaction execute \
  --zone=ZONE_NAME \
  --project=PROJECT_ID
```
Where:
- ZONE_NAME is the name of the zone you created in the first step.
- PROJECT_ID is the ID of the project that hosts your GKE private cluster.

Configuring the service perimeter

After configuring the DNS records, either create a new service perimeter or update an existing perimeter, and then add the Container Registry or Artifact Registry service to the list of services you want to protect using the service perimeter.

In addition:

Add other supported services that you use with the registry to the service perimeter, such as Cloud Build, Container Analysis, and Binary Authorization.
For Container Registry, you must also add Cloud Storage to the service perimeter.

Verifying the perimeter works

After configuring the service perimeter, verify the configuration by deploying a sample container.

Create a deployment in your private cluster with the following command:

Artifact Registry

kubectl create deployment hello-server --image=us-docker.pkg.dev/google-samples/containers/gke/hello-app:1.0

Container Registry

kubectl create deployment hello-server --image=gcr.io/google-samples/hello-app:1.0

Check the status of the pod with the command:

kubectl get pods

The command returns a table similar to the following example. If the image was pulled and deployed successfully, the pod status is RUNNING.

NAME                            READY   STATUS    RESTARTS   AGE
hello-server-dbd86c8c4-h5wsf   1/1     Running   0          45s

You can use the kubectl describe pod command to see more details about the deployment. For the pod in the previous example, the command is:

kubectl describe pod hello-server-dbd86c8c4-h5wsf