Enabling Cloud Run on existing GKE clusters

This guide shows how to enable Cloud Run on existing GKE clusters.

Note that enabling Cloud Run for Anthos on Google Cloud installs Knative Serving into the cluster to manage your stateless workloads. For more information, see Architectural overview of Cloud Run for Anthos on Google Cloud.

Prerequisites

This page assumes that you have already set up your environment and have a cluster with the following minimum configuration:

  • Nodes with 2 vCPU
  • Scopes: https://www.googleapis.com/auth/logging.write, https://www.googleapis.com/auth/monitoring.write

Enabling an existing cluster for Cloud Run

You can use either the gcloud command line or the console to enable Cloud Run for a cluster:

Console

To enable an existing cluster for Cloud Run:

  1. Go to the Google Kubernetes Engine page in the Cloud Console:

    Go to Google Kubernetes Engine

  2. Click on the cluster you want to enable for Cloud Run.

  3. Click Edit

  4. Select Enable Cloud Run for Anthos on Google Cloud.

  5. Click Save. After the update completes, the cluster will support Cloud Run.

Command line

To enable an existing cluster for Cloud Run:

  1. Enable the cluster using the command:

    gcloud container clusters update \
    CLUSTER_NAME \
    --update-addons=CloudRun=ENABLED,HttpLoadBalancing=ENABLED \
    --zone=ZONE

    Replace

    • CLUSTER_NAME with the name you want for your cluster.
    • ZONE with the zone you are using for your cluster, for example, us-central1-a.
  2. Wait for the enabling to complete: upon success, you will see a message similar to Updating your-cluster-name...done..

Enabling deployments on a private cluster

You can skip the following instructions, if you're using Cloud Run for Anthos on a GKE cluster with the following versions:

  • 1.16.8-gke.7+
  • 1.15.11-gke.9+

To deploy a service to Cloud Run for Anthos on a private GKE cluster, you must allow TCP connections from master servers to nodes on port 8443 and manually specify port 8443 in your list of allowed TCP connections by editing the firewall rules in your project:

  1. View the cluster master's CIDR block and record the value in the masterIpv4CidrBlock field:

    gcloud container clusters describe CLUSTER_NAME
  2. View and record the value in the TARGET_TAGS field:

    gcloud compute firewall-rules list \
              --filter 'name~^gke-CLUSTER_NAME' \
               --format 'table(
                       name,
                       network,
                       direction,
                       sourceRanges.list():label=SRC_RANGES,
                       allowed[].map().firewall_rule().list():label=ALLOW,
                       targetTags.list():label=TARGET_TAGS
               )'
  3. Add a firewall rule using the values you recorded above:

    gcloud compute firewall-rules create FIREWALL_RULE_NAME \
         --action ALLOW \
         --direction INGRESS \
         --source-ranges masterIpv4CidrBlock \
         --rules tcp:8443 \
         --target-tags TARGET_TAGS

    For more information, see Creating firewall rules.

Enabling deployments for a private, internal network

Deploying services on an internal network is useful for enterprises that provide internal apps to their staff, and for services that are used by clients that run outside the Cloud Run for Anthos on Google Cloud cluster. This configuration allows other resources in your network to communicate with the service using a private, internal (RFC 1918) IP address.

You can deploy your Cloud Run for Anthos on Google Cloud services on an internal IP address in your VPC network by changing Istio's Ingress Gateway to use Internal TCP/UDP Load Balancing instead of Network Load Balancing. You must be an admin on your cluster to do this.

To configure the Ingress Gateway:

  1. Update the Istio Ingress Gateway to use Internal TCP/UDP Load Balancing:

    kubectl -n gke-system patch svc istio-ingress -p \
        '{"metadata":{"annotations":{"cloud.google.com/load-balancer-type":"Internal"}}}'
    

    It might take a few minutes for the change to take effect.

  2. Run the following command to poll your GKE cluster for change:

    kubectl -n gke-system get svc istio-ingress --watch
    
    1. Look for the value of EXTERNAL-IP to change to a private IP address.
    2. Press Ctrl+C to stop the polling when you see a private IP address in the EXTERNAL-IP field.

To verify internal connectivity after your changes:

  1. Deploy a service called sample to Cloud Run for Anthos on Google Cloud in the default namespace:

    gcloud run deploy sample \
    --image gcr.io/knative-samples/simple-api \
    --namespace default \
    --platform gke
    
  2. Create a Compute Engine virtual machine (VM) in the same zone as the GKE cluster:

    VM=cloudrun-gke-ilb-tutorial-vm
    
    gcloud compute instances create $VM
    
  3. Store the private IP address of the Istio Ingress Gateway in an environment variable called EXTERNAL_IP and a file called external-ip.txt:

    export EXTERNAL_IP=$(kubectl -n gke-system get svc istio-ingress \
        -o jsonpath='{.status.loadBalancer.ingress[0].ip}' | tee external-ip.txt)
    
  4. Copy the file containing the IP address to the VM:

    gcloud compute scp external-ip.txt $VM:~
    
  5. Connect to the VM using SSH:

    gcloud compute ssh $VM
    
  6. While in the SSH session, test the sample service:

    curl -s -w'\n' -H Host:sample.default.example.com $(cat external-ip.txt)
    

    The output is as follows:

    OK
    
  7. Leave the SSH session:

    exit
    

Enabling metrics on a cluster with Workload Identity

When enabling Workload Identity, Cloud Run for Anthos doesn't report certain metrics to Google Cloud's operations suite. To enable metrics, you need to manually set permissions to write metrics to Cloud Monitoring by granting the Monitoring Metric Writer role to the Google service account (GSA) associated with your Cloud Run for Anthos service.

Grant the Monitoring Metric Writer role permissions to your service's GSA:

gcloud projects add-iam-policy-binding PROJECT_ID 
--member=GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com
--role=roles/monitoring.metricWriter

Replace:

  • PROJECT_ID with the project ID for a cluster project that hosts your KSA.
  • GSA_PROJECT with the project ID for a GSA that's not in the cluster. You can use any GSA in your organization.

For more information, see Granting, changing, and revoking access to resources.

Enabling automatic Istio sidecar injection

To enable automatic Istio sidecar injection on your deployed service's namespace, you must use a separate Istio installation.

Developing in a multi-tenant setup

In multi-tenant use cases, you'll need to manage and deploy Cloud Run for Anthos services to a Google Kubernetes Engine cluster that is outside your current project. This section instructs you how to develop Cloud Run for Anthos on Google Cloud services in a multi-tenant cluster setup.

To manage and deploy Cloud Run for Anthos services to a Google Kubernetes Engine cluster outside your current project:

  1. Ensure you have read access to the Google Cloud project ID of the cluster you are deploying to.

  2. Update your local kubeconfig file with credentials for the target GKE cluster:

    gcloud container clusters get-credentials NAME \
    --region=REGION \
    --project=PROJECT-ID
    • REGION is the Compute Engine region of your target cluster.
    • PROJECT-ID is the project you have read access to.

    For more information, see the gcloud container clusters get-credentials command reference documentation.

  3. Use the gcloud command line to communicate with the GKE cluster by setting the default platform to kubernetes:

    gcloud config set run/platform kubernetes
    

You can now run commands on the target GKE cluster specified in your kubeconfig file.

For example, the following command will deploy a Cloud Run for Anthos service using a specified container image to the GKE cluster whose credentials are stored in the kubeconfig file:

gcloud run deploy SERVICE-NAME --image IMAGE-NAME

Disabling Cloud Run for a cluster

You can disable Cloud Run for a cluster at any time, using the console or the command line.

Console

To disable Cloud Run on a cluster:

  1. Go to the Google Kubernetes Engine page in the Cloud Console:

    Go to Google Kubernetes Engine

  2. Click the cluster where you want to disable Cloud Run.

  3. Select Disable Cloud Run for Anthos on Google Cloud.

  4. Optionally disable any addons you will not use in your cluster without Cloud Run. This may include Istio.

  5. Click Save.

Command line

To delete a cluster:

  1. Invoke the following command:

    gcloud container clusters update \
    CLUSTER_NAME \
    --update-addons=CloudRun=DISABLED \
    --zone=ZONE

    Replace

    • CLUSTER_NAME with the name you want for your cluster.
    • ZONE with the zone you are using for your cluster, for example, us-central1-a.
  2. Wait for the enabling to complete: upon success, you will see a message similar to Updating your-cluster-name...done..