Setting up Cloud Run for Anthos on Google Cloud

This guide shows how to set up a new Google Kubernetes Engine cluster with Cloud Run for Anthos on Google Cloud enabled. Because you can use either the Cloud Console or the gcloud command line, the instructions cover both of these. If you are enabling Cloud Run on an already existing cluster, refer to Enabling Cloud Run for Anthos on Google Cloud on existing clusters.

Note that enabling Cloud Run for Anthos on Google Cloud installs Istio and Knative Serving into the cluster to connect and manage your stateless workloads. For more information, see Architectural overview of Cloud Run for Anthos on Google Cloud.

Prerequisites

  1. You need an Anthos subscription. A free trial is available until March 31, 2021. Learn more about pricing.
  2. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  3. In the Cloud Console, on the project selector page, select or create a Cloud project.

    Go to the project selector page

  4. Verifica che la fatturazione sia attivata per il tuo progetto.

    scopri come attivare la fatturazione

Setting up gcloud

Although you can use either the Cloud Console or the gcloud command line to use Cloud Run for Anthos on Google Cloud, you may need to use the gcloud command line for some tasks.

To set up the gcloud command line for Cloud Run for Anthos on Google Cloud:

  1. Install and initialize the Cloud SDK.

  2. You should set your default project setting for gcloud to the one you just created:

    gcloud config set project PROJECT-ID

    Replace PROJECT-ID with the project ID of the project you created.

  3. Set zone to the desired zone for the new cluster. You can use any zone where GKE is supported, for example:

    gcloud config set compute/zone ZONE

    Replace ZONE with your zone.

  4. Enable the following APIs for the project, which are needed to create a cluster, build and publish a container into the Google Container registry:

    gcloud services enable container.googleapis.com containerregistry.googleapis.com cloudbuild.googleapis.com
  5. Update installed gcloud components:

    gcloud components update
  6. Install the kubectl command-line tool:

    gcloud components install kubectl

Creating a cluster with Cloud Run enabled

These instructions create a cluster with this configuration:

  • Cloud Run for Anthos on Google Cloud enabled
  • Kubernetes version: see Available GKE versions
  • Nodes with 2 vCPU

These are the recommended settings for a new cluster.

You can use either the gcloud command line or the console to create a cluster. Click the appropriate tab for instructions.

Console

To create a cluster and enable it for Cloud Run for Anthos on Google Cloud:

  1. Go to the Google Kubernetes Engine page in the Cloud Console:

    Go to Google Kubernetes Engine

  2. Click Create cluster to open the Create a Kubernetes cluster page.

  3. In the Cluster basics section, complete the following:

    • Enter the Name for your cluster.
    • Choose either Zonal or regional for the location type: either will work with Cloud Run for Anthos on Google Cloud. Zonal clusters are less expensive, but will incur downtime during control plane (master) upgrades.
    • Select a zone or region for the cluster, depending on your choice in the previous step. Choose a zone or region close to you, for example, us-central1-a.
    • From the dropdown list, select one of the available versions as the control plane cluster version.
  4. In the Features section, select the Enable Cloud Run for Anthos checkbox.

  5. Click Create to create and provision the cluster with the configuration you just completed. It may take a few moments for this process to finish.

Command line

To create a new cluster that enables Cloud Run for Anthos on Google Cloud:

  1. Create a new cluster:

    gcloud container clusters create CLUSTER-NAME \
    --zone=ZONE \
    --addons=HttpLoadBalancing,CloudRun \
    --machine-type=n1-standard-2 \
    --num-nodes=3 \
    --cluster-version=GKE-VERSION \
    --enable-stackdriver-kubernetes

    Note that these instructions will not enable cluster autoscaling to resize clusters for demand, Cloud Run for Anthos on Google Cloud will automatically scale instances within the cluster.

  2. Wait for the cluster creation to complete.

Creating a private cluster with Cloud Run enabled

By default, the cluster you've created above can configure access from its pods to public networks. If you'd like to create a Cloud Run-enabled private cluster that isolates pods, nodes, and workloads from having connectivity to public networks, see the following instructions; otherwise, skip to the next section.

External clients can still call the IP address of your private cluster's external load balancer. To create a cluster that will never be exposed to the public, see Setting up a private, internal network.

  1. Create a private cluster with at least 4 vCPUs.

    Create private cluster

  2. You can skip the following instructions, if you're using Cloud Run for Anthos on a GKE cluster with the following versions:
    • 1.16.8-gke.7+
    • 1.15.11-gke.9+
    To deploy a service to Cloud Run for Anthos on a private GKE cluster, you must allow TCP connections from the control plane (master) servers to nodes on port 8443 and manually specify port 8443 in your list of allowed TCP connections by editing the firewall rules in your project:
    1. View the cluster CIDR block of the control plane and record the value in the masterIpv4CidrBlock field:

      gcloud container clusters describe CLUSTER_NAME
    2. View and record the value in the TARGET_TAGS field:

      gcloud compute firewall-rules list \
                --filter 'name~^gke-CLUSTER_NAME' \
                 --format 'table(
                         name,
                         network,
                         direction,
                         sourceRanges.list():label=SRC_RANGES,
                         allowed[].map().firewall_rule().list():label=ALLOW,
                         targetTags.list():label=TARGET_TAGS
                 )'
    3. Add a firewall rule using the values you recorded above:

      gcloud compute firewall-rules create FIREWALL_RULE_NAME \
           --action ALLOW \
           --direction INGRESS \
           --source-ranges masterIpv4CidrBlock \
           --rules tcp:8443 \
           --target-tags TARGET_TAGS

      For more information, see Creating firewall rules.

Configuring gcloud for cluster and platform

After you create the cluster,

  • Set your default platform to gke.
  • Optionally set defaults for cluster name, and cluster location to avoid subsequent prompts for these when you use the command line.
  • Get credentials that allow the gcloud command line to access your cluster.

To set defaults:

  1. Set the default platform to gke, set your default cluster and cluster location, and then get credentials as follows:

    gcloud config set run/platform gke
    gcloud config set run/cluster CLUSTER
    gcloud config set run/cluster_location ZONE
    gcloud container clusters get-credentials CLUSTER

    Replace

    • CLUSTER with the name of the cluster
    • ZONE with the location of the cluster.
  2. Kubernetes clusters come with a namespace named default. For information on namespaces, and why you might want to create and use a namespace other than default, refer to namespace in the Kubernetes documentation. To create a new namespace, run:

    kubectl create namespace NAMESPACE

    Replace NAMESPACE with the Namespace you want to create.

  3. If you created a new namespace in the previous step, and want to use it rather than the default namespace, set that new namespace as the one to be used by default when you invoke the gcloud command line:

    gcloud config set run/namespace NAMESPACE

Enabling metrics on a cluster with Workload Identity

When enabling Workload Identity, Cloud Run for Anthos doesn't report certain metrics, such as revision request count or request latency to Google Cloud's operations suite, but continues reporting metrics for CPU and memory.

To enable all metrics, you need to manually set permissions to write metrics to Cloud Monitoring by granting the Monitoring Metric Writer role to the Google service account (GSA) associated with your Cloud Run for Anthos service.

Grant the Monitoring Metric Writer role permissions to your service's GSA:

gcloud projects add-iam-policy-binding PROJECT_ID 
--member=serviceAccount:GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com
--role=roles/monitoring.metricWriter

Replace:

  • PROJECT_ID with the project ID for a cluster project that hosts your KSA.
  • GSA_PROJECT with the project ID for a GSA that's not in the cluster. You can use any GSA in your organization.

For more information, see Granting, changing, and revoking access to resources.

To set up services provided by Google Cloud APIs such as the Compute APIs, Storage and Database APIs, or Machine Learning APIs from within your GKE cluster, see Using Workload Identity.

Developing in a multi-tenant setup

In multi-tenant use cases, you'll need to manage and deploy Cloud Run for Anthos services to a Google Kubernetes Engine cluster that is outside your current project. This section instructs you how to develop Cloud Run for Anthos on Google Cloud services in a multi-tenant cluster setup.

To manage and deploy Cloud Run for Anthos services to a Google Kubernetes Engine cluster outside your current project:

  1. Ensure you have read access to the Google Cloud project ID of the cluster you are deploying to.

  2. Update your local kubeconfig file with credentials for the target GKE cluster:

    gcloud container clusters get-credentials NAME \
    --region=REGION \
    --project=PROJECT-ID
    • REGION is the Compute Engine region of your target cluster.
    • PROJECT-ID is the project you have read access to.

    For more information, see the gcloud container clusters get-credentials command reference documentation.

  3. Use the gcloud command line to communicate with the GKE cluster by setting the default platform to kubernetes:

    gcloud config set run/platform kubernetes
    

You can now run commands on the target GKE cluster specified in your kubeconfig file.

For example, the following command will deploy a Cloud Run for Anthos service using a specified container image to the GKE cluster whose credentials are stored in the kubeconfig file:

gcloud run deploy SERVICE-NAME --image IMAGE-NAME

Setting up a private, internal network

Deploying services on an internal network is useful for enterprises that provide internal apps to their staff, and for services that are used by clients that run outside the Cloud Run for Anthos on Google Cloud cluster. This configuration allows other resources in your network to communicate with the service using a private, internal (RFC 1918) IP address that can't be accessed by the public.

To create your internal network, you configure Istio's Ingress Gateway to use Internal TCP/UDP Load Balancing instead of a public, external network load balancer. You can then deploy your Cloud Run for Anthos on Google Cloud services on an internal IP address within your VPC network.

Before you begin

  • You must have admin permissions on your cluster.
  • Only Cloud SDK versions 310.0 or above are supported. For more details, see Setting up gcloud.

To set up the internal load balancer:

  1. Update the Istio Ingress Gateway to use Internal TCP/UDP Load Balancing by creating a new cluster or updating an existing cluster:

    • Create a new cluster with an internal load balancer:

      gcloud container clusters create CLUSTER_NAME \
      --addons=HttpLoadBalancing,CloudRun \
      --machine-type=n1-standard-2  \
      --num-nodes=3  \
      --enable-stackdriver-kubernetes \
      --cloud-run-config=load-balancer-type=INTERNAL
    • Update an existing cluster to use an internal load balancer:

      gcloud container clusters update CLUSTER_NAME
      --update-addons=CloudRun=ENABLED \
      --cloud-run-config=load-balancer-type=INTERNAL

    It might take a few minutes for the change to take effect.

  2. Run the following command to watch updates to your GKE cluster:

    kubectl -n gke-system get svc istio-ingress --watch
    
    1. Note the annotation `cloud.google.com/load-balancer-type: Internal".
    2. Look for the value of IP in the Ingress load balancer to change to a private IP address.
    3. Press Ctrl+C to stop the updates once you see a private IP address in the IP field.

To verify internal connectivity after your changes:

  1. Deploy a service called sample to Cloud Run for Anthos on Google Cloud in the default namespace:

    gcloud run deploy sample \
    --image gcr.io/knative-samples/simple-api \
    --namespace default \
    --platform gke
    
  2. Create a Compute Engine virtual machine (VM) in the same zone as the GKE cluster:

    VM=cloudrun-gke-ilb-tutorial-vm
    
    gcloud compute instances create $VM
    
  3. Store the private IP address of the Istio Ingress Gateway in an environment variable called EXTERNAL_IP and a file called external-ip.txt:

    export EXTERNAL_IP=$(kubectl -n gke-system get svc istio-ingress \
        -o jsonpath='{.status.loadBalancer.ingress[0].ip}' | tee external-ip.txt)
    
  4. Copy the file containing the IP address to the VM:

    gcloud compute scp external-ip.txt $VM:~
    
  5. Connect to the VM using SSH:

    gcloud compute ssh $VM
    
  6. While in the SSH session, test the sample service:

    curl -s -w'\n' -H Host:sample.default.example.com $(cat external-ip.txt)
    

    The output is as follows:

    OK
    
  7. Leave the SSH session:

    exit
    

Using a separate Istio installation

The following instructions show you how to connect Anthos Service Mesh, the Istio on GKE add-on, or a custom Istio installation with Cloud Run for Anthos in addition to the Istio components already installed by default in Cloud Run for Anthos on Google Cloud.

The Istio components included in the default Cloud Run for Anthos on Google Cloud install doesn't currently support automatic sidecar injection; however, you can use an additional Istio installation to enable Istio sidecar injection for your deployed service's namespace.

To use an additional Istio installation, you need to verify that the Istio Ingress Gateway is named istio-ingressgateway in the istio-system namespace. Cloud Run for Anthos can support and handle external traffic from Istio Ingress Gateways installed at:

  • The istio-system namespace, with the cluster local domain istio-ingressgateway.istio-system.svc.cluster.local that is set up by default when you use an additional Istio installation.
  • The gke-system namespace, with the cluster local domain istio-ingress.gke-system.svc.cluster.local that is set up with the default Cloud Run for Anthos on Google Cloud install.

To verify the additional Istio Ingress Gateway Cloud Run for Anthos uses:

  1. Open the config-istio ConfigMap:

    kubectl get configmap config-istio --namespace knative-serving -oyaml
    
  2. Verify your additional Istio Ingress Gateway is named istio-ingressgateway and is in the istio-system namespace.

Enabling HTTPS and custom domains

If you want to use HTTPS and custom domains that apply to the cluster, refer to Enabling HTTPS and automatic TLS certs and mapping custom domains.

Disabling Cloud Run for Anthos on Google Cloud

To disable Cloud Run for Anthos on Google Cloud in your cluster:

  1. Go to the Google Kubernetes Engine page in the Cloud Console:

    Go to Google Kubernetes Engine

  2. Click the cluster where you want to disable Cloud Run for Anthos on Google Cloud .

  3. Click Edit.

  4. From the Cloud Run for Anthos dropdown, select Disable.

  5. Click Save.

What's next