Deploying internal services using Cloud Run for Anthos on Google Cloud

This tutorial demonstrates how to expose services deployed to Cloud Run for Anthos on Google Cloud on your internal network. This type of configuration allows other resources in your network to communicate with the service using a private, internal (RFC 1918) IP address. Exposing services on an internal network is useful for enterprises that provide internal apps to their staff, and for services that are used by clients that run outside the Cloud Run for Anthos on Google Cloud cluster.

Cloud Run for Anthos provides a developer-focused experience for deploying and serving apps and functions running on GKE. By default, Cloud Run for Anthos on Google Cloud exposes services outside the cluster by using Istio's ingress gateway. This gateway is a Kubernetes service of type LoadBalancer, which means by default it's exposed on a public IP address using Network Load Balancing.

This tutorial shows you how to expose your Cloud Run for Anthos on Google Cloud services on an internal IP address in your VPC network by changing Istio's ingress gateway to use Internal TCP/UDP Load Balancing instead of Network Load Balancing.

Objectives

  • Create a GKE cluster with Cloud Run enabled.
  • Update the Istio ingress gateway to use Internal TCP/UDP Load Balancing.
  • Test the app by deploying a sample service to Cloud Run for Anthos on Google Cloud.

Costs

This tutorial uses the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Cleaning up.

Before you begin

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. In the Cloud Console, on the project selector page, select or create a Cloud project.

    Go to the project selector page

  3. Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.

  4. In the Cloud Console, go to Cloud Shell.

    Go to Cloud Shell

    At the bottom of the Cloud Console, a Cloud Shell session opens and displays a command-line prompt. Cloud Shell is a shell environment with the Cloud SDK already installed, including the gcloud command-line tool, and with values already set for your current project. It can take a few seconds for the session to initialize.

  5. You run all commands in this tutorial from Cloud Shell.
  6. In Cloud Shell, enable the Cloud Run API, GKE API, and Cloud APIs:
    gcloud services enable \
        cloudapis.googleapis.com \
        container.googleapis.com \
        run.googleapis.com

Setting up the environment

  • In Cloud Shell, define environment variables and the gcloud tool defaults for the Compute Engine zone and GKE cluster name that you want to use for this tutorial:

    ZONE=us-central1-f
    CLUSTER=cloudrun-gke-ilb-tutorial
    
    gcloud config set compute/zone $ZONE
    gcloud config set run/cluster $CLUSTER
    gcloud config set run/cluster_location $ZONE
    

    The examples in this tutorial use us-central1-f as the zone and cloudrun-gke-ilb-tutorialas the cluster name. You can use different values. For more information, see Geography and regions.

Creating a GKE cluster with Cloud Run enabled

  1. In Cloud Shell, create a GKE cluster with the Cloud Run add-on:

    gcloud beta container clusters create $CLUSTER \
        --addons HorizontalPodAutoscaling,HttpLoadBalancing,CloudRun \
        --enable-ip-alias \
        --enable-stackdriver-kubernetes \
        --machine-type n1-standard-2
    

Configuring Internal TCP/UDP Load Balancing

  1. In Cloud Shell, patch the Istio ingress gateway to use Internal TCP/UDP Load Balancing:

    kubectl -n gke-system patch svc istio-ingress -p \
        '{"metadata":{"annotations":{"cloud.google.com/load-balancer-type":"Internal"}}}'
    

    It might take a few minutes for the change to take effect. Run the following command to poll your GKE cluster for change. Look for the value of EXTERNAL-IP to change to a private IP address:

    kubectl -n gke-system get svc istio-ingress --watch
    

    Press Ctrl+C to stop the polling when you see a private IP address in the EXTERNAL-IP field.

Deploy a sample service

  • In Cloud Shell, deploy a service called sample to Cloud Run for Anthos on Google Cloud in the default namespace:

    gcloud run deploy sample \
        --image gcr.io/knative-samples/simple-api \
        --namespace default \
        --platform gke
    

Verify internal connectivity

  1. In Cloud Shell, create a Compute Engine virtual machine (VM) in the same zone as the GKE cluster:

    VM=cloudrun-gke-ilb-tutorial-vm
    
    gcloud compute instances create $VM
    
  2. Store the private IP address of the Istio ingress gateway in an environment variable called EXTERNAL_IP and a file called external-ip.txt:

    export EXTERNAL_IP=$(kubectl -n gke-system get svc istio-ingress \
        -o jsonpath='{.status.loadBalancer.ingress[0].ip}' | tee external-ip.txt)
    
  3. Copy the file containing the IP address to the VM:

    gcloud compute scp external-ip.txt $VM:~
    
  4. Connect to the VM using SSH:

    gcloud compute ssh $VM
    
  5. While in the SSH session, test the sample service:

    curl -s -w'\n' -H Host:sample.default.example.com $(cat external-ip.txt)
    

    The output is as follows:

    OK
    
  6. Leave the SSH session:

    exit
    

Troubleshooting

If you run into problems with this tutorial, review the following documents:

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

Delete the project

  1. In the Cloud Console, go to the Manage resources page.

    Go to the Manage resources page

  2. In the project list, select the project that you want to delete and then click Delete .
  3. In the dialog, type the project ID and then click Shut down to delete the project.

Delete the individual resources

If you want to keep the Google Cloud project you used in this tutorial, delete the individual resources:

  1. Delete the GKE cluster:

    gcloud container clusters delete $CLUSTER --quiet --async
    
  2. Delete the Compute Engine instance:

    gcloud compute instances delete $VM --quiet
    

What's next