Architectural overview of Cloud Run for Anthos (Knative serving)

This page provides an architectural overview of Cloud Run for Anthos (Knative serving) and covers the changes that occur when you enable Cloud Run for Anthos in your Google Kubernetes Engine cluster.

This information is useful for the following types of users:

  • Users getting started with Cloud Run for Anthos.
  • Operators with experience in running GKE clusters.
  • Application developers who need to know more about how Cloud Run for Anthos integrates with Kubernetes clusters to design better applications or configure their Cloud Run for Anthos application.

Components in the default installation

Cloud Run for Anthos installs Knative Serving into your cluster to connect and manage your stateless workloads. Knative components are created in the knative-serving namespace.

Cloud Run for Anthos uses Anthos Service Mesh to route traffic. By default, Anthos Service Mesh installs components in the istio-system namespace.

Here's a list of the components installed by Cloud Run for Anthos and Anthos Service Mesh:

  • Components installed by Cloud Run for Anthos in the knative-serving namespace:

    • Activator: When pods are scaled in to zero or become overloaded with requests sent to the revision, Activator temporarily queues the requests and sends metrics to Autoscaler to spin up more pods. Once Autoscaler scales the revision based on the reported metrics and available pods, Activator forwards queued requests to the revision. Activator is a data plane component; data plane components manage all functions and processes forwarding user traffic.
    • Autoscaler: Aggregates and processes metrics from Activator and the queue proxy sidecar container, a component in the data plane that enforces request concurrency limits. Autoscaler then calculates the observed concurrency for the revision and adjusts the size of the deployment based on the desired pod count. When pods are available in the revision, Autoscaler is a control plane component; otherwise, when pods are scaled in to zero, Autoscaler is a data plane component.
    • Controller: Creates and updates the child resources of Autoscaler and the Service objects. Controller is a control plane component; control plane components manage all functions and processes establishing the request path of user traffic.
    • Metrics Collector: Collects metrics from Cloud Run for Anthos components then forwards them to Cloud Monitoring.
    • Webhook: Sets default values, rejects inconsistent and invalid objects, and validates and mutates Kubernetes API calls against Cloud Run for Anthos resources. Webhook is a control plane component.
  • Components installed by Anthos Service Mesh running in the istio-system namespace:

    • Cluster Local Gateway: Load balancer in the data plane responsible for handling internal traffic that arrives from one Cloud Run for Anthos to another. The Cluster Local Gateway can only be accessed from within your GKE cluster and does not register an external domain to prevent accidental exposure of private information or internal processes.
    • Istio Ingress Gateway: Load balancer in the data plane that is responsible for receiving and handling incoming traffic from outside the cluster, including traffic from either external or internal networks.
    • Istiod: Configures the Cluster Local Gateway and the Istio Ingress Gateway to handle HTTP requests at the correct endpoints. Istiod is a control plane component. For more information, see Istiod.

Cloud Run for Anthos components are updated automatically with any GKE control plane cluster updates. For more information, see Available GKE versions.

Cluster resource usage

The initial installation for Cloud Run for Anthos approximately requires 1.5 virtual CPU and 1 GB of memory for your cluster. The number of nodes in your cluster do not affect the space and memory requirements for a Cloud Run for Anthos installation.

An Activator can consume requests at a maximum of 1000 milliCPU and 600 MiB RAM. When an existing Activator can't support the number of incoming requests, an additional Activator spins up, which requires a reservation of 300 milliCPU and 60 MiB RAM.

Every pod created by the Cloud Run for Anthos service creates a queue proxy sidecar that enforces request concurrency limits. The queue proxy reserves 25 milliCPU and has no memory reservation. The queue proxy's consumption depends on how many requests are getting queued and the size of the requests; there are no limits on the CPU and memory resources it can consume.

Creating a Service

Diagram showing Cloud Run for Anthos service architecture
Cloud Run for Anthos Service architecture (click to enlarge)

Cloud Run for Anthos extends Kubernetes by defining a set of Custom Resource Definitions (CRDs): Service, Revision, Configuration, and Route. These CRDs define and control how your applications behave on the cluster:

  • Cloud Run for Anthos Service is the top level custom resource defined by Cloud Run for Anthos. It is a single application that manages the whole lifecycle of your workload. Your service ensures your app has a route, a configuration, and a new revision for each update of the service.
  • Revision is a point-in-time, immutable snapshot of the code and configuration.
  • Configuration maintains the current settings for your latest revision and records a history of all past revisions. Modifying a configuration creates a new revision.
  • Route defines an HTTP endpoint and associates the endpoint with one or more revisions to which requests are forwarded.

When a user creates a Cloud Run for Anthos Service, the following happen:

  1. The Cloud Run for Anthos Service object defines:

    1. A configuration for how to serve your revisions.
    2. An immutable revision for this version of your service.
    3. A route to manage specified traffic allocation to your revision.
  2. The route object creates VirtualService. The VirtualService object configures Ingress Gateway and Cluster Local Gateway to route gateway traffic to the correct revision.

  3. The revision object creates the following control plane components: a Kubernetes Service object and a Deployment object.

  4. Network configuration connects Activator, Autoscaler, and load balancers for your app.

Request handling

The following diagram shows a high level overview of a possible request path for user traffic through the Cloud Run for Anthos data plane components on a sample Google Kubernetes Engine cluster:

Diagram showing Cloud Run for Anthos cluster architecture
Cloud Run for Anthos cluster architecture (click to enlarge)

The next diagram expands from the diagram above to give an in depth view into the user traffic's request path, also described in detail below:

Diagram showing Cloud Run for Anthos request path
Cloud Run for Anthos request path (click to enlarge)

For a Cloud Run for Anthos request path:

  1. Traffic arrives through:

    • The Ingress Gateway for traffic from outside of clusters
    • The Cluster Local Gateway for traffic within clusters
  2. The VirtualService component, which specifies traffic routing rules, configures the gateways so that user traffic is routed to the correct revision.

  3. Kubernetes Service, a control plane component, determines the next step in the request path dependent on the availability of pods to handle the traffic:

    • If there are no pods in the revision:

      1. Activator temporarily queues the request received and pushes a metric to Autoscaler to scale more pods.
      2. Autoscaler scales to desired state of pods in Deployment.
      3. Deployment creates more pods to receive additional requests.
      4. Activator retries requests to the queue proxy sidecar.
    • If the service is scaled out (pods are available), the Kubernetes Service sends the request to the queue proxy sidecar.

  4. The queue proxy sidecar enforces request queue parameters, single or multi-threaded requests, that the container can handle at a time.

  5. If the queue proxy sidecar has more requests than it can handle, Autoscaler creates more pods to handle additional requests.

  6. The queue proxy sidecar sends traffic to the user container.