Design workload separation

This document provides an overview for workload management in Google Distributed Cloud (GDC) air-gapped. The following topics are covered:

Although some of the workload deployment designs are recommended, it's not required to follow them exactly as prescribed. Each GDC universe has unique requirements and considerations that must be satisfied on a case-by-case basis.

Where to deploy workloads

On the GDC platform, operations to deploy virtual machine (VM) workloads and container workloads are different. This section introduces the differences and where you deploy each resource.

VM-based workloads

You can create VMs to host your VM-based workloads. You have many configuration options for your VM's shape and size to help best meet your VM-based workload requirements. You must create a VM in a project, which can have many VMs and VM workloads. For more information, see the VMs overview.

Projects containing only VM-based workloads don't require a Kubernetes cluster. Therefore, you don't need to provision Kubernetes clusters for VM-based workloads.

Container-based workloads

You can deploy container-based workloads to a pod on a Kubernetes cluster. Kubernetes clusters can be attached to one or many projects, but they are not a child resource of a project. We recommend only attaching clusters to projects in the appropriate deployment environment. For example, a cluster for production workloads is attached to a project for production workloads.

For pod scheduling within a Kubernetes cluster, GDC adopts the general Kubernetes concepts of scheduling, preemption, and eviction. Best practices on scheduling pods within a cluster vary based on the requirements of your workload.

For more information on Kubernetes clusters, see the Kubernetes cluster overview. See the Container workloads overview for details on managing your containers in a Kubernetes cluster.

Best practices for designing Kubernetes clusters

This section introduces best practices for designing Kubernetes clusters:

Create separate clusters per deployment environment

In addition to separate projects per deployment environment, we recommend that you design separate Kubernetes clusters per deployment environment. By separating both the Kubernetes cluster and project per environment, you isolate resource consumption, access policies, maintenance events, and cluster-level configuration changes between your production and non-production workloads.

The following diagram shows a sample Kubernetes cluster design for multiple workloads that span projects, clusters, deployment environments, and machine classes.

GDC configuration

This sample architecture assumes that workloads within a deployment environment are allowed to share clusters. Each deployment environment has a separate set of Kubernetes clusters. You then assign projects to the Kubernetes cluster of the appropriate deployment environment. A Kubernetes cluster might be further subdivided into multiple node pools for different machine class requirements.

Alternatively, designing multiple Kubernetes clusters is useful for container operations like the following scenarios:

  • You have some workloads pinned to a specific Kubernetes version, so you maintain different clusters at different versions.
  • You have some workloads that require different cluster configuration needs, such as the backup policy, so you create multiple clusters with different configurations.
  • You run copies of a cluster in parallel to facilitate disruptive version upgrades or a blue-green deployment strategy.
  • You build an experimental workload that risks throttling the API server or other single point of failures within a cluster, so you isolate it from existing workloads.

The following diagram shows an example where multiple clusters are configured per deployment environment due to requirements such as the container operations described in the previous section.

GDC configuration

Create fewer clusters

For efficient resource utilization, we recommend designing the fewest number of Kubernetes clusters that meet your requirements for separating deployment environments and container operations. Each additional cluster incurs additional overhead resource consumption, such as additional control plane nodes required. Therefore, a larger cluster with many workloads utilizes underlying compute resources more efficiently than many small clusters.

When there are multiple clusters with similar configurations, it creates additional maintenance overhead to monitor cluster capacity and plan for cross-cluster dependencies.

If a cluster is approaching capacity, we recommend that you add additional nodes to a cluster instead of creating a new cluster.

Create fewer node pools within a cluster

For efficient resource utilization, we recommend designing fewer, larger node pools within a Kubernetes cluster.

Configuring multiple node pools is useful when you need to schedule pods that require a different machine class than others. Create a node pool for each machine class your workloads require, and set the node capacity to autoscaling to allow for efficient usage of compute resources.