GKE on Bare Metal supports multiple deployment models to meet different availability, isolation, and resource footprint needs. This page defines the concepts all deployment models share, and describes each deployment model.
User clusters
A user cluster is a Kubernetes cluster that runs your containerized workloads. It consists of control plane nodes and worker nodes. GKE on Bare Metal supports one or more user clusters. User clusters must contain one or more worker nodes that run user workloads.
Admin clusters
An admin cluster is a Kubernetes cluster that manages one or more user clusters. Your admin cluster can perform the following tasks:
- Create user clusters
- Upgrade user clusters
- Update user clusters
- Delete user clusters
To create a user cluster, your admin cluster sets up the Anthos on bare metal components on the user cluster's control plane and worker nodes. Your admin cluster only has control plane nodes because the GKE on Bare Metal components run on the control plane nodes.
Your admin cluster contains the following types of sensitive data:
- SSH credentials: Used to enable remote installation
- Google Cloud service account keys: Used to access features like Container Registry
To protect the sensitive data, restrict access to your admin cluster.
High availability
You can run admin or user clusters in high availability (HA) mode. This mode requires three or more (odd number) control plane nodes running in your cluster. If you run a cluster in non-HA mode, your cluster requires only one control plane node.
To avoid having a single point of failure, use the HA mode for production deployments. Use the non-HA mode for non-mission critical environments, for example a testing environment, where you can re-create the cluster if the single control plane node fails. An HA user cluster must have two or more worker nodes in case a worker node fails.
When you upgrade your clusters, an HA deployment also reduces the risk of the cluster becoming inaccessible if there's an error.
Deployment models
GKE on Bare Metal supports the following deployment models to meet different requirements:
Admin and user cluster deployment
Use this deployment model if you have multiple clusters in the same datacenter that you want to manage from a centralized place, and for larger deployments that need isolation between different teams or between development and production workloads.
This deployment model consists of the following clusters:
- One admin cluster: The central management point that provides an API to manage user clusters. Your admin cluster only runs management components.
- One or more user clusters: Contain the control plane nodes and the worker nodes, which run user workloads.
This model meets the following requirements:
- Provides a centralized control plane and API to manage your user clusters' lifecycles.
- Provides isolation between different teams.
- Provides isolation between development and production workloads.
- You don't have to share SSH credentials and service account keys with cluster owners.
- You can integrate your deployment with your own control planes
Footprint
An admin and user cluster deployment requires the following nodes:
Admin cluster
- One control plane node for non-HA
- Three or more control plane nodes for HA
User clusters - You can configure HA for each user cluster independently.
Control plane nodes:
- One control plane node for non-HA
- Three or more control plane nodes for HA
Worker nodes:
- One or more worker nodes for non-HA
- Two or more worker nodes for HA
Hybrid cluster deployment
This deployment model is a specialized multi-cluster deployment. A hybrid cluster is an admin cluster that can run user workloads. Your hybrid cluster still manages additional user clusters.
Features of this model:
- Allocating a set of machines for an admin cluster is often wasteful because admin clusters use relatively few resources. The hybrid cluster deployment allows you to reclaim unused capacity on those machines because it allows you to run user workloads within the admin cluster.
- The admin cluster contains sensitive data such as SSH credentials (used by the admin cluster to manage user clusters on remote machines) and Google Cloud service account keys (used by the admin cluster to access Google Cloud services such as Cloud Storage). Hybrid cluster deployments run user workloads within the admin cluster, and this potentially exposes the admin cluster's sensitive data to user workloads.
Footprint
A hybrid cluster deployment requires the following nodes:
Hybrid cluster
Control plane nodes:
- One control plane node for non-HA
- Three or more control plane nodes for HA
Worker nodes:
- One or more worker nodes for non-HA
- Two or more worker nodes for HA and depending on workload type
User clusters - You can configure HA for each user cluster independently.
Control plane nodes:
- One control plane node for non-HA
- Three or more control plane nodes for HA
Worker nodes:
- One or more worker nodes for non-HA
- Two or more worker nodes for HA
Standalone cluster deployment
This deployment model has a single cluster that serves as a user cluster and as an admin cluster.
This model has the following advantages:
- It doesn't require a separate admin cluster
- It supports the edge profile, which has significantly reduced system resource requirements and is recommended for edge devices with high resource constraints.
This model has some security tradeoffs, because workloads run on a cluster with the following sensitive data:
- SSH credentials
- Google Cloud service account keys
Use this model if you meet any of the following conditions:
- You manage every cluster independently.
- You have a small number of worker nodes.
- You support a single team.
- You run a single workload type.
This model works well in the following situations:
- Every cluster is independently managed with different SSH keys and Google Cloud credentials
- Clusters run in network isolated partitions like demilitarized zones (DMZs)
- Clusters run in edge locations
Footprint
A standalone cluster deployment requires the following nodes:
Control plane nodes:
- One control plane node for non-HA
- Three or more control plane nodes for HA
Worker nodes:
- One or more worker nodes for non-HA
- Two or more worker nodes for HA