This step explains how to create the cluster in which you will run Apigee hybrid. The instructions vary depending on the platform in which you are running hybrid. Before you begin, make sure to review the following information:
- Minimum cluster configurations
- Configuring dedicated node pools
- Configure Cassandra for production
- Configuring authentication for Cassandra
- StorageClass configuration
- Secure ports usage
Create your cluster
Follow the steps for your selected platform:
GKE
Create a cluster on GKE
These steps explain how to configure and create a GKE cluster in your Google Cloud project.
Apigee recommends creating a regional cluster
rather than a zonal cluster. If you are unfamiliar with the distinction between regions and zones, see
Regions and zones.
The available regions are listed in
Available regions and
zones. Just be aware that, for example, us-west1
is a valid region
name, while us-west1-a
is a zone in the region.
- Make sure you are using a version of GKE that is supported for hybrid version 1.12.1. See Apigee hybrid supported platforms and versions.
- Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, make sure they are synchronized with NTP across all regions.
- (GKE private clusters only), If you are creating a private cluster on GKE, add a firewall
rule to allow port 9443 for communication between GKE master nodes and GKE
worker nodes and to allow the GKE masters to access Apigee mutating webhooks. Follow the
procedure in
Adding firewall
rules for specific use cases in the Google Kubernetes Engine documentation. For more
information see Private
clusters in GKE.
You do not need to add this rule if you are creating a standard or public cluster.
- Create a standard cluster by following the instructions at
Create a regional cluster with a multi-zone node pool.
It's okay to create the cluster with just the default node pool. You will configure and create the
required Apigee hybrid node pools in the next step.
Go to the next step only after the cluster creation completes successfully.
- Create two node pools by following the instructions
in Add and
manage node pools. Be sure to configure the node pools with the minimum
requirements listed in the table below.
Minimum node pool requirements
Be sure to satisfy these minimum requirements when creating the node pools. Be sure to select the value for Prod or Non-prod depending on the the Apigee hybrid installation you are building. Non-prod installations are suitable for testing and demonstration but not sufficeint for production traffic. If you are using the Cloud console, be sure to configure both the Node pool details and Nodes sections.
Node pool name Description Minimum nodes Prod
Minimum
machine typeNon-prod
Minimum
machine typeapigee-data
A stateful node pool used for the Cassandra database. 1 per zone
(3 per region)e2-standard-8
(8 vCPU, 32 GB memory)e2-standard-4
(4 vCPU, 16 GB memory)apigee-runtime
A stateless node pool used by the runtime message processor. 1 per zone
(3 per region)e2-standard-8
(8 vCPU, 32 GB memory)e2-standard-4
(4 vCPU, 16 GB memory)For more details about node pool configuration see Configure dedicated node pools.
- (Optional) If you wish, you can delete the
default
node pool. See Delete a node pool. - If you haven't already, create the following environment variables. These variables are used
in the following gcloud commands.
Linux / MacOS
export CLUSTER_NAME="YOUR_CLUSTER_NAME"
export CLUSTER_LOCATION="YOUR_CLUSTER_LOCATION"
export PROJECT_ID="YOUR_PROJECT_ID"
Windows
set CLUSTER_NAME="YOUR_CLUSTER_NAME"
set CLUSTER_LOCATION=YOUR_CLUSTER_LOCATION
set PROJECT_ID=YOUR_PROJECT_ID
Where:
-
CLUSTER_NAME
: The name of your cluster. -
CLUSTER_LOCATION
: The region in which you created your cluster. -
PROJECT_ID
: Your Google Cloud project ID.
-
- Verify the node pool configurations:
Regional clusters
gcloud container node-pools list \ --cluster=${CLUSTER_NAME} \ --region=${CLUSTER_LOCATION} \ --project=${PROJECT_ID}
Zonal clusters
gcloud container node-pools list \ --cluster=${CLUSTER_NAME} \ --zone=${CLUSTER_LOCATION} \ --project=${PROJECT_ID}
- Make sure your cluster is set as the default cluster for
kubectl
by getting thegcloud
credentials of the cluster you just created:Regional clusters
gcloud container clusters get-credentials ${CLUSTER_NAME} \ --region ${CLUSTER_LOCATION} \ --project ${PROJECT_ID}
Zonal clusters
gcloud container clusters get-credentials ${CLUSTER_NAME} \ --zone ${CLUSTER_LOCATION} \ --project ${PROJECT_ID}
-
Configure persistent solid state disk (SSD) storage for Cassandra. We do not support using local SSDs. For more information, see Change the default storage class in the Kubernetes documentation.
- Get the name of the current default StorageClass:
kubectl get sc
For example:
kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h standard kubernetes.io/gce-pd Delete Immediate true 15h standard-rwo (default) pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h - Describe the StorageClass named
standard-rwo
. Note that its type ispd-balanced
:kubectl describe sc standard-rwo
For example:
kubectl describe sc standard-rwo
Name: standard-rwo IsDefaultClass: Yes Annotations: components.gke.io/layer=addon,storageclass.kubernetes.io/is-default-class=false Provisioner: pd.csi.storage.gke.io Parameters: type=pd-balanced AllowVolumeExpansion: True MountOptions: <none> ReclaimPolicy: Delete VolumeBindingMode: WaitForFirstConsumer Events: <none>
- Create a new file called
storageclass.yaml
. - Add this code to the file. Note that the name of the new storage class
is
apigee-sc
. You can use any name you like. Also, note that the storage type ispd-ssd
:--- kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: "apigee-sc" provisioner: kubernetes.io/gce-pd parameters: type: pd-ssd replication-type: none volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true
- Apply the new StorageClass to your Kubernetes cluster:
kubectl apply -f storageclass.yaml
- Execute the following two commands to change the default StorageClass:
kubectl patch storageclass standard-rwo \ -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
kubectl patch storageclass apigee-sc \ -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
- Execute this command to verify that the new default StorageClass is called
apigee-sc
:kubectl get sc
For example:
kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE apigee-sc (default) kubernetes.io/gce-pd Delete WaitForFirstConsumer true 14h premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h standard kubernetes.io/gce-pd Delete Immediate true 15h standard-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h
- Get the name of the current default StorageClass:
- Enable workload identity for the cluster. Workload Identity allows workloads in your GKE
clusters to impersonate Identity and Access Management (IAM) service accounts to access Google
Cloud services. This operation can take up to 30 minutes:
Regional clusters
gcloud container clusters update ${CLUSTER_NAME} \ --workload-pool=${PROJECT_ID}.svc.id.goog \ --project ${PROJECT_ID} \ --region ${CLUSTER_LOCATION}
Zonal clusters
gcloud container clusters update ${CLUSTER_NAME} \ --workload-pool=${PROJECT_ID}.svc.id.goog \ --zone ${CLUSTER_LOCATION} \ --project ${PROJECT_ID}
- Verify whether Workload identity is successfully enabled with the following command;
Regional clusters
gcloud container clusters describe ${CLUSTER_NAME} \ --project ${PROJECT_ID} \ --region ${CLUSTER_LOCATION} | grep -i "workload"
Zonal clusters
gcloud container clusters describe ${CLUSTER_NAME} \ --zone ${CLUSTER_LOCATION} \ --project ${PROJECT_ID} | grep -i "workload"
When you have a cluster installed and running, go to the next step.
Google Distributed Cloud on VMware
Create a cluster on VMware
These steps explain how to configure and create a GKE cluster for Apigee hybrid as part of a software-only Google Distributed Cloud on VMware deployment.
- Make sure you are using a version of Google Distributed Cloud that is supported for hybrid version 1.12.1. See Apigee hybrid supported platforms and versions.
- Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
- Create the cluster by following the instructions at
Create basic clusters.
It's okay to create the cluster with just the default node pool. You will configure and create the
required Apigee hybrid node pools in the next step.
Go to the next step only after the cluster creation completes successfully.
- Create two node pools by following the instructions
in Creating
and managing node pools. Configure the node pools with the minimum
requirements listed in the table below.
Minimum node pool requirements
Be sure to satisfy these minimum requirements when creating the node pools. Select the values for Prod or Non-prod depending on the the Apigee hybrid installation you are building. Non-prod installations are suitable for testing and demonstration but not sufficeint for production traffic.
Node pool name Description Minimum nodes Prod
Minimum
machine typeNon-prod
Minimum
machine typeapigee-data
A stateful node pool used for the Cassandra database. 1 per zone
(3 per region)e2-standard-8
(8 vCPU, 16 GB memory)e2-standard-4
(4 vCPU, 16 GB memory)apigee-runtime
A stateless node pool used by the runtime message processor. 1 per zone
(3 per region)e2-standard-8
(8 vCPU, 16 GB memory)e2-standard-4
(4 vCPU, 16 GB memory)For more details about node pool configuration see Configure dedicated node pools.
- (Optional) If you wish, you can delete the
default
node pool. See Delete a node pool. -
Configure persistent solid state disk (SSD) storage for Cassandra. We do not support using local SSDs. For more information, see Change the default storage class in the Kubernetes documentation.
- Get the name of the current default StorageClass:
kubectl get sc
For example:
kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h standard kubernetes.io/gce-pd Delete Immediate true 15h standard-rwo (default) pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h - Describe the StorageClass named
standard-rwo
. Note that its type ispd-balanced
:kubectl describe sc standard-rwo
For example:
kubectl describe sc standard-rwo
Name: standard-rwo IsDefaultClass: Yes Annotations: components.gke.io/layer=addon,storageclass.kubernetes.io/is-default-class=false Provisioner: pd.csi.storage.gke.io Parameters: type=pd-balanced AllowVolumeExpansion: True MountOptions: <none> ReclaimPolicy: Delete VolumeBindingMode: WaitForFirstConsumer Events: <none>
- Create a new file called
storageclass.yaml
. - Add this code to the file. Note that the name of the new storage class
is
apigee-sc
. You can use any name you like. Also, note that the storage type ispd-ssd
:--- kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: "apigee-sc" provisioner: kubernetes.io/gce-pd parameters: type: pd-ssd replication-type: none volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true
- Apply the new StorageClass to your Kubernetes cluster:
kubectl apply -f storageclass.yaml
- Execute the following two commands to change the default StorageClass:
kubectl patch storageclass standard-rwo \ -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
kubectl patch storageclass apigee-sc \ -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
- Execute this command to verify that the new default StorageClass is called
apigee-sc
:kubectl get sc
For example:
kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE apigee-sc (default) kubernetes.io/gce-pd Delete WaitForFirstConsumer true 14h premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h standard kubernetes.io/gce-pd Delete Immediate true 15h standard-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h
- Get the name of the current default StorageClass:
When you have a cluster installed and running, go to the next step.
Google Distributed Cloud on bare metal
Create a cluster on bare metal
These steps explain how to configure and create a GKE cluster for Apigee hybrid as part of a software-only deployment of Google Distributed Cloud on bare metal. Google Distributed Cloud on bare metal lets you run Kubernetes clusters directly on your own machine resources.
- Make sure you are using a version of Google Distributed Cloud that is supported for hybrid version 1.12.1. See Apigee hybrid supported platforms and versions.
- Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
- Review the Installation prerequisites overview and Creating clusters: overview.
- Create the cluster with two node pools configured as described below:
- Follow the cluster creation instructions in Create basic clusters.
- Create a minimum of two node pools. Configure the node pools with the minimum requirements
listed in the table below.
The minimum configurations for your cluster are:
Configuration Stateful node pool Stateless node pool Purpose A stateful node pool used for the Cassandra database. A stateless node pool used by the runtime message processor. Label name apigee-data apigee-runtime Number of nodes 1 per zone (3 per region) 1 per zone (3 per region) CPU 8 (Prod)
4 (Non-prod)8 (Prod)
4 (Non-prod)RAM 32 (Prod)
16 (Non-prod)32 (Prod)
16 (Non-prod)Storage dynamic Managed with the ApigeeDeployment CRD Minimum disk IOPS 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. Network bandwidth for each machine instance type 1 Gbps 1 Gbps For more details on minimum cluster configuration see: Minimum cluster configurations
When you have a cluster installed and running, go to the next step.
AKS
Create a cluster on AKS
These steps explain how to configure and create a cluster for Apigee hybrid on AKS.
- Make sure you are using a version of AKS that is supported for hybrid version 1.12.1. See Apigee hybrid supported platforms and versions.
- Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
- Create the cluster using either the Azure CLI
or Azure Portal, and create two node pools as described below.
The minimum configurations for your cluster are:
Configuration Stateful node pool Stateless node pool Purpose A stateful node pool used for the Cassandra database. A stateless node pool used by the runtime message processor. Label name apigee-data apigee-runtime Number of nodes 1 per zone (3 per region) 1 per zone (3 per region) CPU 8 (Prod)
4 (Non-prod)8 (Prod)
4 (Non-prod)RAM 32 (Prod)
16 (Non-prod)32 (Prod)
16 (Non-prod)Storage dynamic Managed with the ApigeeDeployment CRD Minimum disk IOPS 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. Network bandwidth for each machine instance type 1 Gbps 1 Gbps For more details on minimum cluster configuration see: Minimum cluster configurations
When you have a cluster installed and running, go to the next step.
EKS
Create a cluster on EKS
These steps explain how to configure and create a cluster for Apigee hybrid on EKS.
- Make sure you are using a version of EKS that is supported for hybrid version 1.12.1. See Apigee hybrid supported platforms and versions.
- Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
- If you are using Kubernetes version 1.24 or newer, make sure you have installed the Kubernetes CSI driver for Amazon EBS.
- Use the following instructions to create a user cluster, and create two node pools as
described below.
The minimum configurations for your cluster are:
Configuration Stateful node pool Stateless node pool Purpose A stateful node pool used for the Cassandra database. A stateless node pool used by the runtime message processor. Label name apigee-data apigee-runtime Number of nodes 1 per zone (3 per region) 1 per zone (3 per region) CPU 8 (Prod)
4 (Non-prod)8 (Prod)
4 (Non-prod)RAM 32 (Prod)
16 (Non-prod)32 (Prod)
16 (Non-prod)Storage dynamic Managed with the ApigeeDeployment CRD Minimum disk IOPS 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. Network bandwidth for each machine instance type 1 Gbps 1 Gbps For more details on minimum cluster configuration see: Minimum cluster configurations
When you have a cluster installed and running, go to the next step.
GKE on AWS
Create a cluster on GKE on AWS
These steps explain how to configure and create a cluster for Apigee hybrid on GKE on AWS.
- Make sure you are using a version of GKE on AWS that is supported for hybrid version 1.12.1. See Apigee hybrid supported platforms and versions.
- Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
- Use the following instructions to create a user cluster, and create two node
pools as described below.
The minimum configurations for your cluster are:
Configuration Stateful node pool Stateless node pool Purpose A stateful node pool used for the Cassandra database. A stateless node pool used by the runtime message processor. Label name apigee-data apigee-runtime Number of nodes 1 per zone (3 per region) 1 per zone (3 per region) CPU 8 (Prod)
4 (Non-prod)8 (Prod)
4 (Non-prod)RAM 32 (Prod)
16 (Non-prod)32 (Prod)
16 (Non-prod)Storage dynamic Managed with the ApigeeDeployment CRD Minimum disk IOPS 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. Network bandwidth for each machine instance type 1 Gbps 1 Gbps For more details on minimum cluster configuration see: Minimum cluster configurations
When you have a cluster installed and running, go to the next step.
OpenShift
Create a cluster on OpenShift
These steps explain how to configure and create a cluster for Apigee hybrid on OpenShift.
- Make sure you are using a version of OpenShift that is supported for hybrid version 1.12.1. See Apigee hybrid supported platforms and versions.
- Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
- Build the OpenShift cluster to deploy on the runtime plane,
install Apigee on your OpenShift user cluster, and create two node pools.
As part of the OpenShift install, install and configure the
oc
CLI tool. See Getting started with the OpenShift CLI in the OpenShift documentation.The minimum configurations for your cluster are:
Configuration Stateful node pool Stateless node pool Purpose A stateful node pool used for the Cassandra database. A stateless node pool used by the runtime message processor. Label name apigee-data apigee-runtime Number of nodes 1 per zone (3 per region) 1 per zone (3 per region) CPU 8 (Prod)
4 (Non-prod)8 (Prod)
4 (Non-prod)RAM 32 (Prod)
16 (Non-prod)32 (Prod)
16 (Non-prod)Storage dynamic Managed with the ApigeeDeployment CRD Minimum disk IOPS 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. 2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS. Network bandwidth for each machine instance type 1 Gbps 1 Gbps For more details on minimum cluster configuration see: Minimum cluster configurations
When you have installed a cluster, go to the next step.