Step 6: Configure the cluster

Specify configuration overrides

The Apigee hybrid installer uses defaults for many settings; however, there are a few settings that do not have defaults. You must provide values for these settings, as explained next.

Before you begin

We recommend you review the following scenarios to determine if you want to configure your cluster for them. These configurations are optional.

Configure the cluster

By convention, configuration overrides are written in a file named overrides.yaml in your $HYBRID_FILES/overrides directory.

  1. Create a new file named overrides.yaml in your $HYBRID_FILES/overrides directory. For example:
    vi $HYBRID_FILES/overrides/overrides.yaml

    The overrides.yaml provides the configuration for your unique Apigee hybrid installation. The overrides file in this step provides a basic configuration for a small-footprint hybrid runtime installation, suitable for your first installation.

  2. In overrides.yaml, add the required property values, shown below. A detailed description of each property is also provided below.

    If you are installing Apigee hybrid on GKE and you enabled Workload identity on your node pools in Step 1: Create a cluster, select the GKE - Workload Identity tab to configure your overrides.yaml file to use Workload identity.

    For all other installations, select the tab for either non-production, Non-prod or production, Prod environments, depending on your choice in Step 4: Create service accounts and credentials.

    For installations in production environments, look at the storage requirements for the Cassandra database in Configure Cassandra for production.

    GKE - Workload Identity

    Make sure the overrides.yaml file has the following structure and syntax. Values in red, bold italics are property values that you must provide. They are described in the table below.

    If you are installing Apigee hybrid on GKE, you have an alternative to authenticate and make requests to Google APIs, Workload Identity. For overviews of Workload Identity, see:

    To use Workload Identity with Apigee hybrid on GKE, use this template and then follow the steps in Step 8: Install hybrid runtime to create the Kubernetes service accounts and associate them with the Google service accounts you created in Step 4: Create service accounts and credentials.

    gcp:
      region: ANALYTICS_REGION
      projectID: GCP_PROJECT_ID
      workloadIdentityEnabled: true
    
    k8sCluster:
      name: CLUSTER_NAME
      region: CLUSTER_LOCATION # Must be the closest Google Cloud region to your cluster.
    org: ORG_NAME
    
    instanceID: "UNIQUE_INSTANCE_IDENTIFIER"
    
    cassandra:
      hostNetwork: false
        # false for all GKE installations.
        # See Multi-region deployment: Prerequisites
      replicaCount: 3
        # Use 1 for demo installations and multiples of 3 for production.
        # See Configure Cassandra for production for guidelines.
      backup:
        enabled: true
        # Set to true for initial installation.
        # This triggers apigeectl to create the apigee-cassandra-backup Kubernetes service account.
        # See Cassandra backup overview for instructions on using cassandra.backup.
    
    virtualhosts:
    - name: ENVIRONMENT_GROUP_NAME
      selector:
        app: apigee-ingressgateway
        ingress_name: INGRESS_NAME
      sslCertPath: ./certs/CERT_NAME.pem
      sslKeyPath: ./certs/KEY_NAME.key
    
    ingressGateways:
    - name: INGRESS_NAME # maximum 17 characters.
      replicaCountMin: 2
      replicaCountMax: 10
      svcAnnotations:  # optional. If you are on AKS, see Known issue #260772383
        SVC_ANNOTATIONS_KEY: SVC_ANNOTATIONS_VALUE
      svcLoadBalancerIP: SVC_LOAD_BALANCER_IP  # optional
    
    envs:
    - name: ENVIRONMENT_NAME
    
    logger:
      enabled: false # Set to false for all GKE installations.
    
    

    Non-prod

    Make sure the overrides.yaml file has the following structure and syntax. Values in red, bold italics are property values that you must provide. They are described in the table below.

    There are differences between the different platforms for the Google Cloud project region and Kubernetes cluster region. Choose the platform where you are installing Apigee hybrid.

    gcp:
      region: ANALYTICS_REGION
      projectID: GCP_PROJECT_ID
    
    k8sCluster:
      name: CLUSTER_NAME
      region: CLUSTER_LOCATION # Must be the closest Google Cloud region to your cluster.
    org: ORG_NAME
    
    instanceID: "UNIQUE_INSTANCE_IDENTIFIER"
    
    cassandra:
      replicaCount: 1
        # Use 1 for non-prod or "demo" installations and multiples of 3 for production.
        # See Configure Cassandra for production for guidelines.
      hostNetwork: false
        # Set to false for single region installations and multi-region installations
        # with connectivity between pods in different clusters, for example GKE installations.
        # Set to true  for multi-region installations with no communication between
        # pods in different clusters, for example GKE On-prem, GKE on AWS, Anthos on bare metal,
        # AKS, EKS, and OpenShift installations.
        # See Multi-region deployment: Prerequisites
    
    virtualhosts:
    - name: ENVIRONMENT_GROUP_NAME
      selector:
        app: apigee-ingressgateway
        ingress_name: INGRESS_NAME
      sslCertPath: ./certs/CERT_NAME.pem
      sslKeyPath: ./certs/KEY_NAME.key
    
    ingressGateways:
    - name: INGRESS_NAME # maximum 17 characters.
      replicaCountMin: 2
      replicaCountMax: 10
      svcAnnotations:  # optional. If you are on AKS, see Known issue #260772383
        SVC_ANNOTATIONS_KEY: SVC_ANNOTATIONS_VALUE
      svcLoadBalancerIP: SVC_LOAD_BALANCER_IP  # optional
    
    envs:
    - name: ENVIRONMENT_NAME
      serviceAccountPaths:
        synchronizer: NON_PROD_SERVICE_ACCOUNT_FILEPATH
          # For example: "./service-accounts/GCP_PROJECT_ID-apigee-non-prod.json"
        udca: NON_PROD_SERVICE_ACCOUNT_FILEPATH
        runtime: NON_PROD_SERVICE_ACCOUNT_FILEPATH
    
    
    mart:
      serviceAccountPath: NON_PROD_SERVICE_ACCOUNT_FILEPATH
    
    connectAgent:
      serviceAccountPath: NON_PROD_SERVICE_ACCOUNT_FILEPATH
    
    metrics:
      serviceAccountPath: NON_PROD_SERVICE_ACCOUNT_FILEPATH
    
    udca:
      serviceAccountPath: NON_PROD_SERVICE_ACCOUNT_FILEPATH
    
    watcher:
      serviceAccountPath: NON_PROD_SERVICE_ACCOUNT_FILEPATH
    
    logger:
      enabled: false
            # Set to false to disable logger for GKE installations.
            # Set to true for all platforms other than GKE.
            # See apigee-logger in Service accounts and roles used by hybrid components.
      serviceAccountPath: NON_PROD_SERVICE_ACCOUNT_FILEPATH
    
    

    Prod

    Make sure the overrides.yaml file has the following structure and syntax. Values in red, bold italics are property values that you must provide. They are described in the table below.

    There are differences between the different platforms for the Google Cloud project region and Kubernetes cluster region. Choose the platform where you are installing Apigee hybrid.

    gcp:
      region: ANALYTICS_REGION
      projectID: GCP_PROJECT_ID
    
    k8sCluster:
      name: CLUSTER_NAME
      region: CLUSTER_LOCATION # Must be the closest Google Cloud region to your cluster.
    org: ORG_NAME
    
    instanceID: "UNIQUE_INSTANCE_IDENTIFIER"
    
    cassandra:
      hostNetwork: false
        # Set to false for single region installations and multi-region installations
        # with connectivity between pods in different clusters, for example GKE installations.
        # Set to true  for multi-region installations with no communication between
        # pods in different clusters, for example GKE On-prem, GKE on AWS, Anthos on bare metal,
        # AKS, EKS, and OpenShift installations.
        # See Multi-region deployment: Prerequisites
      replicaCount: 3
        # Use multiples of 3 for production.
        # See Configure Cassandra for production for guidelines.
      storage:
        capacity: 500Gi
      resources:
        requests:
          cpu: 7
          memory: 15Gi
      maxHeapSize: 8192M
      heapNewSize: 1200M
        # Minimum storage requirements for a production environment.
        # See Configure Cassandra for production.
    
    virtualhosts:
    - name: ENVIRONMENT_GROUP_NAME
      selector:
        app: apigee-ingressgateway
        ingress_name: INGRESS_NAME
      sslCertPath: ./certs/CERT_NAME.pem
      sslKeyPath: ./certs/KEY_NAME.key
    
    ingressGateways:
    - name: INGRESS_NAME # maximum 17 characters. See Known issue 243167389.
      replicaCountMin: 2
      replicaCountMax: 10
      svcAnnotations:  # optional. If you are on AKS, see Known issue #260772383
        SVC_ANNOTATIONS_KEY: SVC_ANNOTATIONS_VALUE
    
    envs:
    - name: ENVIRONMENT_NAME
      serviceAccountPaths:
        synchronizer: SYNCHRONIZER_SERVICE_ACCOUNT_FILEPATH
          # For example: "./service-accounts/GCP_PROJECT_ID-apigee-synchronizer.json"
        udca: UDCA_SERVICE_ACCOUNT_FILEPATH
          # For example: "./service-accounts/GCP_PROJECT_ID-apigee-udca.json"
        runtime: RUNTIME_SERVICE_ACCOUNT_FILEPATH
          # For example: "./service-accounts/GCP_PROJECT_ID-apigee-runtime.json"
    
    mart:
      serviceAccountPath: MART_SERVICE_ACCOUNT_FILEPATH
            # For example: "./service-accounts/GCP_PROJECT_ID-apigee-mart.json"
    
    connectAgent:
      serviceAccountPath: MART_SERVICE_ACCOUNT_FILEPATH
            # Use the same service account for mart and connectAgent
    
    metrics:
      serviceAccountPath: METRICS_SERVICE_ACCOUNT_FILEPATH
            # For example: "./service-accounts/GCP_PROJECT_ID-apigee-metrics.json"
    
    udca:
      serviceAccountPath: UDCA_SERVICE_ACCOUNT_FILEPATH
            # For example: "./service-accounts/GCP_PROJECT_ID-apigee-udca.json"
    
    watcher:
      serviceAccountPath: WATCHER_SERVICE_ACCOUNT_FILEPATH
            # For example: "./service-accounts/GCP_PROJECT_ID-apigee-watcher.json"
    
    logger:
      enabled: false
            # Set to false to disable logger for GKE installations.
            # Set to true for all platforms other than GKE.
            # See apigee-logger in Service accounts and roles used by hybrid components.
      serviceAccountPath: LOGGER_SERVICE_ACCOUNT_FILEPATH
            # For example: "./service-accounts/GCP_PROJECT_ID-apigee-logger.json"
    

    Example

    The following example shows a completed overrides file with example property values added:

    gcp:
      region: us-central1
      projectID: hybrid-example
    
    k8sCluster:
      name: apigee-hybrid
      region: us-central1
    
    org: hybrid-example
    
    instanceID: "my_hybrid_example"
    
    cassandra:
      hostNetwork: false
      replicaCount: 3
    
    virtualhosts:
    - name: example-env-group
      selector:
        app: apigee-ingressgateway
        ingress_name: my-ingress-1
      sslCertPath: ./certs/keystore.pem
      sslKeyPath: ./certs/keystore.key
    
    ingressGateways:
    - name: my-ingress-1
      replicaCountMin: 2
      replicaCountMax: 10
    
    envs:
    - name: test
      serviceAccountPaths:
        synchronizer: ./service-accounts/my-hybrid-project-apigee-non-prod.json
          # for production environments, my-hybrid-project-apigee-synchronizer.json
        udca: ./service-accounts/my-hybrid-project-apigee-non-prod.json
          # for production environments, my-hybrid-project-apigee-udca.json
        runtime: ./service-accounts/my-hybrid-project-apigee-non-prod.json
          # for production environments, my-hybrid-project-apigee-runtime.json
    
    mart:
      serviceAccountPath: ./service-accounts/my-hybrid-project-apigee-non-prod.json
        # for production environments, my-hybrid-project-apigee-mart.json
    
    connectAgent:
      serviceAccountPath: ./service-accounts/my-hybrid-project-apigee-non-prod.json
        # for production environments, example-hybrid-apigee-mart.json
    
    metrics:
      serviceAccountPath: ./service-accounts/my-hybrid-project-apigee-non-prod.json
        # for production environments, my-hybrid-project-apigee-metrics.json
    
    udca:
      serviceAccountPath: ./service-accounts/my-hybrid-project-apigee-non-prod.json
        # for production environments, my-hybrid-project-apigee-udca.json
    
    watcher:
      serviceAccountPath: ./service-accounts/my-hybrid-project-apigee-non-prod.json
        # for production environments, my-hybrid-project-apigee-watcher.json
    
    logger:
      enabled: false # Set to "false" for GKE. Set to "true" for all other Kubernetes platforms.
      serviceAccountPath: ./service-accounts/my-hybrid-project-apigee-non-prod.json
        # for production environments, LOGGER_SERVICE_ACCOUNT_NAME.json
    
  3. When you are finished, save the file.

The following table describes each of the property values that you must provide in the overrides file. For more information, see Configuration property reference.

Variable Description
ANALYTICS_REGION In GKE, You must set this value to the same region where the cluster is running. In all other platforms, select the closest analytics region to your cluster that has Analytics support (see the table in Part 1, Step 2: Create an organization.

This is the value you assigned to the environment variable ANALYTICS_REGION previously.

GCP_PROJECT_ID Identifies the Google Cloud project where the apigee-logger and the apigee-metrics push their data. This is the value assigned to the environment variable PROJECT_ID.
CLUSTER_NAME Your Kubernetes cluster name. This is the value assigned to the environment variable CLUSTER_NAME.
CLUSTER_LOCATION The region where the cluster is running. This is the region where you created the cluster in Step 1: Create a cluster.

This is the value you assigned to the environment variable CLUSTER_LOCATION previously.

ORG_NAME The ID of your Apigee hybrid organization. This is the value assigned to the environment variable ORG_NAME.
UNIQUE_INSTANCE_IDENTIFIER

A unique string to identify this instance. This can be any combination of letters and numbers up to 63 characters in length.

You can create multiple organizations in the same cluster, but the instanceID must be the same for all orgs in the same Kubernetes cluster.

ENVIRONMENT_GROUP_NAME The name of the environment group your environments are assigned to. This is the group you created in Project and org setup - Step 3: Create an environment group. This is the value assigned to the environment variable ENV_GROUP.
CERT_NAME
KEY_NAME
Enter the name of the self-signed TLS key and certificate files that you generated previously in Step 5: Create TLS certificates. These files must be located in the base_directory/hybrid-files/certs directory. For example:
sslCertPath: ./certs/keystore.pem
sslKeyPath: ./certs/keystore.key
INGRESS_NAME The name of the Apigee ingress gateway for your deployment. This can be any name that meets the following requirements:
  • Have a maximum length of 17 characters
  • Contain only lowercase alphanumeric characters, '-' or '.'
  • Start with an alphanumeric character
  • End with an alphanumeric character

See ingressGateways[].name in the Configuration property reference

SVC_ANNOTATIONS_KEY: SVC_ANNOTATIONS_VALUE (Optional). This is a key-value pair that provides annotations for your default ingress service. Annotations are used by your cloud platform to help configure your hybrid installation, for example setting the loadbalancer type to either internal or external.

Annotations vary from platform to platform. Refer to your platform documentation for required and suggested annotations.

Comment out or delete this section if you are not using it.

SVC_LOAD_BALANCER_IP (Optional). An IP address you have reserved for your load balancer. On platforms that support specifying the load balancer IP address, the load balancer will be created with this IP address. On platforms that do not allow you to specify the load balancer IP, this property is ignored.

Comment out or delete this section if you are not using it.

ENVIRONMENT_NAME Use the same name that you used when you created an environment in the UI, as explained in Project and org setup - Step 3: Create an environment group.
*_SERVICE_ACCOUNT_FILEPATH The path and filename account of the service account JSON files in your service-accounts/ directory. The names must include the path to the service account file. This can be a full path, or the path relative to your hybrid-files/ directory. If you include a relative path, you must call apigeectl, the command to apply this configuration, from your hybrid-files/ directory.

For non-production environments, the name of the single service account is GCP_PROJECT_ID-non-prod.json by default.

For production environments, the name of the service account key file that you generated with the create-service-account tool in Hybrid runtime setup - Step 4: Create service accounts and credentials.

You can see the list of service account files in your service-accounts/ directory.

The default names of the production environment service accounts are:

  • Cassandra: GCP_PROJECT_ID-apigee-cassandra.json
  • Logger: GCP_PROJECT_ID-apigee-logger.json
  • MART: GCP_PROJECT_ID-apigee-mart.json
  • Connect agent: GCP_PROJECT_ID-apigee-mart.json
  • Metrics: GCP_PROJECT_ID-apigee-metrics.json
  • Apigee runtime: GCP_PROJECT_ID-
  • Synchronizer: GCP_PROJECT_ID-apigee-synchronizer.json
  • UDCA: GCP_PROJECT_ID-apigee-udca.json
  • Watcher: GCP_PROJECT_ID-apigee-watcher.json

Summary

The configuration file tells Kubernetes how to deploy the hybrid components to a cluster. Next, you will enable synchronizer access so the Apigee runtime and management planes will be able to communicate.

Next step

1 2 3 4 5 6 (NEXT) Step 7: Enable Synchronizer access 8 9 10