Upgrading Apigee hybrid to version 1.8

Introducing Apigee ingress gateway

Starting in version 1.8, Apigee hybrid offers a new feature to manage the ingress gateway for your hybrid installation, Apigee ingress gateway. Anthos Service Mesh is no longer a prerequisite for hybrid installation. With Apigee ingress gateway, Apigee will stop supplying routing configuration to Anthos Service Mesh. After the upgrade, you have to migrate the traffic to the new Apigee ingress gateway before you can start using the feature.

Apigee uses a small subset of Anthos Service Mesh features for the ingress gateway. Starting with hybrid version 1.8 Apigee hybrid includes an ingress gateway which is installed and upgraded as part of Apigee hybrid upgrades. Therefore you do not need to build expertise around Anthos Service Mesh to install, upgrade, and manage Apigee hybrid. Issues around ingress gateway versions and compatibility with Apigee hybrid releases are handled automatically.

Two scenarios for migrating are:

  • Multi-cluster or multi-region migration (recommended):

    Before switching to a new Ingress for Apigee, drain all the traffic to another cluster or region from the cluster you are migrating. This will give you time to test if the new Apigee ingress gateway is working as expected. Then shift the traffic back to the upgraded cluster.

  • In-place upgrade (not recommended in production environments):

    During the upgrade Apigee will bring up the new ingress gateway with an IP address you specify. You can then test if the new Apigee ingress gateway is working as expected, and then shift traffic to the new ingress. There might be downtime during this upgrade.

When upgrading Apigee hybrid to version 1.8, you must configure Apigee ingress gateway in your overrides file. After upgrading, you control which type of ingress gateway your clusters will use by directing the A or CNAME records at your registrar to the IP address for Apigee ingress gateway or for Anthos Service Mesh.

Upgrading to version 1.8.8 overview

The procedures for upgrading Apigee hybrid are organized in the following sections:

  1. Prepare to upgrade.
  2. Install hybrid runtime version 1.8.8.
  3. For the ingress gateway, pick one of the following options:

Prerequisite

These upgrade instructions assume you have Apigee hybrid version 1.7.x or an earlier patch release of version 1.8.x installed and wish to upgrade it to version 1.8.8. If you are updating from an earlier version see the instructions for Upgrading Apigee hybrid to version 1.7.

If you prefer to continue using Anthos Service Mesh, you must ensure that Anthos Service Mesh is upgraded to a supported version. See the Supported platforms table for supported versions of Anthos Service Mesh.

Prepare to upgrade to version 1.8

  1. These instructions use the environment variable APIGEECTL_HOME for the directory in your file system where you have installed apigeectl. If needed, change directory into your apigeectl directory and define the variable with the following command:

    Linux

    export APIGEECTL_HOME=$PWD
    echo $APIGEECTL_HOME

    Mac OS

    export APIGEECTL_HOME=$PWD
    echo $APIGEECTL_HOME

    Windows

    set APIGEECTL_HOME=%CD%
    echo %APIGEECTL_HOME%
  2. Make a backup copy of your version 1.7 $APIGEECTL_HOME/ directory. For example:
    tar -czvf $APIGEECTL_HOME/../apigeectl-v1.7-backup.tar.gz $APIGEECTL_HOME
  3. Back up your Cassandra database following the instructions in Cassandra backup and recovery

Add the Cloud Trace Agent role to the service account for the Apigee runtime. (Optional)

Optional: If you plan to use Cloud trace and you have not already performed this step on your hybrid v1.7 installation, ensure your service account for your Apigee runtime services has the Cloud Trace Agent Google role. (roles/cloudtrace.agent).

For production environments, this is usually the apigee-runtime service account. For non-production environments, this is usually the apigee-non-prod service account.

You can add the role in the Cloud console > IAM & Admin > Service accounts UI or with the following commands:

  1. Get the email address for your service account with the following command:

    Production

    gcloud iam service-accounts list --filter "apigee-runtime"

    If it matches the pattern apigee-runtime@$ORG_NAME.iam.gserviceaccount.com, you can use that pattern in the next step.

    Non-Prod

    gcloud iam service-accounts list --filter "apigee-non-prod"

    If it matches the pattern apigee-non-prod@$ORG_NAME.iam.gserviceaccount.com, you can use that pattern in the next step.

  2. Assign the Cloud Trace Agent role to the service account:

    Production

    gcloud projects add-iam-policy-binding $PROJECT_ID \
        --member="serviceAccount:apigee-runtime@$PROJECT_ID.iam.gserviceaccount.com" \
        --role="roles/cloudtrace.agent"

    Non-Prod

    gcloud projects add-iam-policy-binding $PROJECT_ID \
        --member="serviceAccount:apigee-non-prod@$PROJECT_ID.iam.gserviceaccount.com" \
        --role="roles/cloudtrace.agent"

    Example

    gcloud projects add-iam-policy-binding hybrid-example-project \
        --member="serviceAccount:apigee-runtime@hybrid-example-project.iam.gserviceaccount.com" \
        --role="roles/cloudtrace.agent"

    Where: $PROJECT_ID is the name of the Google Cloud project where Apigee hybrid is installed.

Prepare to install Apigee ingress gateway

To install Apigee ingress gateway as part of the upgrade. You need to add the following ingressGateways property to your overrides file.

Syntax

ingressGateways:
- name: INGRESS_NAME
  replicaCountMin: REPLICAS_MIN
  replicaCountMax: REPLICAS_MAX
  resources:
    requests:
      cpu: CPU_COUNT_REQ
      memory: MEMORY_REQ
    limits:
      cpu: CPU_COUNT_LIMIT
      memory: MEMORY_LIMIT
  svcAnnotations:  # optional. See Known issue 243599452.
    SVC_ANNOTATIONS_KEY: SVC_ANNOTATIONS_VALUE
  svcLoadBalancerIP: SVC_LOAD_BALANCER_IP # optional

Example

ingressGateways:
- name: prod1
  replicaCountMin: 2
  replicaCountMax: 100
  resources:
    requests:
      cpu: 1
      memory: 1Gi
    limits:
      cpu: 2
      memory: 2Gi 
  • INGRESS_NAME is the name of the ingress deployment. This can be any name that meets the following requirements:
    • Have a maximum length of 17 characters
    • Contain only lowercase alphanumeric characters, '-' or '.'
    • Start with an alphanumeric character
    • End with an alphanumeric character

    See ingressGateways[].name in the Configuration property reference

  • REPLICAS_MIN and REPLICAS_MAX are the minimum and maximum replica counts for Apigee ingress gateway in your installation. For more information and default settings, see ingressGateways[].replicaCountMin and ingressGateways[].replicaCountMax in the Configuration property reference.
  • CPU_COUNT_REQ and MEMORY_REQ are the CPU and memory request for each replica of Apigee ingress gateway in your installation.

    For more information and default settings, see ingressGateways[].resources.requests.cpu and ingressGateways[].resources.requests.memory in the Configuration property reference.

  • CPU_COUNT_LIMIT and MEMORY_LIMIT The maximum CPU and memory limits for each replica of Apigee ingress gateway in your installation.

    For more information and default settings, see ingressGateways[].resources.limits.cpu and ingressGateways[].resources.limits.memory in the Configuration property reference.

  • SVC_ANNOTATIONS_KEY and SVC_ANNOTATIONS_VALUE (optional):

    This is a key-value pair that provides annotations for your default ingress service. Annotations are used by your cloud platform to help configure your hybrid installation, for example setting the loadbalancer type to either internal or external. For example:

    ingressGateways:
      svcAnnotations:
        networking.gke.io/load-balancer-type: "Internal"

    Annotations vary from platform to platform. Refer to your platform documentation for required and suggested annotations.

    See ingressGateways[].svcAnnotations in the Configuration property reference.
  • SVC_LOAD_BALANCER_IP (optional) Allows you to assign a static IP address for your load balancer. On platforms that support specifying the load balancer IP address, the load balancer will be created with this IP address. On platforms that do not allow you to specify the load balancer IP address, this property is ignored.

    If you do not have a static IP address allocated for your load balancer, leave this property out of your overrides file.

    See ingressGateways[].svcLoadBalancerIP in the Configuration property reference.

Make additional changes to your overrides file to enable or disable optional v1.8 features

Add the following properties to your overrides.yaml file to enable new features in hybrid v1.8. These features are optional.

  • Org-scoped UDCA is now on by default. Using a single UDCA deployment to handle traffic for all environments prevents under-utilization of UDCA pods and increases availability of node resources for other Apigee components. Org-scoped UDCA uses a single service account for all environments, apigee-udca.

    If you are using different service accounts for UDCA in different environments, be aware that it will now use the service account specified at the org level in your overrides file with udca:serviceAccountPath, instead of the ones specified at the env level with envs:udca:serviceAccountPath.

    Apigee hybrid v 1.8 supports environment-scoped UDCA. To keep per-environment UDCA, set orgScopedUDCA: false.

    See orgScopedUDCA in the Configuration properties reference.

  • Enable validateOrg to require strict validation that the Apigee organization and environment are active and work with the Google Cloud Platform project specified in your overrides file.
    validateOrg: true

    See validateOrg in the Configuration properties reference.

Install the hybrid 1.8.8 runtime

  1. Be sure you are in the hybrid base directory (the parent of the directory where the apigeectl executable file is located):
    cd $APIGEECTL_HOME/..
  2. Download the release package for your operating system using the following command. Be sure to select your platform in the following table:

    Linux

    Linux 64 bit:

    curl -LO \
      https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.8.8/apigeectl_linux_64.tar.gz

    Mac OS

    Mac 64 bit:

    curl -LO \
      https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.8.8/apigeectl_mac_64.tar.gz

    Windows

    Windows 64 bit:

    curl -LO ^
      https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.8.8/apigeectl_windows_64.zip
  3. Rename your current apigeectl/ directory to a backup directory name. For example:

    Linux

    mv $APIGEECTL_HOME/ $APIGEECTL_HOME-v1.7/

    Mac OS

    mv $APIGEECTL_HOME/ $APIGEECTL_HOME-v1.7/ 

    Windows

    rename %APIGEECTL_HOME% %APIGEECTL_HOME%-v1.7 
  4. Extract the downloaded gzip file contents into your hybrid base directory. The hybrid base directory is the directory where the renamed apigeectl-v1.7 directory is located:

    Linux

    tar xvzf filename.tar.gz -C ./

    Mac OS

    tar xvzf filename.tar.gz -C ./

    Windows

    tar xvzf filename.zip -C ./
  5. The tar contents are, by default, expanded into a directory with the version and platform in its name. For example: ./apigeectl_1.8.8-xxxxxxx_linux_64. Rename that directory to apigeectl using the following command:

    Linux

    mv apigeectl_1.8.8-xxxxxxx_linux_64 apigeectl

    Mac OS

    mv apigeectl_1.8.8-xxxxxxx_mac_64 apigeectl

    Windows

    rename apigeectl_1.8.8-xxxxxxx_windows_64 apigeectl
  6. Change to the apigeectl directory:
    cd ./apigeectl

    This directory is the apigeectl home directory. It is where the apigeectl executable command is located.

  7. These instructions use the environment variable $APIGEECTL_HOME for the directory in your file system where the apigeectl utility is installed. If needed, change directory into your apigeectl directory and define the variable with the following command:

    Linux

    export APIGEECTL_HOME=$PWD
    echo $APIGEECTL_HOME

    Mac OS

    export APIGEECTL_HOME=$PWD
    echo $APIGEECTL_HOME

    Windows

    set APIGEECTL_HOME=%CD%
    echo %APIGEECTL_HOME%
  8. Verify the version of apigeectl with the version command:
    ./apigeectl version
    Version: 1.8.8
  9. Move to the hybrid-base-directory/hybrid-files directory. The hybrid-files directory is where configuration files such as the overrides file, certs, and service accounts are located. For example:
    cd $APIGEECTL_HOME/../hybrid-files
  10. Verify that kubectl is set to the correct context using the following command. The current context should be set to the cluster in which you are upgrading Apigee hybrid.
    kubectl config get-contexts | grep \*
  11. In the hybrid-files directory:
    1. Update the following symbolic links to $APIGEECTL_HOME. These links allow you to run the newly installed apigeectl command from inside the hybrid-files directory:
      ln -nfs $APIGEECTL_HOME/tools tools
      ln -nfs $APIGEECTL_HOME/config config
      ln -nfs $APIGEECTL_HOME/templates templates
      ln -nfs $APIGEECTL_HOME/plugins plugins
    2. To check that the symlinks were created correctly, execute the following command and make sure the link paths point to the correct locations:
      ls -l | grep ^l
  12. Do a dry run initialization to check for errors:
    ${APIGEECTL_HOME}/apigeectl init -f OVERRIDES_FILE --dry-run=client

    Where OVERRIDES_FILE is the name of your overrides file, for example ./overrides/overrides.yaml.

  13. If there are no errors, initialize hybrid 1.8.8. This command also installs and configures Apigee ingress gateway:
    $APIGEECTL_HOME/apigeectl init -f OVERRIDES_FILE
  14. Check the initialization status:
    $APIGEECTL_HOME/apigeectl check-ready -f OVERRIDES_FILE

    On success, the output says: All containers ready.

    As a further check, you can also run this command to check ApigeeDataStore status:

    kubectl describe apigeeds -n apigee

    In the output, look for State: running.

  15. Check for errors with a dry run of the apply command using the --dry-run flag:
    $APIGEECTL_HOME/apigeectl apply -f OVERRIDES_FILE --dry-run=client
  16. If there are no errors, apply your overrides. Select and follow the instructions for production environments or non-prod environments, depending on your installation.

    Production

    For production environments you should upgrade each hybrid component individually, and check the status of the upgraded component before proceeding to the next component.

    1. Be sure you are in the hybrid-files directory.
    2. Apply your overrides to upgrade Cassandra:
      $APIGEECTL_HOME/apigeectl apply -f OVERRIDES_FILE --datastore
    3. Check completion:
      $APIGEECTL_HOME/apigeectl check-ready -f OVERRIDES_FILE

      Proceed to the next step only when the pods are ready.

    4. Apply your overrides to upgrad