This procedure covers upgrading from Apigee hybrid version 1.9.x to Apigee hybrid version 1.10.5 and from previous releases of hybrid 1.10.x to version 1.10.5.
Use the same procedures for minor version upgrades (for example version 1.9 to 1.10) and for patch release upgrades (for example 1.10.0 to 1.10.5).
Upgrading to version 1.10.5 overview
The procedures for upgrading Apigee hybrid are organized in the following sections:
Prerequisites
These upgrade instructions assume you have Apigee hybrid version 1.9.x installed and wish to upgrade it to version 1.10.5. If you are updating from an earlier version, see the instructions for Upgrading Apigee hybrid to version 1.9.
Prepare to upgrade to version 1.10
Back up your hybrid installation (recommended)
- These instructions use the environment variable APIGEECTL_HOME for the directory
in your file system where you have installed
apigeectl
. If needed, change directory into yourapigeectl
directory and define the variable with the following command:export APIGEECTL_HOME=$PWD
echo $APIGEECTL_HOME
export APIGEECTL_HOME=$PWD
echo $APIGEECTL_HOME
set APIGEECTL_HOME=%CD%
echo %APIGEECTL_HOME%
- Make a backup copy of your version 1.9
$APIGEECTL_HOME/
directory. For example:tar -czvf $APIGEECTL_HOME/../apigeectl-v1.9-backup.tar.gz $APIGEECTL_HOME
- Back up your Cassandra database following the instructions in Cassandra backup and recovery.
Upgrade your Kubernetes version
Check your Kubernetes platform version and if needed, upgrade your Kubernetes platform to a version that is supported by both hybrid 1.9 and hybrid 1.10. Follow your platform's documentation if you need help.
Click to expand a list of supported platforms
1.10
|
1.11 | 1.12 | |||
---|---|---|---|---|---|
GKE on Google Cloud | 1.24.x
1.25.x 1.26.x 1.27.x(≥ 1.10.5) 1.28.x(≥ 1.10.5) |
1.25.x
1.26.x 1.27.x 1.28.x(≥ 1.11.2) 1.29.x(≥ 1.11.2) |
1.26.x
1.27.x 1.28.x 1.29.x(≥ 1.12.1) |
||
GKE on AWS | 1.13.x (K8s v1.24.x)
1.14.x (K8s v1.25.x) 1.26.x(12) 1.27.x(≥ 1.10.5) 1.28.x(≥ 1.10.5) |
1.14.x (K8s v1.25.x)
1.26.x(12) 1.27.x 1.28.x(≥ 1.11.2) 1.29.x(≥ 1.11.2) |
1.26.x(12)
1.27.x 1.28.x 1.29.x(≥ 1.12.1) |
||
GKE on Azure | 1.13.x
1.14.x 1.26.x(12) 1.27.x(≥ 1.10.5) 1.28.x(≥ 1.10.5) |
1.14.x
1.26.x(12) 1.27.x 1.28.x(≥ 1.11.2) 1.29.x(≥ 1.11.2) |
1.26.x(12)
1.27.x 1.28.x 1.29.x(≥ 1.12.1) |
||
Google Distributed Cloud (software only) on VMware (1) (13) | 1.13.x
1.14.x 1.15.x (K8s v1.26.x) 1.16.x (K8s v1.27.x) 1.27.x(≥ 1.10.5) 1.28.x(≥ 1.10.5) |
1.14.x
1.15.x (K8s v1.26.x) 1.16.x (K8s v1.27.x) 1.28.x(≥ 1.11.2) 1.29.x(≥ 1.11.2) |
1.15.x (K8s v1.26.x)
1.16.x (K8s v1.27.x) 1.28.x(12) 1.29.x(≥ 1.12.1) |
||
Google Distributed Cloud (software only) on bare metal (1) | 1.13.x
1.14.x (K8s v1.25.x) 1.15.x (K8s v1.26.x) 1.16.x (K8s v1.27.x) 1.27.x(12)(≥ 1.10.5) 1.28.x(≥ 1.10.5) |
1.14.x
1.15.x (K8s v1.26.x) 1.16.x (K8s v1.27.x) 1.28.x(12)(≥ 1.11.2) 1.29.x(≥ 1.11.2) |
1.15.x (K8s v1.26.x)
1.16.x (K8s v1.27.x) 1.28.x(12) 1.29.x(≥ 1.12.1) |
||
EKS(7) | 1.24.x
1.25.x 1.26.x 1.27.x(≥ 1.10.5) 1.28.x(≥ 1.10.5) |
1.25.x
1.26.x 1.27.x 1.28.x(≥ 1.11.2) 1.29.x(≥ 1.11.2) |
1.26.x
1.27.x 1.28.x 1.29.x(≥ 1.12.1) |
||
AKS(7) | 1.24.x
1.25.x 1.26.x 1.27.x(≥ 1.10.5) 1.28.x(≥ 1.10.5) |
1.25.x
1.26.x 1.27.x 1.28.x(≥ 1.11.2) 1.29.x(≥ 1.11.2) |
1.26.x
1.27.x 1.28.x 1.29.x(≥ 1.12.1) |
||
OpenShift(7) | 4.11
4.12 4.14(≥ 1.10.5) 4.15(≥ 1.10.5) |
4.12
4.13 4.14 4.15(≥ 1.11.2) 4.16(≥ 1.11.2) |
4.12
4.13 4.14 4.15 4.16(≥ 1.12.1) |
||
Rancher Kubernetes Engine (RKE) |
v1.26.2+rke2r1
1.27.x(≥ 1.10.5) 1.28.x(≥ 1.10.5) |
v1.26.2+rke2r1
v1.27.x 1.28.x(≥ 1.11.2) 1.29.x(≥ 1.11.2) |
v1.26.2+rke2r1 v1.27.x 1.28.x 1.29.x(≥ 1.12.1) |
||
VMware Tanzu | N/A | N/A | v1.26.x | ||
Components |
1.10 | 1.11 | 1.12 | ||
Anthos Service Mesh | 1.17.x(10) | 1.18.x(10) | 1.19.x(10) | ||
JDK | JDK 11 | JDK 11 | JDK 11 | ||
cert-manager |
1.10.x 1.11.x 1.12.x |
1.11.x 1.12.x 1.13.x |
1.11.x 1.12.x 1.13.x |
||
Cassandra | 3.11 | 3.11 | 4.0 | ||
Kubernetes | 1.24.x 1.25.x 1.26.x |
1.25.x 1.26.x 1.27.x |
1.26.x 1.27.x 1.28.x |
||
kubectl | 1.27.x 1.28.x 1.29.x |
||||
Helm | 3.10+ | 3.10+ | 3.14.2+ | ||
Secret Store CSI driver | 1.3.4 | 1.4.1 | |||
Vault | 1.13.x | 1.15.2 | |||
(1) On Anthos on-premises (Google Distributed Cloud) version 1.13, follow these instructions to avoid conflict with (2) Support available with Apigee hybrid version 1.7.2 and newer. (3) The official EOL dates for Apigee hybrid versions 1.10 and older have been reached. Regular monthly patches are no longer available. These releases are no longer officially supported except for customers with explicit and official exceptions for continued support. Other customers must upgrade. (4)Anthos on-premises (Google Distributed Cloud) versions 1.12 and earlier are out of support. See the Distributed Cloud on bare metal Version Support Policy and the Distributed Cloud on VMware supported versions list. (5)Google Distributed Cloud on bare metal or VMware requires Anthos Service Mesh 1.14 or later. We recommend that you upgrade to hybrid v1.8 and switch to Apigee ingress gateway which no longer requires you to install Anthos Service Mesh on your hybrid cluster. (6) Support available with Apigee hybrid version 1.8.4 and newer. (7) Not supported with Apigee hybrid version 1.8.4 and newer. (8) Support available with Apigee hybrid version 1.7.6 and newer. (9) Not supported with Apigee hybrid version 1.8.5 and newer. (10) Anthos Service Mesh is automatically installed with Apigee hybrid 1.9 and newer. (11) Support available with Apigee hybrid version 1.9.2 and newer. (12) GKE on AWS version numbers now reflect the Kubernetes versions. See GKE Enterprise version and upgrade support for version details and recommended patches. (13) Vault is not certified on Google Distributed Cloud for VMware. (14) Support available with Apigee hybrid version 1.10.5 and newer. (15) Support available with Apigee hybrid version 1.11.2 and newer. (16) Support available with Apigee hybrid version 1.12.1 and newer. |
About attached clusters
For Apigee hybrid versions 1.7.x and older, you must use GKE attached clusters if you want to run Apigee hybrid in a multi-cloud context on Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), or another supported third-party Kubernetes service provider. Cluster attachment allows Google to measure the usage of Anthos Service Mesh.
For Apigee hybrid version 1.8.x, GKE attached clusters are required if you are using Anthos Service Mesh for your ingress gateway. If you are using Apigee ingress gateway, attached clusters are optional.
Install the hybrid 1.10.5 runtime
- Be sure you are in the hybrid base directory (the parent of the directory where
the
apigeectl
executable file is located):cd $APIGEECTL_HOME/..
-
Download the release package for your operating system using the following command. Be sure to select your platform in the following table:
Linux 64 bit:
curl -LO \ https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.10.5/apigeectl_linux_64.tar.gz
Mac 64 bit:
curl -LO \ https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.10.5/apigeectl_mac_64.tar.gz
Windows 64 bit:
curl -LO ^ https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.10.5/apigeectl_windows_64.zip
- Rename your current
apigeectl/
directory to a backup directory name. For example:mv $APIGEECTL_HOME/ $APIGEECTL_HOME-v1.9/
mv $APIGEECTL_HOME/ $APIGEECTL_HOME-v1.9/
rename %APIGEECTL_HOME% %APIGEECTL_HOME%-v1.9
-
Extract the downloaded gzip file contents into your hybrid base directory. The hybrid base directory is the directory where the renamed
apigeectl-v1.9
directory is located:tar xvzf
filename .tar.gz -C ./tar xvzf
filename .tar.gz -C ./tar xvzf
filename .zip -C ./ -
The tar contents are, by default, expanded into a directory with the version and platform in its name. For example:
./apigeectl_1.10.5-xxxxxxx_linux_64
. Rename that directory toapigeectl
using the following command:mv
apigeectl_1.10.5-xxxxxxx_linux_64 apigeectlmv
apigeectl_1.10.5-xxxxxxx_mac_64 apigeectlrename
apigeectl_1.10.5-xxxxxxx_windows_64 apigeectl -
Change to the
apigeectl
directory:cd ./apigeectl
This directory is the
apigeectl
home directory. It is where theapigeectl
executable command is located. - These instructions use the environment variable
$APIGEECTL_HOME
for the directory in your file system where theapigeectl
utility is installed. If needed, change directory into yourapigeectl
directory and define the variable with the following command:export APIGEECTL_HOME=$PWD
echo $APIGEECTL_HOME
export APIGEECTL_HOME=$PWD
echo $APIGEECTL_HOME
set APIGEECTL_HOME=%CD%
echo %APIGEECTL_HOME%
- Verify the version of
apigeectl
with theversion
command:./apigeectl version
Version: 1.10.5
- Create a
hybrid-base-directory/hybrid-files
directory, and then move into it. Thehybrid-files
directory is where configuration files such as the overrides file, certs, and service accounts are located. For example:mkdir $APIGEECTL_HOME/../hybrid-files
cd $APIGEECTL_HOME/../hybrid-files
mkdir $APIGEECTL_HOME/../hybrid-files
cd $APIGEECTL_HOME/../hybrid-files
mkdir %APIGEECTL_HOME%/../hybrid-files
cd %APIGEECTL_HOME%/../hybrid-files
- Verify that
kubectl
is set to the correct context using the following command. The current context should be set to the cluster in which you are upgrading Apigee hybrid.kubectl config get-contexts | grep \*
- In the
hybrid-files
directory:-
Update the following symbolic links to
$APIGEECTL_HOME
. These links allow you to run the newly installedapigeectl
command from inside thehybrid-files
directory:ln -nfs
$APIGEECTL_HOME
/tools toolsln -nfs
$APIGEECTL_HOME
/config configln -nfs
$APIGEECTL_HOME
/templates templatesln -nfs
$APIGEECTL_HOME
/plugins plugins -
To check that the symlinks were created correctly, execute the following command and make
sure the link paths point to the correct locations:
ls -l | grep ^l
-
Update the following symbolic links to
- Make the following change to your overrides.yaml file to enable the
apigee-operator
chart ot use the correct tag,1.10.5-hotfix.1
:ao: image: url: "gcr.io/apigee-release/hybrid/apigee-operators" tag: "1.10.5-hotfix.1"
- Do a dry run initialization to check for errors:
${APIGEECTL_HOME}/apigeectl init -f
OVERRIDES_FILE --dry-run=clientWhere OVERRIDES_FILE is the name of your overrides file, for example
./overrides/overrides.yaml
. - If there are no errors, initialize hybrid 1.10.5:
$APIGEECTL_HOME/apigeectl init -f
OVERRIDES_FILE - Check the initialization status:
$APIGEECTL_HOME/apigeectl check-ready -f
OVERRIDES_FILE On success, the output says:
All containers ready.
kubectl describe apigeeds -n apigee
In the output, look for
State: running
. - Check for errors with a dry run of the
apply
command using the--dry-run
flag:$APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE --dry-run=client - If there are no errors, apply your overrides. Select and follow the instructions for production environments or
non-prod environments, depending on your installation.
For production environments, upgrade each hybrid component individually, and check the status of the upgraded component before proceeding to the next component.
- Be sure you are in the
hybrid-files
directory. - Apply your overrides to upgrade Cassandra:
$APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE --datastore - Check completion:
$APIGEECTL_HOME/apigeectl check-ready -f
OVERRIDES_FILE Proceed to the next step only when the pods are ready.
- Apply your overrides to upgrade Telemetry components and check completion:
$APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE --telemetry$APIGEECTL_HOME/apigeectl check-ready -f
OVERRIDES_FILE - Bring up Redis components:
$APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE --redis - Apply your overrides to upgrade the org-level components (MART, Watcher and Apigee
Connect) and check completion:
$APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE --org$APIGEECTL_HOME/apigeectl check-ready -f
OVERRIDES_FILE - Apply your overrides to upgrade your environments. You have two choices:
- Environment by environment: Apply your overrides to one environment at a time and check completion. Repeat
this step for each environment:
$APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE --envENV_NAME $APIGEECTL_HOME/apigeectl check-ready -f
OVERRIDES_FILE Where ENV_NAME is the name of the environment you are upgrading.
- All environments at one time: Apply your overrides to all environments at once and check completion:
$APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE --all-envs$APIGEECTL_HOME/apigeectl check-ready -f
OVERRIDES_FILE
- Environment by environment: Apply your overrides to one environment at a time and check completion. Repeat
this step for each environment:
- Apply your overrides to upgrade the
virtualhosts
components and check completion:$APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE --settings virtualhosts$APIGEECTL_HOME/apigeectl check-ready -f
OVERRIDES_FILE
In most non-production, demo, or experimental environments, you can apply the overrides to all components at once. If your non-production environment is large and complex or closely mimics a production environment, you may want to use the instructions for upgrading production environments.
- Be sure you are in the
hybrid-files
directory. $APIGEECTL_HOME/apigeectl apply -f
OVERRIDES_FILE - Check the status:
$APIGEECTL_HOME/apigeectl check-ready -f
OVERRIDES_FILE
- Be sure you are in the
Rolling back an upgrade
Follow these steps to roll back a previous upgrade:
- Clean up completed jobs for the hybrid runtime namespace, where NAMESPACE is the
namespace specified in your overrides file, if you specified a namespace. If not, the default namespace
is
apigee
:kubectl delete job -n
NAMESPACE \ $(kubectl get job -nNAMESPACE \ -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}') - Clean up completed jobs for the
apigee-system
namespace:kubectl delete job -n apigee-system \ $(kubectl get job -n apigee-system \ -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}')
- Change the
APIGEECTL_HOME
variable to point to the directory that contains the previous version ofapigeectl
. For example:export APIGEECTL_HOME=
PATH_TO_PREVIOUS_APIGEECTL_DIRECTORY - In the root directory of the installation you want to roll back to, run
apigeectl apply
, check the status of your pods, and then runapigeectl init
. Be sure to use the original overrides file for the version you wish to roll back to:- In the hybrid-files directory, run
apigeectl apply
:$APIGEECTL_HOME
/apigeectl apply -fORIGINAL_OVERRIDES_FILE Where ORIGINAL_OVERRIDES_FILE is the relative path and filename of the overrides file for your previous version hybrid installation, for example,
./overrides/overrides1.9.yaml
. - Check the status of your pods:
kubectl -n
NAMESPACE get podsWhere NAMESPACE is your Apigee hybrid namespace.
- Check the status of
apigeeds
:kubectl describe apigeeds -n apigee
Your output should look something like:
Status: Cassandra Data Replication: Cassandra Pod Ips: 10.8.2.204 Cassandra Ready Replicas: 1 Components: Cassandra: Last Successfully Released Version: Revision: v1-f8aa9a82b9f69613 Version: v1 Replicas: Available: 1 Ready: 1 Total: 1 Updated: 1 State: running Scaling: In Progress: false Operation: Requested Replicas: 0 State: running
Proceed to the next step only when the
apigeeds
pod is running. - Run the following command to make note of what your new replica count values will be for
the message processor after the upgrade. If these values do not match what you have set
previously, change the values in your overrides file to match your previous
configuration.
apigeectl apply -f
ORIGINAL_OVERRIDES_FILE --dry-run=client --print-yaml --env ENV_NAME 2>/dev/null |grep "runtime:" -A 25 -B 1| grep "autoScaler" -A 2Your output should look something like:
autoScaler: minReplicas: 2 maxReplicas: 10
- Run
apigeectl init
:$APIGEECTL_HOME
/apigeectl init -fORIGINAL_OVERRIDES_FILE
- In the hybrid-files directory, run