Apigee hybrid Helm migration tool

This migration tool assists in migrating an apigeectl-based hybrid cluster to a Helm-based hybrid cluster. This tool performs no actual replacement of any cluster components. It is idempotent and can be run many times on the same cluster, preparing a subset of components and orgs each time.

You can migrate all apigee components at one time, and Helm upgrade operations can be done on a per-component basis after the tool is run.

See Installing and managing Apigee hybrid with Helm charts for information about managing hybrid clusters you have migrated to Helm management with this tool.

Prerequisites

  • Helm version 3.10+.
  • A working kubeconfig file pointing to a cluster with a working Apigee hybrid 1.10 installation.
  • Permissions to modify the metadata and annotations on the Kubernetes resources of the hybrid components that you wish to migrate.

Scope

This tool supports the following options at run-time:

  • Customization of namespace for apigee resources. Default namespace: apigee
  • Migration of selected hybrid components only. Default: all components are migrated
  • Migration of a single org only
  • Migration of a single env only
  • Migration of a single env-group (apigee-virtualhost) only
  • Customization of Helm release names for orgs, envs, and env-groups.

Limitations

  • This tool does not support customizing Helm release names for these hybrid components: apigee-operator, apigee-datastore, apigee-redis, apigee-telemetry, and apigee-ingress-manager.
  • Interactive customizations made to the Helm release names for orgs, envs, and env-groups do not automatically persist between runs. You can edit the temp file and supply it as an option in later runs.
  • Env and env-group filtering is done by name only. In some cases this could result in multiple envs and env-groups being migrated on multi-org clusters.

    For example, on a multi-org cluster with orgs org1 and org2, if the env prod is present in both orgs, and only --env=prod is specified, both envs will be migrated. If you wish to only migrate a single env, you must also specify an org filter --org=org1 or --org=org2.

Usage

Syntax

apigee-helm-migration [--apigee-namespace=] [--components=] [--dry-run] [--env=org1] [--env-group=org2] [--org=baz] [--kubeconfig=] [-y] [-v] [-f /path/to/releaseNames.yaml]

Generated Helm release names

Every Helm Chart that is deployed on a cluster needs to have a release name, which must be unique within a namespace. Helm release names don't have any naming convention or restriction relative to the chart name. The migration tool generates unique Helm release names for every component.

Chart Single-org cluster Multi-org cluster
apigee-operator operator operator
apigee-datastore datastore datastore
apigee-telemetry telemetry telemetry
apigee-redis redis redis
apigee-ingress-manager ingress-manager ingress-manager
apigee-org ORG_NAME ORG_NAME
apigee-env ENV_NAME[-env[-n]](1) ORG_NAME-ENV_NAME[-env[-n]](1)
apigee-virtualhost (envgroup) VH_NAME[-env-group[-n]](1) ORG_NAME-VH_NAME[-env-group[-n]](1)

(1) Names are suffixed with -env or -env-group if the generated name conflicts with another generated name. They are further suffixed with -1 or -2 … if they still conflict.

Customizing Helm release names

The migration tool allows interactive customization of Helm release names. If you want to customize the Helm release names non-interactively:

  1. Run the tool once and exit at the first prompt to create a temporary file containing the auto-generated release names. You should see a line like:
    INFO: 21:32:56 using temp file for release names:  /tmp/apigee-helm-migration-1229129207-releaseNames
    
  2. Move or copy and then edit this file. You can pass this edited file in with the -f option when you run the migration tool. The auto-generated release names look like:

    orgs:
      example-apigee-org:
        helmReleaseName: example-apigee-org
        envs:
          prod:
            helmReleaseName: prod
        envGroups:
          prod-envgroup:
            helmReleaseName: prod-envgroup
    

    To customize the Helm release names for an org, env, or envgroup, edit the helmReleaseName field of that object. For example, to rename the org release to custom-org, the env release to custom-env, and the envgroup release to custom-group, the resulting file looks like:

    orgs:
      example-apigee-org:
        helmReleaseName: custom-org
        envs:
          prod:
            helmReleaseName: custom-env
        envGroups:
          prod-envgroup:
            helmReleaseName: custom-group
    

Using custom namespaces

Apigee hybrid runs in two Kubernetes namespaces:

  • apigee-system: The apigee-operator component always runs in the apigee-system namespace. The Helm migration tool will update the apigee-operator component in the apigee-system namespace regardless of what you specify with the --apigee-namespace flag.
  • apigee: All hybrid components except apigee-operator run in this namespace. apigee is the default name. You can use any custom namespace for these components.

    If you use a custom namespace, you must specify the it with the --apigee-namespace my_custom_namespace flag when you run the Helm migration tool.

    You must also add the namespace: my_custom_namespace top-level property to your overrides file.

Directions

  1. Download the migration tool.

    Linux

    1. Store the latest version number in a variable using the following command:
      export VERSION=$(curl -s \
        "https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/current-version.txt?ignoreCache=1")
    2. Check that the variable was populated with a version number using the following command. If you want to use a different version, you can save that in an environment variable instead.
      echo $VERSION

      For example:

      echo $VERSION
      1.0.5
    3. Download the release package for your operating system using the following command:

      curl -LO \
        https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/${VERSION}/apigee-helm-migration_linux_64.tar.gz
    4. Extract the compressed files using the following command:

      tar -xzf apigee-helm-migration_linux_64.tar.gz

    Mac OS

    1. Store the latest version number in a variable using the following command:
      export VERSION=$(curl -s \
        "https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/current-version.txt?ignoreCache=1")
    2. Check that the variable was populated with a version number using the following command. If you want to use a different version, you can save that in an environment variable instead.
      echo $VERSION

      For example:

      echo $VERSION
      1.0.5
    3. Download the release package for your operating system using the following command:

      curl -LO \
        https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/${VERSION}/apigee-helm-migration_mac_64.tar.gz
    4. Extract the compressed files using the following command:

      tar -xzf apigee-helm-migration_mac_64.tar.gz

    Windows

    1. Store the latest version number in a variable using the following command:
      for /f "tokens=*" %a in ('curl -s ^
        https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/current-version.txt') ^
        do set VERSION=%a
    2. Check that the variable was populated with a version number using the following command. If you want to use a different version, you can save that in an environment variable instead.
      echo %VERSION%

      For example:

      echo %VERSION%
      1.0.5
    3. Download the release package for your operating system using the following command:

      curl -LO ^
        https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/%VERSION%/apigee-helm-migration_windows_64.tar.gz
    4. Extract the compressed files using the following command:

      tar xzvf apigee-helm-migration_windows_64.tar.gz
  2. Run the migration tool. If the default options are acceptable, it is sufficient to run the tool without any arguments, and approve the prompt if the generated helm release names are satisfactory. Some example scenarios are given below:
    • A simple installation, using the default kubeconfig (~/.kube/config), default apigee namespace, and default Helm release names.

      The following command should be sufficient for most, if not all, installations. Helm upgrade operations can be done on a per-component basis after the tool is run.

      ./apigee-helm-migration
      
    • Migrating all components using a custom namespace:
      ./apigee-helm-migration --apigee-namespace my_custom_namespace
      
    • Migrating only the operator and datastore components:

      ./apigee-helm-migration --components operator,datastore
      
        INFO: 00:22:48 using kubeconfig file  /usr/local/google/home/example/.kube/config
        INFO: 00:22:48 namespace for apigee resources:
        INFO: 00:22:48 	 apigee
        INFO: 00:22:48 processing all organizations in cluster
        INFO: 00:22:48 Components to migrate:
        INFO: 00:22:48 	 operator,datastore
        INFO: 00:22:48 dry-run:
        INFO: 00:22:48 	 false
        Continue with patching apigee resources for Helm migration? [y/n]: y
        INFO: 00:22:52 Processing component:  operator
        INFO: 00:22:54 Processing component:  datastore
        INFO: 00:22:55 Migration successful!
      
    • Pointing to a specific kubeconfig file and specifying a different name for the apigee namespace.

      ./apigee-helm-migration --kubeconfig /abs/path/to/kubeconf --namespace org1_namespace
      
    • Migrating all components, but only a single org:

      ./apigee-helm-migration --org=some-test-org
      

    The following shows an example output from a successful migration:

    INFO: 21:32:55 using kubeconfig file  /usr/local/google/home/example/.kube/config
    INFO: 21:32:55 namespace for apigee resources:
    INFO: 21:32:55 	 apigee
    INFO: 21:32:55 processing all organizations in cluster
    INFO: 21:32:55 processing all components
    INFO: 21:32:55 dry-run:
    INFO: 21:32:55 	 false
    INFO: 21:32:55 cluster Apigee information:
    INFO: 21:32:55 Apigee Organizations found:
    INFO: 21:32:56 	 example-hybrid-dev
    INFO: 21:32:56 Apigee Environments found (org: env):
    INFO: 21:32:56 	 example-hybrid-dev : prod
    INFO: 21:32:56 Apigee EnvGroups(apigeerouteconfigs) found (org: envGroup):
    INFO: 21:32:56 	 example-hybrid-dev : prod-envgroup
    INFO: 21:32:56 using temp file for release names:  /tmp/apigee-helm-migration-1229129207-releaseNames
    INFO: 21:32:56 Helm release names for Apigee orgs/envs/envgroups:
    orgs:
    example-hybrid-dev:
    helmReleaseName: example-hybrid-dev
    envs:
      prod:
        helmReleaseName: prod
    envGroups:
      prod-envgroup:
        helmReleaseName: prod-envgroup
    Make changes to the release names for Apigee orgs/env/envgroups? [y/n]: n
    Continue with patching apigee resources for Helm migration? [y/n]: y
    INFO: 21:32:59 Processing component:  operator
    INFO: 21:33:01 Processing component:  datastore
    INFO: 21:33:01 Processing component:  redis
    INFO: 21:33:02 Processing component:  ingress-manager
    INFO: 21:33:02 Processing component:  telemetry
    INFO: 21:33:03 Processing component:  orgs
    INFO: 21:33:05 Processing component:  envs
    INFO: 21:33:06 Processing component:  env-groups
    INFO: 21:33:07 Migration successful!
    

    Potential errors:

    • Error parsing release names file: Check the passed in release names file.
    • Resources not found: Check that Apigee hybrid is fully installed, and you have permissions to access the apigee resources.

Next step

Continue your installation of the Apigee hybrid Helm charts with the instructions in Installing and managing Apigee hybrid with Helm charts.

Output of -help

./apigee-helm-migration --help
Usage of ./apigee-helm-migration:
  -apigee-namespace string
      namespace used for apigee resources (default "apigee")
  -components string
      CSV of components to migrate. If empty then all components are migrated. Valid values are: operator,datastore,redis,ingress-manager,telemetry,orgs,envs,env-groups
  -dry-run
      perform a dry-run
  -env string
      restrict migration to a singular supplied env. If empty then all envs detected in the cluster are migrated
  -env-group string
      restrict migration to a singular supplied envGroup. If empty then all envGroups detected in the cluster are migrated
  -kubeconfig string
      (optional) absolute path to the kubeconfig file (default "/usr/local/google/home/example/.kube/config")
  -org string
      restrict migration to a singular supplied org. If empty then all orgs detected in the cluster are migrated
  -v	Increased logging verbosity
  -y	don't prompt for confirmation or for configuration of Helm releases