Restoring a single region deployment

This page describes how to restore Cassandra in a single region.

In a single region deployment, Apigee hybrid is deployed in a single data center or a region. If you have multiple Apigee organizations in your deployment, the restore process restores data for all the organizations. In a multi-organization setup, you cannot restore a specific organization.

Note: Before you begin restoring a single region, consider whether the following prerequisite steps are applicable:

If you want to preserve an existing setup for troubleshooting and root cause analysis (RCA), you should delete all the org and env components from the Kubernetes cluster except the Apigee controller, and then retain the cluster. The cluster will contain the existing Apigee datastore (Cassandra) which you can use for troubleshooting. Create a new Kubernetes cluster and then restore Cassandra in the new cluster.
If your hybrid installation was set up with multiple organizations, get the overrides files for each organization before proceeding with restore in a single region. You can add the restore configuration as described in Step 3 to any one of the overrides files. Do not add the restore configuration to any other overrides file.

Restoring a region from a backup

In your configuration, the Cassandra backup can reside either on Cloud Storage or on a remote server. In either case, perform the following steps to restore:

Verify the hybrid version.
```
apigeectl version
```
Ensure the version is the same version that created the backup files in storage.
Tip: Run every apigeectl command with the --dry-run=true flag first to check for any apigeectl errors.
Confirm that the Kubernetes cluster you are restoring to does not have a prior Apigee hybrid installation. If you are restoring to the existing cluster, use the following command to delete the existing Apigee hybrid installation:
```
apigeectl delete --all -f overrides.yaml
```

Open your overrides.yaml file and set the restore properties to the desired values:

Parameters

namespace: YOUR_RESTORE_NAMESPACE # Use the namespace as in your original cluster.
cassandra:
  ...
  restore:
    enabled: true
    keyFile: "PATH_TO_PRIVATE_KEY_FILE"
    server: "BACKUP_SERVER_IP"
    storageDirectory: "/home/apigee/BACKUP_DIRECTORY"
    cloudProvider: "HYBRID"  # required verbatim "HYBRID" (all caps)
    snapshotTimestamp: "TIMESTAMP"
  ...
  backup:
    enabled: true
    keyFile: "PATH_TO_PRIVATE_KEY_FILE"
    server: "BACKUP_SERVER_IP"
    storageDirectory: "/home/apigee/BACKUP_DIRECTORY"
    cloudProvider: "HYBRID" # required verbatim "HYBRID" (all caps)
    schedule: "SCHEDULE"

Example

namespace: apigee
cassandra:
...
  restore:
    enabled: true
    keyFile: "/Users/exampleuser/apigee-hybrid/hybrid-files/service-accounts/private.key"
    server: "34.56.78.90"
    storageDirectory: "/home/apigee/cassbackup"
    cloudProvider: "HYBRID"
    snapshotTimestamp: "20201001183903"
  ...
  backup:
    enabled: true
    keyFile: "/Users/exampleuser/apigee-hybrid/hybrid-files/service-accounts/private.key"
    server: "34.56.78.90"
    storageDirectory: "/home/apigee/cassbackup"
    cloudProvider: "HYBRID"
    schedule: "0 2 * * *"
  ...

Where:

Property	Description
`namespace`	`YOUR_RESTORE_NAMESPACE` Namespace for restore. Use the namespace as in your original cluster.
`restore:enabled`	Restore is disabled by default. You must set this property to `true`.
`restore:keyFile`	`PATH_TO_PRIVATE_KEY_FILE` The path on your local file system to the SSH private key file (named `ssh_key` in the step where you created the SSH key pair).
`restore:server`	`BACKUP_SERVER_IP` The IP address of your backup server.
`restore:storageDirectory`	`BACKUP_DIRECTORY` The name of the backup directory on your backup server. This must be a directory within `home/apigee` (the backup directory is named `cassandra_backup` in the step where you created the backup directory).
`restore:cloudProvider`	`HYBRID` The `cloudProvider: "HYBRID"` property is required.
`restore:snapshotTimestamp`	`TIMESTAMP` The timestamp of the backup snapshot to restore. To check what timestamps can be used, go to the `dbStorageBucket` and look at the files that are present in the bucket. Each file name contains a timestamp value. For example, `backup_20210203213003_apigee-cassandra-default-0.tgz` Where `20210203213003` is the `snapshotTimestamp` value you would use if you wanted to restore the backups created at that point in time.
`backup:enabled`	Backup is disabled by default. You must set this property to `true`.
`backup:keyFile`	`PATH_TO_PRIVATE_KEY_FILE` The path on your local file system to the SSH private key file (named `ssh_key` in the step where you created the SSH key pair).
`backup:server`	`BACKUP_SERVER_IP` The IP address of your backup server.
`backup:storageDirectory`	`BACKUP_DIRECTORY` The name of the backup directory on your backup server. This must be a directory within `home/apigee` (the backup directory is named `cassandra_backup` in the step where you created the backup directory).
`backup:cloudProvider`	`HYBRID` The `cloudProvider: "HYBRID"` property is required.
`backup:schedule`	`SCHEDULE` The time when the backup starts, specified in standard crontab syntax. Default: `0 2 * * ` Note:* Avoid scheduling a backup that starts a short time after you apply the backup configuration to your cluster. When you apply the backup configuration, Kubernetes recreates the Cassandra nodes. If the backup starts before the nodes restart (possibly several minutes) the backup will fail.

Note: Apigee restores Cassandra from a Cloud Storage backup or a remote server backup based on the backup configuration in the overrides.yaml file.

Create a new hybrid runtime deployment. This will create a new Cassandra cluster and begin restoring the backup data into the cluster:
```
${APIGEECTL_HOME}/apigeectl init  -f overrides/overrides.yaml
```
```
${APIGEECTL_HOME}/apigeectl check-ready -f overrides/overrides.yaml
```
Note: The --restore flag was introduced in hybrid v1.6.6. If you are on an earlier version, execute the following command without the --restore flag.
```
${APIGEECTL_HOME}/apigeectl apply -f overrides/overrides.yaml --restore
```
```
${APIGEECTL_HOME}/apigeectl check-ready -f overrides/overrides.yaml
```
Verify the restoration job progress and confirm that apigeeds and all the other pods are up:
- To check apigeeds:
```
kubectl get apigeeds -n apigee
```
- To check all other pods:
```
kubectl get pods -n apigee
```

Upon successful completion of the restore and confirmation that the runtime components are healthy, we recommend configuring a backup on the cluster:

Remove the restore configuration from the overrides-restore.yaml file.
Add the backup configuration to the overrides-restore.yaml file.
Apply the backup configuration with the following command:
```
./apigeectl apply -f ../overrides-restore.yaml
```