本部分讨论如何为在 Apigee 混合运行时平面中安装的 Apache Cassandra 数据库环配置数据备份和恢复。另请参阅 Cassandra 数据库。
关于 Cassandra 备份,您需要了解哪些信息?
Cassandra 是复制的数据库,配置为在每个地区或数据中心至少有 3 个数据副本。Cassandra 使用流式复制和读取修复来维护在每个地区或数据中心中给定时间点的数据副本。
在混合环境中,Cassandra 备份默认不启用。不过,最好启用 Cassandra 备份,以防数据被意外删除。
备份了哪些内容?
本主题中介绍的备份配置可备份以下实体:
- Cassandra 架构,包括用户架构(Apigee 密钥空间定义)
- 每个节点的 Cassandra 分区令牌信息
- Cassandra 数据的快照
备份数据存储在何处?
备份数据会存储在您必须创建的 Google Cloud Storage (GCS) 存储分区中。本主题涵盖创建和配置存储分区。
安排 Cassandra 备份
备份作为运行时平面中的 Cron 作业进行安排。如需安排 Cassandra 备份,请执行以下操作:
- 运行以下
create-service-account
命令以创建具有标准roles/storage.objectAdmin
角色的 GCP 服务账号 (SA)。此 SA 角色允许您将备份数据写入 Google Cloud Storage (GCS)。在 Hybrid 安装根目录下执行以下命令:./tools/create-service-account apigee-cassandra output-dir
例如:./tools/create-service-account apigee-cassandra ./service-accounts
如需详细了解 GCP 服务账号,请参阅创建和管理服务账号。 create-service-account
命令会保存包含服务账号私钥的 JSON 文件。该文件会保存在执行命令的目录中。执行以下步骤将需要此文件的路径。- 创建 GCS 存储分区。为存储分区指定合理的数据保留政策。Apigee 建议采用 15 天的数据保留政策。
- 打开您的
overrides.yaml
文件。 - 添加以下
cassandra.backup
属性以启用备份。请勿移除任何已配置的属性。cassandra: ... backup: enabled: true serviceAccountPath: sa_json_file_path dbStorageBucket: gcs_bucket_path schedule: backup_schedule_code ...
其中:属性 说明 enabled
备份默认处于停用状态。您必须将此属性设为 true
serviceAccountPath
文件系统上指向运行 ./tools/create-service-account
时所下载的服务账号 JSON 文件的路径dbStorageBucket
GCS 存储分区路径采用以下格式: gs://bucket_name
。gs://
是必需的。schedule
备份的开始时间,使用标准 crontab 语法指定。默认: 0 2 * * *
注意:避免将备份计划为在将备份配置应用于集群后很短时间内开始。应用备份配置时,Kubernetes 会重新创建 Cassandra 节点。如果备份在节点重启(可能需要几分钟)之前开始,则备份将失败。
... cassandra: storage: type: gcepd capacity: 50Gi gcepd: replicationType: regional-pd sslRootCAPath: "/Users/myhome/ssh/cassandra.crt" sslCertPath: "/Users/myhome/ssh/cassandra.crt" sslKeyPath: "/Users/myhome/ssh/cassandra.key" auth: default: password: "abc123" admin: password: "abc234" ddl: password: "abc345" dml: password: "abc456" nodeSelector: key: cloud.google.com/gke-nodepool value: apigee-data backup: enabled: true serviceAccountPath: "/Users/myhome/.ssh/my_cassandra_backup.json" dbStorageBucket: "gs://myname-cassandra-backup" schedule: "45 23 * * 6" ...
- 将配置更改应用到新集群。例如:
./apigeectl apply -c cassandra -f my-overrides.yaml
恢复备份
恢复会从备份位置获取数据,并将其恢复到具有相同 pod 数的新 Cassandra 集群中。新集群必须具有不同于运行时平面集群的命名空间。
如需恢复 Cassandra 备份,请执行以下操作:
- 使用新的命名空间创建新的 Kubernetes 集群。您不能使用用于原始混合安装的同一集群/命名空间。
- 在根混合安装目录中,创建新的
overrides-restore.yaml
文件。 - 将完整的 Cassandra 配置从原始
overrides.yaml
文件复制到新文件中。 - 添加命名空间元素。请勿使用您原始集群所用的命名空间。
- 创建新的 Cassandra 集群:
./apigeectl apply -c cassandra -f ./overrides-restore.yaml
namespace: your-restore-namespace cassandra: storage: type: gcepd capacity: 50Gi gcepd: replicationType: regional-pd nodeSelector: key: cloud.google.com/gke-nodepool value: apigee-data sslRootCAPath: path_to_root_ca_file sslCertPath: path_to_ssl_cert_file sslKeyPath: path_to_ssl_key_file auth: default: password: your_cassandra_password admin: password: admin_password ddl: password: ddl_password dml: password: dml_password restore: enabled: true snapshotTimestamp: timestamp serviceAccountPath: sa_json_file_path dbStorageBucket: gcs_bucket_path image: pullPolicy: Always其中:
属性 | 说明 |
---|---|
ssl*Path ,auth.* |
使用用于创建原始 Cassandra 数据库的相同 TLS 身份验证凭据。 |
snapshotTimestamp |
要恢复的备份快照的时间戳。 |
serviceAccountPath |
文件系统上指向为备份创建的服务账号的路径。 |
dbStorageBucket |
存储备份的 GCS 存储分区路径,格式为:gs://bucket_name 。gs:// 是必须的。 |
namespace: cassandra-restore cassandra: storage: type: gcepd capacity: 50Gi gcepd: replicationType: regional-pd sslRootCAPath: "/Users/myhome/ssh/cassandra.crt" sslCertPath: "/Users/myhome/ssh/cassandra.crt" sslKeyPath: "/Users/myhome/ssh/cassandra.key" auth: default: password: "abc123" admin: password: "abc234" ddl: password: "abc345" dml: password: "abc456" nodeSelector: key: cloud.google.com/gke-nodepool value: apigee-data restore: enabled: true snapshotTimestamp: "20190417002207" serviceAccountPath: "/Users/myhome/.ssh/my_cassandra_backup.json" dbStorageBucket: "gs://myname-cassandra-backup" image: pullPolicy: Always
其中,snapshotTimestamp
是与您要恢复的备份关联的时间戳。
查看恢复日志
您可以查看 error
的恢复作业日志和 grep 来确保恢复日志没有错误。
验证恢复已完成
如需检查恢复操作是否已完成,请执行以下操作:
kubectl get pods NAME READY STATUS RESTARTS AGE apigee-cassandra-0 1/1 Running 0 1h apigee-cassandra-1 1/1 Running 0 1h apigee-cassandra-2 1/1 Running 0 59m apigee-cassandra-restore-b4lgf 0/1 Completed 0 51m
查看恢复日志
如需查看恢复日志,请执行以下操作:
kubectl logs -f apigee-cassandra-restore-b4lgf Restore Logs: Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] to download file gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1/backup_20190405011309_schema.tgz INFO: download sucessfully extracted the backup files from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 finished downloading schema.cql to create schema from 10.32.0.28 Warnings : dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0 dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0 Warnings : dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0 dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0 INFO: the schema has been restored starting apigee-cassandra-0 in default starting apigee-cassandra-1 in default starting apigee-cassandra-2 in default 84 95 106 waiting on waiting nodes $pid to finish 84 Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] INFO: restore downloaded tarball and extracted the file from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO: restore downloaded tarball and extracted the file from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO: restore downloaded tarball and extracted the file from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO 12:02:28 Configuration location: file:/etc/cassandra/cassandra.yaml …... INFO 12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed Summary statistics: Connections per host : 3 Total files transferred : 2 Total bytes transferred : 0.378KiB Total duration : 5048 ms Average transfer rate : 0.074KiB/s Peak transfer rate : 0.075KiB/s progress: [/10.32.1.155]0:1/1 100% 1:1/1 100% [/10.32.0.28]1:1/1 100% 0:1/1 100% [/10.32.3.220]0:1/1 100% 1:1/1 100% total: 100% 0.000KiB/s (avg: 0.074KiB/s) INFO 12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed progress: [/10.32.1.155]0:1/1 100% 1:1/1 100% [/10.32.0.28]1:1/1 100% 0:1/1 100% [/10.32.3.220]0:1/1 100% 1:1/1 100% total: 100% 0.000KiB/s (avg: 0.074KiB/s) INFO 12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed INFO 12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed INFO: ./apigee/data/cassandra/data/ks1/user-9fbae960571411e99652c7b15b2db6cc restored successfully INFO: Restore 20190405011309 completed INFO: ./apigee/data/cassandra/data/ks1/user-9fbae960571411e99652c7b15b2db6cc restored successfully INFO: Restore 20190405011309 completed waiting on waiting nodes $pid to finish 106 Restore finished
验证备份作业
您还可以在安排备份 Cron 作业后验证备份作业。安排 Cron 作业后,您应该会看到类似如下的内容:
kubectl get pods NAME READY STATUS RESTARTS AGE apigee-cassandra-0 1/1 Running 0 2h apigee-cassandra-1 1/1 Running 0 2h apigee-cassandra-2 1/1 Running 0 2h apigee-cassandra-backup-1554515580-pff6s 0/1 Running 0 54s
检查备份日志
备份作业:
- 创建
schema.cql
文件。 - 将它上传到您的存储桶。
- 回应节点以同时备份数据和上传数据。
- 等待所有数据上传完毕。
kubectl logs -f apigee-cassandra-backup-1554515580-pff6s myusername-macbookpro:cassandra-backup-utility myusername$ kubectl logs -f apigee-cassandra-backup-1554577680-f9sc4 starting apigee-cassandra-0 in default starting apigee-cassandra-1 in default starting apigee-cassandra-2 in default 35 46 57 waiting on process 35 Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false} Snapshot directory: 20190406190808 INFO: backup created cassandra snapshot 20190406190808 tar: Removing leading `/' from member names /apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/ /apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/20190406190808/ /apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Data.db Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false} Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false} Snapshot directory: 20190406190808 INFO: backup created cassandra snapshot 20190406190808 tar: Removing leading `/' from member names /apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/ /apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/20190406190808/ /apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/ /apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/20190406190808/ /apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/ /apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/20190406190808/ /apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/ /apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/20190406190808/ /apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/ /apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/20190406190808/ /apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/20190406190808/manifest.json …… /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/ /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/ /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Filter.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-CompressionInfo.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Index.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Statistics.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Data.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Index.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Statistics.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-TOC.txt /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Statistics.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Summary.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Filter.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Summary.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Index.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Filter.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Digest.crc32 /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Summary.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Data.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-TOC.txt /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/schema.cql /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-CompressionInfo.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Digest.crc32 /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-TOC.txt /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Data.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Digest.crc32 /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-CompressionInfo.db …… /tmp/tokens.txt / [1 files][ 0.0 B/ 0.0 B] Operation completed over 1 objects. / [1 files][ 0.0 B/ 0.0 B] Operation completed over 1 objects. INFO: backup created tarball and transfered the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO: removing cassandra snapshot INFO: backup created tarball and transfered the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO: removing cassandra snapshot Requested clearing snapshot(s) for [all keyspaces] INFO: Backup 20190406190808 completed waiting on process 46 Requested clearing snapshot(s) for [all keyspaces] INFO: Backup 20190406190808 completed Requested clearing snapshot(s) for [all keyspaces] waiting on process 57 INFO: Backup 20190406190808 completed waiting result to get schema from 10.32.0.28 INFO: /tmp/schema.cql has been generated Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] tar: removing leading '/' from member names tmp/schema.cql Copying from... / [1 files][ 0.0 B/ 0.0 B] Operation completed over 1 objects. INFO: backup created tarball and transfered the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 finished uploading schema.cql