Cloud Spanner Import and Export functionality helps you move data in bulk into or out of Cloud Spanner databases using Dataflow. You can export any Cloud Spanner database into a Cloud Storage bucket using either Avro or CSV file formats. You can also import data from Avro or CSV files into a new Cloud Spanner database.
Use cases of Import and Export functionality include:
- Bulk loading: You can use import functionality to load data in bulk into Cloud Spanner.
Long-term backup and archiving: You can export your database at any time and store it in a Cloud Storage bucket location of your choice for long-term backup or archiving. In addition, you can use point-in-time recovery to export a database from a specific past timestamp.
Note that Cloud Spanner also offers managed backup and restore for faster disaster-recovery use cases.
Copying databases to development or test projects: You can export a database from a production project and then import it into your development or test project to use for integration tests or other experiments.
Ingesting for analytics: You can use database export to ingest your operational data in bulk to analytics services such as BigQuery. BigQuery can automatically ingest data in Avro format from a Cloud Storage bucket, making it easier for you to run analytics on your operational data.
Note that if you want to use BigQuery for real-time analysis of Cloud Spanner data without the need to copy or move that data, you can use Cloud Spanner federated queries instead.
File format comparison
The following table compares the capability differences between Avro and CSV file formats when importing and exporting Cloud Spanner data.
For detailed instructions on importing from or exporting to these formats, including information on limitations that may apply, please see What's Next, below.
|Capability||Avro format||CSV format|
|Import or Export an entire database||Yes||No|
|Ability to export only selected tables in a database||Yes*||Yes†|
|Ability to import previously exported table/tables||Yes*||Yes†|
|Export at a past timestamp||Yes||Yes|
|Import or Export using Google Cloud CLI||Yes||Yes|
|Import or Export using the Dataflow page of Google Cloud console||Yes||Yes|
|Import or Export using the Cloud Spanner page of Google Cloud console||Yes*||No|
* See "Notes about Avro import and export", below.
† See "Notes about CSV import and export", below.
Notes about Avro import and export
When exporting to Avro format, you can specify a list of tables to export. Any child tables exported this way need to be accompanied by their parent tables. Cloud Spanner maintains the entire database schema in the exported files.
When importing from Avro format, Cloud Spanner recreates the exported database's whole schema, including all tables. Tables included in the original export receive all their exported data; all other tables remain empty.
The Cloud Spanner page of the Google Cloud console offers limited Avro-format import and export options. For example, it does not make network and subnetwork options available. For a wider set of options, use the Dataflow page instead.
Notes about CSV import and export
Cloud Spanner limits you to exporting only a single table at a time into CSV format. When you export, the schema is not exported, only the data is exported.
Before importing from CSV files, you need to create a JSON Manifest manually.
There are no additional charges from Cloud Spanner for use of the export or import tools; you pay the standard rates for data storage when you import a database to Cloud Spanner. However, there are other potential charges associated with importing and exporting databases. Refer to the Cloud Spanner pricing page for more details.
- Learn how to export databases from Cloud Spanner to Avro
- Learn how to import Cloud Spanner Avro files
- Learn how to import and export data in CSV format
- Learn how to import data from non-Cloud Spanner databases