Create a migration job to a new destination instance

MySQL   |  PostgreSQL   |  PostgreSQL to AlloyDB

Overview

Database Migration Service uses migration jobs to migrate data from your source database instance to the destination AlloyDB database instance.

Creating a migration job includes:

• Defining settings for the migration job
• Specifying information about the connection profile that you created for your source database (source connection profile)
• Defining settings for the destination AlloyDB database instance and creating the instance
• Setting up connectivity between the source and destination database instances
• Testing the migration job to ensure that the connection information you provided for the job is valid

Database Migration Service comes equipped with a wizard to help you create a migration job. This wizard consists of five panels: Get started, Define a source, Create a destination, Define connectivity method, and Test and create migration job. Information on how to populate each panel is provided in the various sections of this page.

You can pause the creation of a migration job by clicking Save and exit. All of the data that was populated until that point is saved in a draft migration job, and you can access this job in a DRAFTS tab.

To finish creating a migration job, navigate to the tab, and then click the job. The creation flow resumes from where it was left off. The job remains a draft until you click CREATE or CREATE & START.

Define settings for the migration job

1. Go to the Migration jobs page in the Google Cloud Console.
2. Click Create migration job at the top of the page.
3. Provide a name for the migration job. Choose a friendly name that helps you identify the migration job. Don't include sensitive or personally identifiable information in the job name.
4. Keep the auto-generated Migration job ID.
5. Select the source database engine.
6. Select AlloyDB for PostgreSQL as the destination engine.

7. Select the destination region for your migration. This is where the Database Migration Service instance is created, and should be selected based on the location of the services that need your data, such as Compute Engine instances and App Engine apps, and other services. After you choose the destination region, this selection can't be changed.

8. Choose Continuous (snapshot + ongoing changes) as the migration job type.

9. Review the required prerequisites that are generated automatically to reflect how the environment must be prepared for a migration job. These prerequisites can include how to configure the source database and how to connect it to the destination AlloyDB database instance. It's best to complete these prerequisites at this step, but you can complete them at any time before you test the migration job or start it. For more information about these prerequisites, see Configure your source.

10. Click Save and continue.

Specify information about the source connection profile

1. If you have created a connection profile, then select it from the list of existing connection profiles.

   If you haven't created a connection profile, then create one by clicking Create a connection profile at the bottom of the drop-down list, and then perform the same steps as in Create a source connection profile.

   It is recommended to create a designated connection profile for your AlloyDB migration.

2. In the Customize data dump configurations section, click Show data dump configurations.

   The speed of data dump parallelism is related to the amount of load on your source database. You can use the following settings:

   • Optimal (recommended): Balanced performance with optimal load on the source database.
   • Maximum: Provides the highest dump speeds, but might cause increased load on the source database.
   • Minimum: Takes the lowest amount of compute resources on the source database, but might have slower dump throughput.

   If you want to use adjusted data dump parallelism settings, make sure to increase the max_replication_slots, max_wal_senders, and max_worker_processes parameters on your source database. You can verify your configuration by running the migration job test at the end of migration job creation.

3. Click Save and continue.

If your source database is a Cloud SQL instance and has only public IP enabled, make sure to enable its private IP before continuing to the next section. Remember to update the connection profile with the private IP.

Define and create the destination AlloyDB instance

You will now create a new destination AlloyDB cluster for your migration job. Clusters are the top-level resources in AlloyDB. They contain a single primary instance for read/write access to the database you create during the migration process.

To create a destination cluster:

1. Choose a cluster type. Database Migration Service currently supports Highly available AlloyDB clusters. They can serve data from more than one zone in a region, with no read pools.
2. Click CONTINUE.
3. Configure your cluster:
   1. In the Cluster ID field, enter an ID for your cluster.
   2. In the Password field, enter a password for the default postgres user. You will need the password to log in to your database.
   3. In the Network field:
      1. Select a network path to define which resources are available when setting the migration connectivity. Clusters can only be configured with a private IP network path. If you plan to connect to the source database via VPC peering, select the VPC where it resides.
      2. If your network isn't configured for private services access, click Set up connection and follow the connection configuration wizard.
      3. If a managed service network was never configured for the selected VPC, you can choose to either select an IP range and click Connect or use an automatically selected IP range and click Allocate & Connect.
   4. Optional: In the Encryption section, specify whether you want to manage the encryption of the data that's migrated from the source to the destination. By default, your data is encrypted with a key that's managed by Google Cloud.

      If you want to manage your encryption, you can use a customer-managed encryption key (CMEK). The key must be in the same location as your AlloyDB cluster. For example, for clusters located in us-west1 can use only keys in us-west1.

      1. Select the Use a customer-managed encryption key (CMEK) radio button.
      2. From the Select a customer-managed key menu, select your CMEK.

      If you don't see your key, then click ENTER KEY RESOURCE NAME to provide the resource name of the key that you want to use. For example, you can enter projects/my-project-name/locations/my-location/keyRings/my-keyring/cryptoKeys/my-key in the Key resource name field, and then click SAVE.

      As part of creating the migration job, Database Migration Service will verify that the CMEK exists, and that Database Migration Service has permissions to use the key.

      If Database Migration Service doesn't have these permissions, then information will appear, specifying that the Database Migration Service service account can't use the CMEK. Click GRANT to give Database Migration Service permissions to use the key.

      For more information about encryption keys, see Use CMEK in AlloyDB documentation.

   5. Click CONTINUE.
4. Configure your primary instance. A primary instance determines a cluster's compute capacity and supports read and write operations:
1. In the Instance ID field, enter an ID for your primary instance.
2. Select a machine type.
3. Optional: Set flags for your instance. You can use flags to customize your instance. For information on supported flags, see AlloyDB documentation. For each flag:
   1. Click ADD FLAG.
   2. Select a flag from the New database flag list.
   3. Provide a value for the flag.
   4. Click DONE.
5. Click Save and continue.
6. Confirm your choice by clicking CREATE DESTINATION & CONTINUE.
7. Wait for the creation of the destination instance to finish.

Your AlloyDB cluster will be in a "Bootstrapping" state while it's being managed by Database Migration Service. While in "Bootstrapping", you will not be able to perform any operations on your cluster. If you still wish to delete the cluster, you can do so using gcloud with the --force option.

Set up connectivity between the source and destination database instances

If you're reusing a source connection profile that was used previously for heterogeneous migrations (for example, from Oracle to Cloud SQL PostgreSQL), then in this step, you must choose a connectivity method. This method will be used for the homogeneous migration from Cloud SQL for PostgreSQL database to AlloyDB. You must select a connectivity method because the heterogeneous connectivity method isn't relevant for this migration job.

1. From the Connectivity method drop-down menu, select a network connectivity method. This method defines how the newly created AlloyDB instance will connect to the source database. Current network connectivity methods include VPC peering, reverse SSH tunnel, and TCP proxy through a cloud-hosted VM.

2. If you select the reverse SSH tunnel network connectivity method, then select the Compute Engine VM instance that will host the tunnel.

   After specifying the instance, Google will provide a script that performs the steps to set up the tunnel between the source and destination databases. You'll need to run the script in the Google Cloud CLI.

   Run the commands from a machine that has connectivity to both the source database and to Google Cloud.

3. If you select the VPC peering network connectivity method, then select the VPC network where the source database resides. The Cloud SQL instance will be updated to connect to this network.

4. If you select the TCP Proxy via cloud-hosted VM connectivity method, then enter the required details for the new Compute Engine VM instance that will host the TCP proxy.

   After specifying the details, the Google Cloud console will provide a script that performs the steps to set up the proxy between the source and destination databases. You'll need to run the script on a machine with an updated Google Cloud CLI.

   After running the script, it will output the newly created VM's private IP. Enter the IP and click Configure & continue.

5. Learn more about how to Configure connectivity.
   Note: In case your source is Cloud SQL for PostgreSQL, depending on whether your Cloud SQL source instance is on the old network architecture or the new network architecture, the connectivity options are limited to VPC Peering and TCP Proxy via a cloud-hosted VM.

6. After selecting the network connectivity method and providing any additional information for the method, click CONFIGURE & CONTINUE.

Test and create the migration job

On this final step, review the summary of the migration job settings, source, destination, and connectivity method, and then test the validity of the migration job setup. If any issues are encountered, then you can modify the migration job's settings. Not all settings are editable.

1. Click TEST JOB to verify that:

   • The source database has been configured correctly, based on the prerequisites.
   • The source and destination instances can communicate with each other.
   • Any updates to private IP addresses needed on the destination are done.

   • The migration job is valid, and the source and destination versions are compatible.

   If the test fails, then you can address the problem in the appropriate part of the flow, and return to re-test.

   For more information about reasons why the test may fail and how to troubleshoot any issues associated with the test failing, see Diagnose issues for AlloyDB.

   The migration job can be created even if the test fails, but after the job is started, it may fail at some point during the run.

2. Click CREATE & START JOB to create the migration job and start it immediately, or click CREATE JOB to create the migration job without immediately starting it.

   If the job isn't started at the time that it's created, then it can be started from the Migration jobs page by clicking START.

   Regardless of when the migration job starts, your organization is charged for the existence of the destination instance.

   Key Point: When you start the migration job, Database Migration Service begins the full dump, briefly locking the source database. If your source is in Amazon RDS or Amazon Aurora, Database Migration Service additionally requires a short (approximately under a minute) write downtime at the start of the migration.

   For more information, see Known limitations.

3. The migration job is added to the migration jobs list and can be viewed directly.

Send feedback