Create a migration job

Database Migration Service uses migration jobs to migrate data from your source databases to the Cloud SQL destination instance. Migration jobs help you organize the source and destination connection profiles, define settings specific to the migration process, monitor the progress, and safely finalize the whole operation.

Creating a migration job includes performing the following tasks:

  • Selecting source and destination connection profiles.

  • Choosing which databases detected in your source instance you want to migrate.

  • Performing a migration job test to ensure that Database Migration Service is able to connect to your data source and destination.

  • Starting the migration job, and monitoring the progress.

  • Promoting the migration job when you want to switch your application to the new instance.

Before you begin

  1. Ensure you meet the following requirements:
    • You have a Cloud Storage bucket for SQL Server backup files.
    • Your full backup and transaction log files are uploaded to the Cloud Storage bucket.
    • You have a source connection profile for the Cloud Storage bucket.
    • You created and configured your destination Cloud SQL for SQL Server instance, and you have a destination connection profile for the instance.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Enable the Database Migration Service, Compute Engine, Cloud SQLAdmin APIs.

    Enable the APIs

Required roles

To get the permissions that you need to create migration jobs in Database Migration Service, ask your administrator to grant you the following IAM roles on your project:

For more information about granting roles, see Manage access.

These predefined roles contain the permissions required to create migration jobs in Database Migration Service. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to create migration jobs in Database Migration Service:

  • datamigration.*
  • resourcemanager.projects.get
  • resourcemanager.projects.list
  • cloudsql.instances.create
  • cloudsql.instances.get
  • cloudsql.instances.list
  • cloudsql.databases.get
  • cloudsql.databases.delete
  • cloudsql.operations.get
  • compute.machineTypes.list
  • compute.machineTypes.get
  • compute.projects.get
  • storage.buckets.list
  • storage.objects.list
  • storage.objects.get

You might also be able to get these permissions with custom roles or other predefined roles.

Define settings and create a migration job

To create a migration job, perform the following steps:

Console

  1. In the Google Cloud console, go to the Migration jobs page.

    Go to Migration jobs

  2. Click Create migration job.

    The migration job configuration wizard page opens. This wizard contains multiple panels that walk you through each configuration step.

    You can pause the creation of a migration job at any point by clicking SAVE & EXIT. All of the data that you enter up to that point is saved in a draft migration job. You can finish your draft migration job later. See Update a draft migration job.

  3. On the Get started page, enter the following information:
    • Migration job name

      This is a human-readable name for your migration job. This value is displayed in the Google Cloud console.

    • Migration job ID

      This is a machine-readable identifier for your migration job. You use this value to work with migration jobs by using Database Migration Service Google Cloud CLI commands or API.

    • From the Source database engine list, select SQL Server or Amazon RDS for SQL Server.

    The Destination database engine and Migration job type fields are populated automatically and can't be changed.

  4. Click Save and continue.
  5. On the Define source to destination page, use the drop-down menus to select your source and destination connection profiles.

    If you don't yet have a source or destination connection profile, you can click Create connection profile directly in the drop-down menu. For more information on creating connection profiles, see Create a source connection profile, Create a destination connection profile.

  6. Click Save and continue.
  7. On the Configure migration objects page, perform the following actions:
    1. The Verify backup file naming convention section describes naming requirements for your backup files. Make sure your files meet the requirements. For more information on backup file naming conventions, see Take SQL Server backups.
    2. In the Select databases to migrate section, use the checkboxes to select which databases you want to include in this migration job. This list is populated based on the folders you created in the Cloud Storage bucket for your backup backup files.

      You can add and remove databases from the migration job later. See Add or remove databases from a migration job.

    3. (Optional) If you use encrypted backups, provide the encryption keys for your backups. For more details on using encrypted backups, see Use encrypted backups.

      Perform the following actions:

      • Click Edit details next to the database you selected for migration.

        The Encryption side panel opens.

      • Use the Encryption key drop-down menus to select your keys.
      • In the Password field, enter the encryption key password.
      • Click Save and exit.
  8. Click Save and continue.
  9. On the Test and create migration job, perform the following actions:
    1. (Optional) Click Test job to verify if Database Migration Service can successfully identify all the backup files and establish the necessary network connections.

      If the test fails, you can consult the error messages to address the problem, and run the test again. For more information on possible errors, see Diagnose issues.

    2. Click Create and start job to begin your migration.

      If you want to perform the migration job at a different time, click Save and return later to run the job. See Start a migration jobs.

gcloud

This sample uses the optional --no-async flag so that all operations are performed synchronously. This means that some commands might take a while to complete. You can skip the --no-async flag to run commands asynchronously. If you do, you need to use the gcloud database-migration operations describe command to verify if your operation is successful.

Before using any of the command data below, make the following replacements:

  • MIGRATION_JOB_ID with a machine-readable identifier for your migration job. You use this value to work with migration jobs by using Database Migration Service Google Cloud CLI commands or API.
  • REGION with the region identifier where you want to save the migration job.
  • MIGRATION_JOB_NAME with a human-readable name for your migration job. This value is displayed in Database Migration Service in the Google Cloud console.
  • SOURCE_CONNECTION_PROFILE_ID with a machine-readable identifier of the source connection profile.
  • DESTINATION_CONNECTION_PROFILE_ID with a machine-readable identifier of the destination connection profile.
  • COMMA_SEPARATED_DATABASE_ID_LIST with a comma-separated list of database identifiers of the SQL Server you want to migrate from your backup files. These identifiers need to be the same as the database folder names in your Cloud Storage.

    For example: --sqlserver-databases=my-business-database,my-other-database

  • (Optional) MAPPING_FILE_FOR_PATHS_TO_ENCRYPTION_KEYS with your local disk path to a configuration file that maps paths to encryption keys in Cloud Storage with their relevant database identifiers. For more information on how to create this mapping file, see Use encrypted backups.

    For example: --sqlserver-encrypted-databases=~/encryption-keys-mapping-file.json

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud database-migration migration-jobs \
create MIGRATION_JOB_ID \
  --no-async \
  --region=REGION \
  --display-name=MIGRATION_JOB_NAME \
  --source=SOURCE_CONNECTION_PROFILE_ID \
  --destination=DESTINATION_CONNECTION_PROFILE_ID \
  --sqlserver-databases=COMMA_SEPARATED_DATABASE_ID_LIST \
  --sqlserver-encrypted-databases=MAPPING_FILE_FOR_PATHS_TO_ENCRYPTION_KEYS \
  --type=CONTINUOUS

Windows (PowerShell)

gcloud database-migration migration-jobs `
create MIGRATION_JOB_ID `
  --no-async `
  --region=REGION `
  --display-name=MIGRATION_JOB_NAME `
  --source=SOURCE_CONNECTION_PROFILE_ID `
  --destination=DESTINATION_CONNECTION_PROFILE_ID `
  --sqlserver-databases=COMMA_SEPARATED_DATABASE_ID_LIST `
  --sqlserver-encrypted-databases=MAPPING_FILE_FOR_PATHS_TO_ENCRYPTION_KEYS `
  --type=CONTINUOUS

Windows (cmd.exe)

gcloud database-migration migration-jobs ^
create MIGRATION_JOB_ID ^
  --no-async ^
  --region=REGION ^
  --display-name=MIGRATION_JOB_NAME ^
  --source=SOURCE_CONNECTION_PROFILE_ID ^
  --destination=DESTINATION_CONNECTION_PROFILE_ID ^
  --sqlserver-databases=COMMA_SEPARATED_DATABASE_ID_LIST ^
  --sqlserver-encrypted-databases=MAPPING_FILE_FOR_PATHS_TO_ENCRYPTION_KEYS ^
  --type=CONTINUOUS

You should receive a response similar to the following:

Waiting for migration job [MIGRATION_JOB_ID]
to be created with [OPERATION_ID]

Waiting for operation [OPERATION_ID] to complete...done.

Created migration job MIGRATION_JOB_ID [OPERATION_ID]
Result

The migration job is now created. You can proceed with the migration process:

What's next