Defining data repositories

As part of performing a migration, Migrate for Anthos writes information to different data repositories:

  1. Docker image files representing a migrated Linux VM are written to a Docker registry.

    These Docker image files represent the files and directories of the migrated Linux VM. This repository is not required when migrating Windows workloads.

  2. Migration artifacts that represent the migrated workload are written to a second repository.

    Artifacts include the configuration YAML files that you can use to deploy the migrated workloads, as well as other files. The exact artifacts depend on whether you are migrating Linux or Windows workloads.

The default implementation and location of these repositories depends on where you created the processing cluster used to perform the migration. Because there are no default repositories for Anthos clusters on VMware and Anthos clusters on AWS, you must configure them as part of installing Migrate for Anthos.

Platform Docker image files registry* Migration artifacts repository
Anthos clusters on Google Cloud Default is Google Container Registry (GCR).

Optionally specify any Docker registry that supports basic authentication.

Default is Google Cloud Storage.

Optionally specify S3 as the artifacts repo for Linux migrations. S3 is not supported for migrating Windows workloads.

Anthos clusters on VMware

No default.

Specify either GCR or any Docker registry that supports basic authentication

No default.

Specify either Cloud Storage or S3 as the artifacts repo.

Anthos clusters on AWS

No default.

Specify either GCR, ECR, or any Docker registry that supports basic authentication

No default.

Specify either Cloud Storage or S3 as the artifacts repo.

* The Docker image files registry is not required for Windows migrations. It is only required for migrating Linux VMs.

Viewing the repository status

After you install Migrate for Anthos, you validate the Migrate for Anthos installation by running the migctl doctor command. As part of this validation, the migctl doctor command checks the status of the repos:

migctl doctor

In the following example output of the migctl doctor command, the check mark indicates that Migrate for Anthos has been successfully deployed but you have not yet configured the necessary data repositories:

  [✓] Deployment 
  [!] Docker Registry 
  [!] Artifacts Repo`
  [!] Source Status

After configuring the repositories, you can run the migctl doctor command again to ensure that the repositories are configured correctly:

  [✓] Deployment
  [✓] Docker registry
  [✓] Artifacts repo
  [!] Source Status

Google Cloud Console support

The Google Cloud Console displays the URLs to items in the repos based on the repository implementation. For example, if the repo is implemented using S3 , the Google Cloud Console shows URLs for a bucket in S3 .

Options for repository location

The location of the data repositories can have an effect on migration performance and cost.

For example, the Docker image files representing a migrated VM can be quite large. If you have an on-prem processing cluster, but write the Docker image files to GCR on Google Cloud, then you incur the performance latency of the data upload, and the cost of storing that data.

For an on-prem processing cluster, you might find it more efficient to define a Docker registry local to the cluster. By having the registry local, you minimize the upload latency and minimize storage costs.

For a GKE cluster deployed on Google Cloud, using the default GCR repos provides the highest level of performance, but you will be charged for that storage. However, you are not required to use GCR with a Cloud-based cluster and can choose to use your own Docker registry instead.

Repository authentication

All repositories used by Migrate for Anthos require authentication. The authentication mechanism depends on the repository type, as shown in the following table:

Repository Implementation Authentication
Docker image files registry GCR JSON key for a Google Cloud service account.

See Creating a service account for accessing Container Registry and Cloud Storage for more.

ECR Access key and secret or credentials file. See Managing access keys for IAM users for more.
Docker registry Username and password for basic authentication.
Migration artifacts repository Cloud Storage JSON key for a Google Cloud service account.

See Creating a service account for accessing Container Registry and Cloud Storage for more.

S3 Access key and secret or credentials file. See Overview of managing access for more.

Supporting TLS

Some repositories are accessible using TLS/SSL over HTTPS. If the HTTPS connection to the repository uses a self-signed cert, then you must pass a PEM file containing either of the following when configuring the repository:

  • The public key of the self-signed cert
  • A concatenation from the root certificate and all intermediate certificates up to the actual server certificate

Configuring a Docker registry

Use the migctl command to configure a Docker registry. The migctl command lets you perform the following actions on a registry configuration:

  • Create
  • Update
  • Delete
  • List
  • Set default

You can define multiple configurations. Migrate for Anthos uses the configuration currently defined as the default. Use the migctl docker-registry list command to view the current configurations, including the default. Use the migctl docker-registry set-default command to set the default configuration.

The following example shows how to configure a Docker registry:

  • GCR

    migctl docker-registry create gcr registry-name --project project-id --json-key=m4a-install.json

    where:

    • registry-name is the user-defined name of the Docker registry configuration.

    • project-id is your Google project ID.

    • m4a-install.json is the name of the JSON key file for the service account for accessing Container Registry and Cloud Storage as described in Configuring a service account.

  • ECR

    migctl docker-registry create ecr registry-name --registry-path url --access-key-id=key-id

    You are prompted to enter the secret key for key-id.

    Alternatively, specify the path to a credentials file:

    migctl docker-registry create ecr registry-name --registry-path url --credentials-file-path file-path 

    where:

    • registry-name is the user-defined name of the Docker registry configuration.

    • url specifies the URL of the registry path without the http:// or https:// prefix.

    • key-id specifies the access key. See Managing access keys for IAM users for more.

    • file-path specifies the path to a CSV file, downloaded from the AWS console, containing the credentials. See Configuring AWS IAM groups and instance roles for more on creating the CSV file.

  • Docker registry

    migctl docker-registry create basic-auth registry-name --registry-path url --username username --ca-pem-file ca-pem-filename

    where:

    • registry-name is the user-defined name of the Docker registry configuration.

    • url specifies the URL of the registry without the http:// or https:// prefix. For example localhost:8080/myregistry.

    • username for the basic authentication credentials of the registry. You will be prompted to enter the password.

    • If the registry uses a self-signed cert, ca-pem-filename specifies a PEM file containing either the public key or the complete CA chain, meaning a concatenation from the intermediate CA certificates up to the root certificate. For example:

      cat int1.pem int2.pem ... root.pem

To later update the registry configuration, run the migctl docker-registry update command with the same arguments as you used to create it:

migctl docker-registry update gcr registry-name same-flags-as-create

When you configure a Docker registry, it becomes the default registry. However, you might have multiple registries defined. To see the current list of registries:

migctl docker-registry list

To set the default registry configuration, meaning the one currently used for migrations, use the following command:

migctl docker-registry set-default registry-name

To delete a registry configuration:

migctl docker-registry delete registry-name

Configuring an artifacts repository

Use the migctl command to configure an artifacts repository. The migctl command lets you perform the following actions on a repository configuration:

  • Create
  • Update
  • Delete
  • List
  • Set default

You can define multiple configurations. Migrate for Anthos uses the configuration currently defined as the default. Use the migctl artifacts-repo list command to view the current configurations, including the default. Use the migctl artifacts-repo set-default command to set the default configuration.

The following example show how to configure an artifacts repository:

  • Cloud Storage

    migctl artifacts-repo create gcs repository-name --bucket-name bucket-name --json-key=m4a-install.json

    where:

    • repository-name is the user-defined name of the artifacts repository configuration.

    • bucket-name specifies an existing bucket in the Cloud Storage repository. If you do not have an existing bucket, create one using the instructions at Creating storage buckets.

      Note: When installing Migrate for Anthos on clusters on Google Cloud, the Migrate for Anthos installer automatically creates a default bucket named:

      GCP_PROJECT-migration-artifacts

      Where GCP_PROJECT is your Google project ID. However, when installing Migrate for Anthos on an on-prem processing cluster or on Anthos clusters on AWS, no default bucket is created. You must create one yourself.

    • project-id is your Google project ID.

    • m4a-install.json is the name of the JSON key file for the service account for accessing Container Registry and Cloud Storage as described in Configuring a service account.

  • S3

    migctl artifacts-repo create s3 repository-name --bucket-name bucket-name --region aws-region --access-key-id=key-id

    You are prompted to enter the secret key for key-id.

    Alternatively, specify the path to a credentials file:

    migctl artifacts-repo create s3 repository-name --bucket-name bucket-name --region aws-region --credentials-file-path file-path

    where:

    • repository-name is the user-defined name of the artifacts repository configuration.

    • bucket-name specifies an existing bucket in the S3 repository. If you do not have an existing bucket, create one using the instructions at Working with Amazon S3 Buckets.

    • aws-region specifies the AWS region for the repository. The processing cluster and the repository can be in separate regions as long as the cluster has permissions to access the repository.

    • key-id specifies the access key. See Overview of managing access for more.

    • file-path specifies the path to a CSV file, downloaded from the AWS console, containing the credentials. See Configuring AWS IAM groups and instance roles for more on creating the CSV file.

To later update the repository configuration, run the migctl docker-registry update command with the same arguments as you used to create it:

migctl artifacts-repo update gcr repository-name same-flags-as-create

When you configure a repository registry, it becomes the default repository. However, you might have multiple repositories defined. To see the current list of repositories:

migctl artifacts-repo list

To set the default repository configuration, meaning the one currently used for migrations, use the following command:

migctl artifacts-repo set-default repository-name

To delete a repository configuration:

migctl artifacts-repo delete repository-name

Next Steps