Migrate a Compute Engine VM by using Migrate to Containers

In this quickstart, you create a Compute Engine virtual machine (VM), then use Migrate to Containers (Migrate for Anthos and GKE) to migrate the VM to a Google Kubernetes Engine (GKE) processing cluster in the Cloud. This section refers strictly to Image Based System Container migrations.

You can perform the steps in the quickstart by using either the migctl tool or the Google Cloud console. migctl is a command-line tool for setting up and managing a Migrate to Containers migration environment on Google Cloud. migctl is available in Cloud Shell.

Before you begin

Enable support for Compute Engine

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

5. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

6. Enable the Compute Engine API.

   Enable the API

Enable and configure GKE

Before you start, make sure you have performed the following tasks:

• Enable the Google Kubernetes Engine API.
Enable Google Kubernetes Engine API
• If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI.
  Note: For existing gcloud CLI installations, make sure to set the compute/region and compute/zone properties. By setting default locations, you can avoid errors in gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location.

Create a Compute Engine VM with a web server

In the following steps, you create a simple Compute Engine VM that serves a "Hello World!" web page. Having the page gives you something to test after migration.

1. Run the following command to create the VM instance that you migrate.

   Specify a zone that is the same zone in which you create a GKE cluster later in this topic.

   gcloud compute instances create quickstart-instance \
     --project=PROJECT_ID  --zone=ZONE --machine-type=e2-standard-2 \
     --subnet=default --scopes="cloud-platform" \
     --tags=http-server,https-server --image=ubuntu-minimal-1604-xenial-v20210119a \
     --image-project=ubuntu-os-cloud --boot-disk-size=10GB --boot-disk-type=pd-standard \
     --boot-disk-device-name=quickstart-instance
   

2. Run the following command to create a firewall rule that allows requests to the instance over HTTP.

   gcloud compute --project=PROJECT_ID firewall-rules create default-allow-http \
     --direction=INGRESS --priority=1000 --network=default --action=ALLOW \
     --rules=tcp:80 --source-ranges=0.0.0.0/0 --target-tags=http-server
   

3. In the Google Cloud console, go to the VM instances page.

   Go to the VM instances page

4. In the list of virtual machine instances, click SSH in the row of the instance you just created.

   For more on connecting by SSH, see Connecting to instances.

5. In the quickstart-instance command window, use the package manager to install the apache2 package.

   sudo apt-get update && sudo apt-get install apache2 -y
   

   After installing Apache, the operating system automatically starts the Apache server.

6. In the same command window, overwrite the Apache web server default web page with the following command:

   echo '<!doctype html><html><body><h1>Hello World!</h1></body></html>' | sudo tee /var/www/html/index.html
   

7. In the VM instances page, locate the row for the instance you created and copy the External IP address.

8. Paste the instance's IP address to your browser address bar. Prefix it with http://.

   You should now see the "Hello World!" page.

9. In the VM instances page, select the checkbox at the left end of the row for the instance you created.

10. At the top of the page, click the Stop button to stop the VM.

    Note: The VM must be stopped for you to migrate it. You can start it again after you create the migration below, either by using the Google Cloud console or by using the 'migctl migration create' command.

Create a processing cluster and install Migrate to Containers

In the following steps, you create a GKE cluster in Google Cloud that you use as a processing cluster. This is where you install Migrate to Containers and execute the migration.

1. In Cloud Shell, use the following command to create a new Kubernetes cluster to use as a processing cluster.

   When specifying the machine-type value, be sure to specify a machine type that has at least 15 GB of memory.

   Note: Be sure you specify the same zone here that you specified when creating the VM.

   gcloud container clusters create migration-processing \
     --project=PROJECT_ID --zone=ZONE --machine-type e2-standard-4 \
     --image-type ubuntu --num-nodes 1 \
     --subnetwork "projects/PROJECT_ID/regions/REGION/subnetworks/default"
   

2. Install Migrate to Containers.

   migctl

   To enable Migrate to Containers to access Container Registry and Cloud Storage, you must create a service account with the storage.admin IAM role:

   1. In Cloud Shell, create the m4a-install service account:

      gcloud iam service-accounts create m4a-install \
        --project=PROJECT_ID

   2. Grant the storage.admin role to the service account:

      gcloud projects add-iam-policy-binding PROJECT_ID  \
        --member="serviceAccount:m4a-install@PROJECT_ID.iam.gserviceaccount.com" \
        --role="roles/storage.admin"

   3. Download the key file for the service account:

      gcloud iam service-accounts keys create m4a-install.json \
        --iam-account=m4a-install@PROJECT_ID.iam.gserviceaccount.com \
        --project=PROJECT_ID 

   4. Connect to the cluster.

      gcloud container clusters get-credentials migration-processing \
        --zone ZONE --project PROJECT_ID
      

   5. In Cloud Shell, set up Migrate to Containers components on your processing cluster by using the migctl command-line tool included with Migrate to Containers:

      migctl setup install --json-key=m4a-install.json
      

   6. Validate the Migrate to Containers installation.

      Use the migctl doctor command to confirm a successful deployment.

      migctl doctor
      

      It may take more than a minute before the command returns the following success result.

      migctl doctor
      [✓] Deployment
      

      Note: Source Status shows [!] until you define your first migration source below.

   Console

   Google Cloud console can help you with the installation process by generating the necessary installation commands and running them in Cloud Shell.

   1. Open the Migrate to Containers page in the Google Cloud console.

      Go to the Migrate to Containers page

   2. Click the Processing Cluster tab, then click Add processing cluster.

   3. Select the workload OS type from the drop down list.

   4. Select the cluster you created above, migration-processing, from the drop-down list.

   5. Assign for Artifact repository and Image repository.

   6. Assign an existing service account or create one.

   7. Click Deploy.

Qualify source workloads

For this quickstart, which uses a Compute Engine VM to serve up a simple "Hello World!" web page, you don't need to run fit assessment. However, for the typical case where you're migrating your own VMs, we recommend that you run the fit assessment tool. This tool runs on a VM workload to determine the workload's fit for migration to a container, and generates an assessment report that describes any issues that must be resolved before migration. For more information see About the fit assessment tool.

Migrate the Compute Engine VM

In the following steps, you create a migration plan with migration details, then use it to migrate the VM.

1. Specify the migration source as a Compute Engine VM. This step adds the source specification to the migration plan.

   migctl

   To use Compute Engine as a migration source, you must first create a service account with the compute.viewer and compute.storageAdmin roles:

   1. In Cloud Shell, create the m4a-ce-src service account:

      gcloud iam service-accounts create m4a-ce-src \
        --project=PROJECT_ID

   2. Grant the compute.viewer role to the service account:

      gcloud projects add-iam-policy-binding PROJECT_ID  \
        --member="serviceAccount:m4a-ce-src@PROJECT_ID.iam.gserviceaccount.com" \
        --role="roles/compute.viewer"

   3. Grant the compute.storageAdmin role to the service account:

      gcloud projects add-iam-policy-binding PROJECT_ID  \
        --member="serviceAccount:m4a-ce-src@PROJECT_ID.iam.gserviceaccount.com" \
        --role="roles/compute.storageAdmin"

   4. Download the key file for the service account:

      gcloud iam service-accounts keys create m4a-ce-src.json \
        --iam-account=m4a-ce-src@PROJECT_ID.iam.gserviceaccount.com \
        --project=PROJECT_ID 

   5. Create the migration source:

      migctl source create ce quickstart-source --project PROJECT_ID --json-key=m4a-ce-src.json
      

      Where m4a-ce-src.json specifies the service account you created above.

   Console

   1. Open the Migrate to Containers page in the Google Cloud console.

      Go to the Migrate to Containers page

   2. Click the Sources & Candidates tab and then click Add Source.

   3. Select a cluster where you have already installed the Migrate to Containers components from the drop-down list .

   4. Click Next.

   5. Specify the name of the migration as quickstart-source.

   6. Set the Source type to Compute Engine.

   7. Click Next.

   8. Select the project that contains the VM that you want to migrate.

   9. Create a service account that enables you to use Compute Engine as a migration source by selecting Create a new service account. You are prompted to enter the name of the new service account. You can also use an existing service account.

   10. Click Next.

   11. Select Add Source.

2. Create a migration.

Create a migration by using the migctl tool or the Google Cloud console.

Important: You must specify the --type flag with the tomcat option when running migctl to create your migration plan.

migctl on Google Cloud

Compute Engine

1. Stop the Compute Engine VM. The VM must be stopped for you to create a migration. After the migration completes, you can restart the VM.

2. Create the migration:

   migctl migration create my-migration --source quickstart-source --vm-id quickstart-instance

   Where --vm-id specifies the name of the Compute Engine instance that you want to migrate, as shown in the Google Cloud console.

VMware

migctl migration create my-migration --source my-vmware-src --vm-id My_VMware_VM

Where --vm-id specifies the name of the VM instance. You can specify either the VM name, such as MyVM2, or the VM MoRef ID (Managed Object Reference ID), such as vm-14.

AWS

migctl migration create my-migration --source my-aws-src --vm-id i-1234567890abcdef0

Where --vm-id specifies the name of the VM instance. The name of the VM instance is shown in the AWS UI and is in form i-##########.

Azure

migctl migration create my-migration --source my-azure-src --vm-id /subscriptions/a1b5c3d4-c3d9-a1a2-e5f6-a1t2c3d4k5f6/resourceGroups/MyResourceGroup/providers/Microsoft.Compute/virtualMachines/my-virtual-machine

Where --vm-id specifies the name of the VM instance. The name of the VM instance is shown in the Azure UI.

migctl on VMware

migctl migration create my-migration --source local-vmware-src --vm-id My_VMware_VM

Where --vm-id specifies the name of the VM instance. You can specify either the VM name, such as MyVM2, or the VM MoRef ID (Managed Object Reference ID), such as vm-14.

migctl on AWS

migctl migration create my-migration --source local-aws-src --vm-id My_AWS_VM

Where --vm-id specifies the name of the AWS instance.

Note: Source VM workloads must be in the same region and account as the destination Anthos clusters on AWS. If necessary, copy the source VM to the account and region of the Anthos clusters on AWS.

Console

Note: You can only use the Console with processing clusters deployed on Google Cloud.

1. If the migration source is a Compute Engine VM, stop the Compute Engine VM. The VM must be stopped for you to create a migration. After the migration completes, you can restart the VM.

2. Open the Migrate to Containers page in the Google Cloud console.

   Go to the Migrate to Containers page

3. Click the Migrations tab to display a table containing the available migrations.

4. Click Create Migration.

5. Enter the Migration name.

6. Select the migration source that you created in Adding a migration source.

7. Set the Workload type as Linux system container.

8. Enter the source VM ID, which is the instance name.

9. Click Create Migration.

   When migration creation completes, the Status column displays Migration plan generated.

CRD

Note:

This example uses the predefined namespace of v2k-system. Migrate to Containers requires that you use the v2k-system namespace in all CRDs.

1. Create a my-migration.yaml file]((migration-crd)) to define a migration.

   In this file, sourceProvider specifies the name of a previously created SourceProvider and sourceId specifies the name of the Compute Engine instance:

   apiVersion: anthos-migrate.cloud.google.com/v1beta2
   kind: Migration
   metadata:
    name: my-migration
    namespace: v2k-system
   spec:
    appType: tomcat-container
    osType: Linux
    sourceSnapshot:
     sourceProvider: quickstart-source
     sourceId: my-id

2. If you are migrating a Compute Engine VM, stop the VM. The VM must be stopped for you to create a migration. After the migration completes, you can restart the VM.

3. Apply the file:

   kubectl apply -f my-migration.yaml

Monitor migration creation

Use the migctl tool or the Google Cloud console to monitor the progress of migration creation.

migctl Google Cloud

1. Wait until the migration has completed:

   migctl migration status my-migration
   
   NAME           CURRENT-OPERATION      PROGRESS   STEP                   STATUS   AGE
   my-migration   GenerateMigrationPlan  [1/3]      CreateSourceSnapshots  Running  13s 

2. If you are migrating a Compute Engine VM, then restart it. The VM must be stopped for you to create a migration.

migctl on VMware

Wait until the migration has Completed:

migctl migration status my-migration

NAME           CURRENT-OPERATION      PROGRESS STEP                   STATUS   AGE
my-migration   GenerateMigrationPlan  [1/3]    CreateSourceSnapshots  Running  13s

migctl on AWS

Wait until the migration has Completed:

migctl migration status my-migration

NAME           CURRENT-OPERATION      PROGRESS STEP                   STATUS   AGE
my-migration   GenerateMigrationPlan  [1/3]    CreateSourceSnapshots  Running  13s

Console

When migration status show that the migration has been created, you can move on to the next step:

1. Open the Migrate to Containers page in the Google Cloud console.

   Go to the Migrate to Containers page

2. Click the Migrations tab to display a table containing the available migrations.

3. View the status of a migration by:

   • Viewing the Status column for the row for your migration.

   • Clicking the migration name. The Migration details page opens. Click the Monitoring tab to see the detailed status, or the Events tab to see migration events.

4. When migration creation completes, the Status displays Migration plan generated.

5. If you are migrating a Compute Engine VM, then restart it. The VM must be stopped for you to create a migration.

CRD

Use kubectl to get migration information and status by specifying a JSONPath to the necessary field.

1. Get the migration operation:

   kubectl get migrations.anthos-migrate.cloud.google.com -n v2k-system  my-migration -o jsonpath={.status.currentOperation}

   Returns GenerateMigrationPlan.

2. Get the migration status:

   kubectl get migrations.anthos-migrate.cloud.google.com -n v2k-system my-migration -o jsonpath={.status.status}

   When the status shows that the operation has completed, you can move on to the next step.

3. If you are migrating a Compute Engine VM, then restart it. The VM must be stopped for you to create a migration.

1. Review the migration plan.

You can edit the migration plan by using the migctl tool or the Google Cloud console.

migctl

You must download the migration plan before you can edit it:

1. Download the migration plan. The plan is represented by GenerateArtifactsFlow:

   migctl migration get my-migration
   

2. Edit the downloaded migration plan, my-migration.yaml, in a text editor.

3. When your edits are complete, save and upload the revised migration plan:

   migctl migration update my-migration --file my-migration.yaml
   

4. Repeat these steps if more edits are necessary.

Console

Edit the migration plan in the Google Cloud console by using the YAML editor.

1. Open the Migrate to Containers page in the Google Cloud console.

   Go to the Migrate to Containers page.

2. Click the Migrations tab to display a table containing the available migrations.

3. In the row for your desired migration, select the migration Name to open the Details tab.

4. Select the YAML tab.

   Note: You can also select Download Migration Plan to download a local copy.

5. Edit the migration plan as necessary.

6. When you are done editing, you can either:

   1. Save the migration plan. You will then have to manually execute the migration to generate the migration artifacts. Use the procedure shown in Executing a migration.

   2. Save and generate the artifacts. Execute the migration by using your edits to generate the migration artifacts. The process is the same as described in Executing a migration. You can then monitor the migration as described in Monitoring a migration.

CRD

You must download the migration plan, edit it, then apply it.

1. Get the name of the GenerateArtifactsFlow:

   kubectl get migrations.anthos-migrate.cloud.google.com -n v2k-system my-migration -o jsonpath={.status.resources.generateArtifacts.name}

   The naming pattern is returned in the form of generate-artifacts-flow-id.

2. Get the GenerateArtifactsFlow by name and write to a file named my-plan.yaml:

   kubectl get generateartifactsflows.anthos-migrate.cloud.google.com -n v2k-system generate-artifacts-flow-id -o yaml > my-plan.yaml

3. Edit the migration plan as necessary.

4. Apply the file:

   kubectl apply -f my-plan.yaml

After the migration begins, check its status until it finishes. The migration can take several minutes. During this time, checking the status reports the migration's current state.

migctl

migctl migration status my-migration

When the migration completes, you should see a message such as the following:

migctl migration status my-migration

NAME            CURRENT-OPERATION       PROGRESS        STEP            STATUS    AGE
my-migration    GenerateArtifacts       [1/1]           ExtractImage    Completed 14m23s

Console

1. Open the Migrate to Containers page in the Google Cloud console.

   Go to the Migrate to Containers page

2. View the status of a migration by:

   • Viewing the Status column for the row for your migration.

   • Clicking the migration name. The Migration details page opens. Click the Monitoring tab to see the status details, or the Events tab to see migration events.

3. When migration creation completes, the Status displays Migration plan generated.

Deploy the migrated workload

1. Use the migctl migration generate-artifacts command or the Google Cloud console to generate the target container artifacts as part of processing a VM for migration.

   migctl

   migctl migration generate-artifacts my-migration

   CRD

   1. Create a GenerateArtifactsTask my-generate.yaml file to define a migration.

      In this file, set the value of the name field for migration to the name you specified in the Migration yaml file when you created the migration:

      apiVersion: anthos-migrate.cloud.google.com/v1beta2
      kind: GenerateArtifactsTask
      metadata:
      name: my-migration-task
      namespace: v2k-system
      spec:
      migration:
       name:  my-migration

   2. Apply the file:

      kubectl apply -f my-generate.yaml

   Console

   1. Open the Migrate to Containers page in the Google Cloud console.

      Go to the Migrate to Containers page

   2. Click the Migrations tab to display a table containing the available migrations.

   3. In the row for the desired migration, select Generate artifacts in the Next steps dropdown.

   4. View the status of a migration by:

      • Viewing the Status column for the row for your migration.

      • Clicking the migration Name. The migration Details tab opens. Click the Monitoring tab to see the status details, or the Events tab to see migration events.

   When you generate artifacts for Tomcat workloads, Migrate to Containers writes the artifacts and then uploads them to a Cloud Storage bucket. The artifact directory file contains:

   • Dockerfile - The Dockerfile used to build the image for your migrated VM.
   • deployment_spec.yaml - The YAML file that configures your workload.

   You can use kubectl apply with this file to deploy the workload to a cluster, such as a production or test cluster.

   • Directories and files that are extracted from the source and used by the Dockerfile.

   To check the migration progress:

   migctl

   migctl migration status my-migration
   
   NAME            CURRENT-OPERATION       PROGRESS        STEP            STATUS  AGE
   my-migration    GenerateArtifacts       [1/1]           ExtractImage    Running 12m2s

   When the migration completes, you should see a message such as the following when you request status with migctl migration status.

   migctl migration status my-migration
   
   NAME            CURRENT-OPERATION       PROGRESS        STEP            STATUS    AGE
   my-migration    GenerateArtifacts       [1/1]           ExtractImage    Completed 14m23s

   You can also add the -v flag to get error and warning information. See Monitoring a migration for more.

   CRD

   Use kubectl to get execution information and status by specifying a JSONPath to the necessary field.

   1. Get the migration operation:

      kubectl get migrations.anthos-migrate.cloud.google.com -n v2k-system  my-migration -o jsonpath={.status.currentOperation}

      Returns GenerateArtifacts.

   2. Get the migration status:

      kubectl get migrations.anthos-migrate.cloud.google.com -n v2k-system my-migration -o jsonpath={.status.status}

      When the status shows that the operation has completed, you can move on to the next step.

   Console

   1. Open the Migrate to Containers page in the Google Cloud console.

      Go to the Migrate to Containers page

   2. Click the Migrations tab to display a table containing the available migrations.

   3. View the status of a migration by:

      • Clicking the migration name. The Migration details panel opens to display the migration status and other information.

      • Viewing the Status column for the row for your migration.

   4. When the Status of the migration is Artifacts generated you can move on to the next step.

   Add the -v flag to get error and warning information. See the next step for more information.

2. From Cloud Shell, download the artifacts with the following command:

   migctl migration get-artifacts my-migration
   

3. Open the deployment_spec.yaml file and locate the Service object whose name is quickstart-instance.

4. Beneath the Service definition, paste another Service that exposes port 80 for access to your web server over HTTP, then save the file.

   Paste the highlighted code so that you have the following in your deployment spec.

   apiVersion: v1
   kind: Service
   metadata:
     creationTimestamp: null
     name: quickstart-instance
   spec:
     clusterIP: None
     selector:
       app: quickstart-instance
     type: ClusterIP
   status:
     loadBalancer: {}
   
   ---
   
   apiVersion: v1
   kind: Service
   metadata:
     name: hello-service
   spec:
     selector:
       app: quickstart-instance
     ports:
       - protocol: TCP
         port: 80
         targetPort: 80
     type: LoadBalancer
   
   

   ---

5. Apply the deployment spec YAML to deploy the workload.

   kubectl apply -f deployment_spec.yaml
   

   It may take a few minutes for the deployment to finish.

6. Check for an external IP address.

   kubectl get service hello-service
   

   When the web server is ready, you see an external IP address for the hello-service you added.

   kubectl get service hello-service
   NAME            TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
   hello-service   LoadBalancer   10.23.241.124   ##.##.###.##   80:32611/TCP   5m4s
   

7. Test the migration by opening a browser and visiting the web page at the external IP address (be sure to use HTTP, rather than HTTPS).

   http://##.##.###.##
   

You can execute bash commands on the container with your migrated workload. For more, see the Troubleshooting topic on executing shell commands on your container.

Clean up

To avoid unnecessary Google Cloud charges, use the Google Cloud console to delete your project if you do not need it.

What's next

• Benefits of migrating to containers
• Migration journey to GKE
• Getting started