Release Notes

This page documents production updates to Migrate for Anthos and GKE. You can periodically check this page for announcements about new or updated features, bug fixes, known issues, and deprecated functionality.

You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud Console, or you can programmatically access release notes in BigQuery.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/migrateanthos-release-notes.xml

August 17, 2021

On August 17, 2021, we released Migrate for Anthos and GKE 1.8.1.

See Upgrading Migrate for Anthos to 1.8.1 for upgrade instructions.

Version 1.8 added the initial support for the preview release of the enhanced runtime, which lets you deploy containers to GKE Autopilot clusters and to Cloud Run. This release adds the following new features to the preview:

  • You no longer set an annotation in the migration plan to enable the enhanced runtime. Instead, you now set v2kServiceManager.
  • The prestart and poststart entries in the config.yaml file now automatically populated.
  • Added support to the config.yaml file that lets you specify environment variables at the global level or at the application level.
  • Added logging support that lets you customize log data written to Cloud Logging.

See Enhanced runtime for more on these new features

Version 1.8 added the initial support for the preview release of the fit assessment tool. The fit assessment tool for version 1.8.1 adds new functionality, including:

  • Ability to collect data for a Windows VM
  • Ability to remotely collect data for Linux and Windows VMs using VMware tools
  • Ability to remotely collect data over SSH

See Using the fit assessment tool for more.

When you generate the migration artifacts, Migrate for Anthos and GKE now generates the new logs.yaml file from the migration plan. This file contains the list of log files detected on the source VM. You can now edit the logs.yaml file to configure logging and the data written to Cloud Logging.

See Customizing log data written to Cloud Logging for more.

Added support for specifying connection strings when migrating a Windows workload. Connection strings define a connection from the migrated container workload to a .NET Framework data provider.

See Setting connection strings for a data provider for more.

The cos-runtime option to the migctl setup install command has been renamed to runtime.

179171930: A migrated container workload can now be deployed to a cluster running GKE 1.20 and later.

Before you run your migrated workloads, you must install migctl with runtime support for Container-Optimized OS nodes on your cluster:

migctl setup install --runtime

See Deploying a Linux workload to a target cluster for more information.

166014117 : The documentation has been updated to describe how to delete the migration to free up the source VM after a successful migration. See Deleting a migration for more.

183082390: The collection script used by the Linux discovery tool uses service --status-all to query system V services. This call no longer takes an arbitrary amount of time to return.

183564181: When you call migctl setup upgrade on Anthos clusters on VMware or Anthos clusters on AWS without the necessary platform argument (--gkeop or --gke-on-aws), migctl no longer uninstalls Migrate for Anthos and GKE.

179028669: migctl doctor no longer crashes if a GKE cluster is currently repairing.

171686793: The migctl setup upgrade --gkeop command no longer creates a new ImageRepositiry or ArtifactRepository object that lacked Google Cloud access credentials.

194186514: When using Anthos clusters on AWS as the processing cluster to perform migrations of AWS workloads, if you have insufficient credentials to create an ECR repository, sometimes the migration succeeds. However, the ECR repository is not created.

Workaround: Update your credentials, then recreate and retry the migration.

197206783: The user credentials passed to the mfit discover ssh ... command must be the credentials of the root user on the VM. Running the command as a non-root user executes the command successfully, but only collects a small part of the data required for a full assessment.

June 29, 2021

On June 29, 2021, we released Migrate for Anthos and GKE 1.8.

See Upgrading Migrate for Anthos to 1.8 for upgrade instructions.

Enhanced runtime support added which lets you deploy containers to GKE Autopilot clusters and to Cloud Run, and simplifies the process of deploying containers to Anthos clusters on AWS that use workload identity. This feature is in preview. See Enhanced runtime for more.

Added support for the preview release of the fit assessment tool that is intended to eventually replace the existing Linux discovery tool. The new fit assessment tool provides you with:

  • Ability to get the inventory information about VMware VMs through direct connection to vCenter.
  • Enhanced HTML output that makes it easier to view the assessment results.
  • New collection script, mfit_linux_collect.sh, and new assessment tool, mfit.

See Using the fit assessment tool for more.

179976237: You can now create a Docker image file registry configuration with the name of a previously deleted configuration.

187922406: A migration might fail due to a LVM (Logical Volume Manager) failure.

Workaround: Recreate and retry the migration.

166014117 : If you are using Migrate for Compute Engine with Migrate for Anthos and GKE to migrate Linux workloads, after you complete a successful migration, delete the migration to free up the source VM.

195341095: Migrate for Anthos and GKE does not support software RAID disks.

179171930: A migrated container workload cannot be deployed to a cluster running GKE 1.20 and later.

May 19, 2021

On May 19, 2021, we released Migrate for Anthos and GKE 1.7.5.

See Upgrading Migrate for Anthos to 1.7.5 for upgrade instructions.

New rule added to the Linux discovery tool to display information when multiple NICs are detected because multiple NICs can increase the effort required to migrate the workload. For a complete list of updates and information on using the new tool, see Using the Linux discovery tool.

A generated migration plan now contains entries for Service endpoints and NFS mounts discovered on the source VM. You can now edit the migration plan to add, remove, or delete these entries. See Customizing a migration plan for more.

Following the Microsoft decision to end support for the Windows 10, version 1909 architecture, the Google Kubernetes Engine (GKE) team removed support for this architecture on its GKE and Anthos clusters. This change means that you must update all Windows containers built for this architecture.

When migrating Windows workloads, you have to perform the following:

  • If you have existing Windows containers that you created using Migrate for Anthos version 1.7 or earlier, then you have to rebuild the containers to create containers capable of running on supported Windows architectures.
  • If you are currently using Migrate for Anthos to migrate Windows workloads, you must update your Migrate for Anthos processing clusters to use version 1.7.5 and update the node pools to use the supported Windows architecture.

Rebuild deployed Windows containers

If you have existing Windows containers created using Migrate for Anthos version 1.7 or earlier, rebuild the container to use a supported Windows architecture. You require the generated artifacts folder generated for the container to make this change:
  1. At the top of the Dockerfile in the generated artifacts folder, edit the FROM command to update the image.

    For example, for the following FROM command:

    FROM mcr.microsoft.com/windows/servercore/iis:windowsservercore:1909

    Replace the image as shown below:

    FROM mcr.microsoft.com/windows/servercore/iis:latest

    The following table shows the existing image and the image used to replace it:

    Existing image name New image name
    mcr.microsoft.com/windows/servercore:1909 mcr.microsoft.com/windows/servercore/iis:latest
    mcr.microsoft.com/windows/servercore/iis:windowsservercore-1909 mcr.microsoft.com/windows/servercore/iis:latest
    mcr.microsoft.com/dotnet/framework/aspnet:3.5-windowsservercore-1909 mcr.microsoft.com/dotnet/framework/aspnet:3.5
    mcr.microsoft.com/dotnet/framework/aspnet:4.8-windowsservercore-1909 mcr.microsoft.com/dotnet/framework/aspnet:4.8
    mcr.microsoft.com/dotnet/framework/wcf:4.8-windowsservercore-1909 mcr.microsoft.com/dotnet/framework/wcf:latest
  2. Rebuild the container images as described at Building the container image.

Update your Migrate for Anthos processing clusters

If you are currently using Migrate for Anthos to migrate new Windows workloads, you must update your version of Migrate for Anthos to 1.7.5 and update your processing cluster to use a node pool of type WINDOWS_LTSC. Node pools of type WINDOWS_SAC are no longer supported.

To update your processing cluster:
  1. Update your version of Migrate for Anthos to 1.7.5.
  2. On your processing cluster, replace the WINDOWS_SAC node pool with WINDOWS_LTSC.

    1. Add the WINDOWS_LTSC new pool:

      gcloud container node-pools create ltsc-pool-name \
        --project=project-name --zone=gcp-zone --cluster=cluster-name \
        --image-type=WINDOWS_LTSC --num-nodes=1 --scopes="cloud-platform" \
        --machine-type=n1-standard-4
    2. Delete the WINDOWS_SAC node pool:

      gcloud container node-pools delete sac-pool-name \
        --project=project-name --zone=gcp-zone --cluster=cluster-name 

Removed from the legacy PV-based Migrate for Anthos versions a Webhook that was simplifying the definition of Migrate for Anthos pods. This Webhook was not being used in any subsequent versions, including the latest 1.6 and 1.7 releases.

186512095: When building a Windows container, you no longer have to edit the generated Dockerfile to change the image tag to latest.

185975192: Webhook certificates no longer expire after one month.

162275866: When generating migration artifacts, you no longer see the following error:

Error: failed to update vgenerateartifactsflow.kb.io

April 6, 2021

On April 6, 2021, we released Migrate for Anthos and GKE 1.7.0.

See Upgrading Migrate for Anthos to 1.7 for upgrade instructions.

This release updates the Linux discovery tool to replace the CSV output with an HTML and JSON output. Also, the fit score of 0-10 has been replaced with a fit assessment value, new rules have been added, and more information about the source VM has been added to the report. For a complete list of updates and information on using the new tool see Using the Linux discovery tool.

By default, Migrate for Anthos automatically disables unneeded services on a VM when you migrate it to a container. You can now edit the migration plan to optionally disable additional services on the container, either services discovered by Migrate for Anthos or your own list of services. See Customizing a migration plan for more.

175000470: The issue caused by adding a source when using a service account without the compute.disks.create permission has been fixed.

178469863: Running migctl setup install with either the --node-selector or --tolerations flag no longer returns an error.

183321483: If you are using a CRD file to create a migration source, and you include a secret in the CRD, then deleting the migration source might also delete the secret.

Workaround: Recreate the secret after deleting the migration source.

162275866: When generating migration artifacts, you might see the following error:

Error: failed to update vgenerateartifactsflow.kb.io:

Post https://controllers-webhook-service.v2k-system.svc:443/validate-anthos-migrate-cloud-google-com-v1beta2-generateartifactsflow?timeout=30s:

no endpoints available for service "controllers-webhook-service"

Workaround: Try generating artifacts again. If not, check the manager status.

  1. Get the pod:

    kubectl get pod -n v2k-system

  2. Look for a pod with name that starts with controllers-controller-manager- and describe it to see its status and event.

179028669: migctl doctor crashes if a GKE cluster is currently repairing.

Workaround: If migctl crashes, then validate that any kubectl command can run on the cluster before retrying the command.

183564181: When you call migctl setup upgrade on Anthos clusters on VMware or Anthos clusters on AWS without the necessary platform argument (--gkeop or --gke-on-aws), migctl uninstalls Migrate for Anthos and fails on the install.

Workaround: Rerun migctl setup upgrade with the correct argument.

179860971: The status error returned when you enter an incorrect VM ID in the console is not helpful:

googleapi: got HTTP response code 404 with body:

Workaround: View the error in the Events tab in the console for more information:

  1. Click the Migrations tab to display a table containing the available migrations.

  2. In the row for the desired migration, select the migration Name to open the Details tab.

  3. Select the Events tab.

182208300: When building a Docker image for a Windows container, examine the logs. Sometimes an internal step can fail, even when the build command shows that the build was successful.

156207267: When using Anthos clusters on VMware for your processing cluster, the migration might get stuck at the image extraction step, and the relevant migration pod shows volume attachment errors similar to the following:

Warning FailedAttachVolume 5s (x10 over 4m15s)
attachdetach-controller AttachVolume.Attach failed for volume migration-b849e1-dd:
Invalid configuration for device 0.

The issue is triggered by a Kubernetes in-tree vsphere driver that is unable to cleanup leftovers from previous migration operations on the same node.

Workaround: Either:

  • Cordon the affected node by:

    1. Running the following command:

      kubectl cordon [node_id]

    2. Restarting the migration.
  • Restart the affected node.

182450827: If you installed Migrate for Compute Engine on your processing cluster to work with Migrate for Anthos, and want to perform four or more concurrent migrations, you cannot use GKE 1.18.

Workaround: Use GKE 1.16 on your processing cluster.

179171930: This release of Migrate for Anthos does not support GKE 1.20.

Workaround: Use GKE 1.19 or earlier.

183082390: The collection script used by the Linux discovery tool uses service --status-all to query system V services. Depending on the services installed on the VM, service --status-all might call custom routines. In some cases, this call might take an arbitrary amount of time depending on the services installed.

186512095: When building a Windows container, you must edit the generated Dockerfile to change the image tag to latest, or to any tag for the image that supports LTSC 1809. For Server Core, replace servercore:1909 with servercore:1809. For example:

from image_name:latest

After completing the installation of Migrate for Anthos, run the following command to apply a required patch:

kubectl patch mutatingwebhookconfigurations.admissionregistration.k8s.io controllers-mutating-webhook-configuration --type json --patch '[{"op": "remove", path: "/webhooks/9"}]'

February 23, 2021

On February 23, 2021, we released Migrate for Anthos and GKE 1.6.2.

180576558: Fixed an issue where the Linux discovery tool calculated an incorrect score.

Fixed an issue where using an Envoy proxy sidecar, not as part of Istio or Anthos Service Mesh, created networking issues with the migrated workload.

February 01, 2021

On February 01, 2021, we released Migrate for Anthos and GKE 1.6.1.

Released a fix, rolling out gradually and taking full effect 2/5/2021, for a migctl setup installation that fails on a GKE cluster when the automatically created bucket already exists.

Released a fix, rolling out gradually and taking full effect 2/5/2021, for a migctl crash when kubectl is not in PATH.

January 25, 2021

On January 25, 2021 we released Migrate for Anthos and GKE 1.6.0.

See Upgrading Migrate for Anthos and GKE to 1.6 for upgrade instructions.

Previous releases of Migrate for Anthos required that you used Google Container Registry (GCR) and Google Cloud Storage for data repositories. This release adds support for additional repositories, including ECR, S3, and Docker registries that support basic authentication. See Defining data repositories for more.

In many on-prem environments, outbound internet access is tightly controlled through the use of an HTTPS proxy server. If your environment uses a proxy server to control outbound internet access, then you can now configure Migrate for Anthos to use that proxy server. See Configuring an HTTPS proxy for more.

Migrate for Anthos now includes the deployment_spec.yaml file in artifacts.zip for Windows migrations. You can use the deployment_spec.yaml file to deploy your migrated Windows workloads. See Deploying a Windows workload to a target cluster for more.

Support added for using Anthos clusters on AWS as processing clusters to perform migrations of AWS workloads. This feature is in preview. See Prerequisites for migrating Linux VMs on AWS for more.

Removed support for the --password option to the migctl command when creating a migration source on Anthos clusters on VMware:

migctl source create local-vmware local-vmware-src --vc '1.2.3.4' --username 'admin' --password 'pass1'

You are now prompted to enter the password. See Adding a migration source for more.

172414359: Exporting multiple cloned VMs simultaneously from the same source might fail.

Workaround: Re-run the migctl migration generate-artifacts command again.

174655315: A migration might stop responding when generating artifacts and remain in the retrying state. You might also see this error in the logs or in the verbose migration status:

D 2020-12-01T18:43:53Z SHELL ERROR: '2020/12/01 18:43:53 appending [/tarlayer/layer.tar.gz]: reading tar "/tarlayer/layer.tar.gz": flate: corrupt input before offset 681999708'

Workaround: Re-run migctl migration generate-artifacts.

175000470: When adding a source when using a service account without the compute.disks.create permission, the source becomes ready but the migration will fail to create disks.

Workaround: Make sure that service account has the compute.disks.create permission.

174299021: When creating a migration source or executing a migration, you might see this error:

"Error: Internal error occurred: failed calling webhook "vmigration.kb.io": Post https://controllers-webhook-service.v2k-system.svc:443/validate-anthos-migrate-cloud-google-com-v1beta2-migration?timeout=30s: unexpected EOF"

Workaround: Recreate the source or migration.

171686793: The migctl setup upgrade --gkeop command might create a new ImageRepositiry or ArtifactRepository object that lacked Google Cloud access credentials.

Workaround: Use the following command to upgrade the cluster:

migctl setup upgrade --json-key key

Where key is the JSON key for the service account required for migctl installation. See Configuring service accounts.

If you try to mount a secret on a deployed pod you will not be able to access it in /run/secrets. This is typically an issue when giving workload identity permissions to the pod (where a secret is added by Kubernetes to hold the workload identity credentials).

The contents of the secrets directory are in /kubernetes-info/secrets.

Workaround: Run the following command on the deployed pod:

ln -s /kubernetes-info/secrets /run/secrets

If the /run mount gets deleted (by a process in the pod, or by a pod reset), you might have to run the command again.

178469863: Running migctl setup install with either the --node-selector or --tolerations flag returns an error.

Note: Running the migctl setup install command with both flags succeeds. This error only occurs when using one flag.

Workaround: Run migctl setup install without the flag, and then manually add the nodeSelectors or tolerations to CSI and Controller pods. See Creating and managing cluster labels and Controlling scheduling with node taints for more.

If you delete the configuration for a Docker image file registry, create a new one with a different configuration name. You cannot recreate a configuration with the name of a previously deleted configuration.

This issue affects Docker image file registries implemented by using GCR or by using Docker registries using basic auth. It does not affect ECR. See Defining data repositories for more information.

Workaround: Use the migctl docker-registry update command to modify an existing configuration rather than deleting it and recreating it.

185975192: Webhook certificates can expire after one month, causing API requests to fail.

November 17, 2020

On November 17, 2020 we released Migrate for Anthos and GKE 1.5.1.

You must upgrade your installation to install 1.5.1, even if you are currently running version 1.5. We strongly recommend that you always upgrade to the latest release.

170604382: Running migctl when not connected to a cluster no longer results in a panic error, but instead returns an error message describing the issue.

169919740: When using a custom services blocklist to disable a service in a workload, if the service was already disabled by default, the migrated container no longer can crash when deployed.

171173082: Mistakenly creating a local VMware source on a Cloud-based cluster, normally used only in an on-prem migration, no longer results in the source being in PROCESSING state forever but instead returns an error.

170566991: For Windows migrations, only HTTP and HTTPS site bindings are supported. See WindowsGenerateArtifacts CRD.

170618192: Similarly to Linux migrations, Windows migrations now add to the generate artifacts object an annotation containing the migration spec and comments.

When creating a migration source for Compute Engine workload, Migrate for Anthos now tests that the GCP specified project exists. Source creation fails when either the project does not exist or the user has no read permissions to the project. An error message is shown indicating that the required project could not be found.

October 21, 2020

On October 21, 2020 we released Migrate for Anthos and GKE 1.5.

Support for migrating Windows VM workloads has moved from the Beta stage to general availability. This release also extends the Google Cloud Console to support migrations of Windows workloads using Migrate for Anthos and GKE. See Migrating a Windows VM for more.

Migrate for Anthos and GKE offers new tools that you run on a Linux or Windows VMs to determine the workload's fit for migration to a container. See Using the Linux discovery tool and Using the Windows discovery tool for more.

Custom Services Blocklist is a new feature that lets you modify the default set of services to disable in a migrated container. See Custom Services Blocklist for more.

The image field value of the GenerateArtifactsFlow CRD defines the names and locations of two images created from a migrated VM. In previous releases, the names contained a predefined tag.

To ensure that the tag value is unique, the format of the tag has changed for this release to specify the timestamp of the migration.

You can also explicitly set the tag if you prefer to another value. See Setting the name of the container image for more.

When you deploy your migrated Windows containers to a cluster, you can now use a Group Managed Service Account (gMSA) to execute the container under a specific service account identity. See Configuring gMSA for more.

Changed the default settings on the Cloud processing cluster for migrating Linux workloads:

  • You no longer have to specify the --scopes "cloud-platform" option when creating Cloud processing clusters for migrating Linux workloads.

  • You now must create a service account, with the necessary permissions, to:

    • Accessing Container Registry and Cloud Storage

    • Use Compute Engine as the migration source

If you currently have a processing cluster that uses the --scopes "cloud-platform" option, your cluster will continue to work. However, for new processing clusters, you should use the new procedure. See Enabling Google services and configuring service accounts for more.

171123825: In some cases, migration process might fail, and Cloud Logging indicate errors such as:

"failed to load map, error 6"

or:

"failed in domap for addition of new path sdd"

Workaround: Delete the migration and restart it. In rare cases, a re-installation of the product is required.

170706786: The Linux Discovery Tool might return exit code 0 even when not all information was collected successfully.

Workaround: Make sure you run the tool as a 'root' user or as a user with full sudo access.

167656057: Installation on a GKE cluster with ACM might fail. Indication of the error can be seen in the Migrate for Anthos and GKE upgrade job, in the v2k-system namespace.

For example:

kubectl logs -n v2k-system controllers-upgrade-fzlmz

Shows this error:

failed to validate admission controller - admission webhook "validation.gatekeeper.sh" does not support dry run

Workaround: gatekeeper is an ACM component. Manually deleting the upgrader job fixes the issue.

For example:

kubectl delete job -n v2k-system controllers-upgrade

157062328: In some cases, adding a service to the blocklist using a configmap will not actually stop that service from running on the deployed workload.

Workaround: Disable the service using the Dockerfile (rather than a config-map), and rebuild the image.

163800225: kubectl port-forward might not work properly for a deployed workload.

Please contact support for more information.

171173082: Mistakenly creating a local VMware source on a Cloud-based cluster, normally used only in an on-prem migration, results in the source being in PROCESSING state forever.

For example, you use migctl to check the source status:

migctl source status local-vmw-src

The State displays as:

PROCESSING Message: Post "https://1.2.3.4/sdk": context deadline exceeded

Workaround: Delete the local VMware source, and create a remote/streaming VMware source.

170604382: Running migctl when not connected to a cluster results in a panic error such as the one below, followed by a stack-trace:

migctl setup install panic: Cannot create kubernetes client

Workaround: Connect a cluster, and re-run migctl.

171714535: In a GKE on-prem environment configured to use an egress HTTP/HTTPS proxy, the migration process might get stuck.

Workaround: Please contact support for more information.

170566991: For Windows migrations, only HTTP and HTTPS site bindings are supported.

Example of unsupported bindings:

<site name="Default Web Site" id="1">
  <application path="/">
    <virtualDirectory path="/" physicalPath="%SystemDrive%\inetpub\wwwroot" />
  </application>
  <bindings>
    <binding protocol="http" bindingInformation="*:80:" />
    <binding protocol="net.tcp" bindingInformation="808:*" />
    <binding protocol="net.pipe" bindingInformation="*" />
    <binding protocol="net.msmq" bindingInformation="localhost" />
    <binding protocol="msmq.formatname" bindingInformation="localhost" />
  </bindings>
</site>

Workaround: Edit the migration-plan to remove the unsupported binding.

169919740: When using a custom services blocklist to disable a service in a workload, ensure that the service is not already disabled by default. See Services disabled by Migrate for Anthos for a list of services disabled by default. If the service was already disabled by default, the migrated container might crash when deployed. Error information about the crash is written to the logs.

Workaround: Remove the already disabled service from your custom services blocklist.

170627229: Migrated workload of a JBoss application might fail at startup. Cloud Logging indicates an error in the form:

ERROR [org.jboss.as.server] (Controller Boot Thread) ...:
Caught exception during boot: java.lang.IllegalStateException: ...:
Could not rename /opt/jboss-7.1.1/standalone/configuration/.../standalone_xml_history/current to
/opt/jboss-7.1.1/standalone/configuration/.../standalone_xml_history/...

Workaround: Update your Dockerfile to remove the directory by adding a line in the form:

RUN rm -rf jboss-path/standalone/configuration/standalone_xml_history/current

Where jboss-path is a path such as /usr/local/share/jboss or /opt/jboss-7.1.1.

144896313: Migrated workloads do not support SELinux. However, if the source VM had SELinux enabled at the time of migration, and you then check the status of SELinux on the migrated workload, SELinux will appear to be enabled. This issue has no affect on migrated workloads.

September 24, 2020

On September 24, 2020 we updated Migrate for Anthos and GKE 1.4.

Changed the default settings on the Cloud processing cluster for migrating Linux workloads:

  • You no longer have to specify the --scopes "cloud-platform" option when creating Cloud processing clusters for migrating Linux workloads.

  • You now must create a service account, with the necessary permissions, to:

    • Accessing Container Registry and Cloud Storage

    • Use Compute Engine as the migration source

If you currently have a processing cluster that uses the --scopes "cloud-platform" option, your cluster will continue to work. However, for new processing clusters, you should use the new procedure. See Enabling Google services and configuring service accounts for more.

You can now use the Google Cloud Console to:

  • Install Migrate for Anthos on a processing cluster
  • Create a migration source for a Compute Engine VM

See Installing Migrate for Anthos and Adding a migration source for more.

July 28, 2020

On July 28, 2020 we released Migrate for Anthos and GKE 1.4.

For instructions on upgrading from version 1.3, see Upgrading Migrate for Anthos and GKE to 1.4.

Added support for Anthos GKE on-prem clusters running on VMware. On-prem support lets you migrate source VM workloads in a vCenter/vSphere environment to a GKE on-prem cluster running in the same vCenter/vSphere environment. See Prerequisites for migrating Linux VMs on-prem for the requirements for on-prem migration.

The Google Cloud Console provides a web-based, graphical user interface that you can use to manage your Google Cloud Console(GCP) projects and resources. Migrate for Anthos and GKE now supports the migration of workloads by using the Google Cloud Console. See Migrate for Anthos management interfaces.

In this release, the Migrate for Anthos and GKE Google Cloud Console does not support migrations for Windows or for on-prem, including monitoring Windows or on-prem migrations.

You can use Migrate for Anthos and GKE to migrate Windows VMs to workloads on GKE. This process clones your Compute Engine VM disks and uses the clone to generate artifacts (including a Dockerfile and a zip archive with extracted workload files and settings) you can use to build a deployable container image. This feature is currently in Beta. See Adding a Windows migration source.

Migrate for Anthos and GKE now includes Custom Resource Definitions (CRDs) that enable you to easily create and manage migrations using an API automation solution or code. For example, you can use these CRDs to build your own automated tools.

For more on the Migrate for Anthos and GKE CRDs, see CRD overview.

Added the new --json-key sa.json option to the migctl source create ce command to create a migration for Compute Engine, where sa.json specifies a service account. See Optionally creating a service account when using Compute Engine as a migration source for more.

To edit the migration plan, you must now use the migctl migration get my-migration command to download the plan. After you are done editing the plan, you have to upload it by using the migctl migration update my-migration command. See Customizing a migration plan for more.

Added the node-selectors and tolerations options to the migctl setup install installation command that lets you install Migrate for Anthos and GKE on a specific set of nodes or node pools in a cluster. See Installing Migrate for Anthos.

The migctl migration cleanup command has been removed and is no longer necessary.

In previous releases, you used a command in the form: migctl source create ce my-ce-src --project my-project --zone zone to create a migration for Compute Engine. The --zone option has been removed when creating a Compute Engine migration. Using the --zone option in this release causes an error.

The migctl migration logs command has been removed. You now use the Google Console to view logs.

By default Migrate for Anthos and GKE installs to and performs migrations in the v2k-system namespace. In previous releases, you could specify the namespace. The option to specify a namespace has been removed.

GKE on-prem preview: If a source was created with migctl source create using the wrong credentials, you could not delete the migration with migctl migration delete. This issue has been fixed in the GA release of on-prem support.

160309992: Editing a migration plan from the GUI console might fail if it was also edited using migctl.

161135630: Attempting multiple migrations of the same remote VM (from VMware, AWS or Azure) simultaneously, might result in a stuck migration process.

Workaround: Delete the stuck migration.

161214397: In case of a missing service-account to upload container images to the Container Registry, the migration might get stuck.

Workaround: Add the service-account. If you are using the Migrate for Anthos and GKE CRD API, delete the GenerateArtifactsTask object using kubectl and recreate it either using the CRD or re-running migctl migration generate-artifacts. Alternatively, you can use migctl to delete the migration and recreate it. You can first download the migration YAML using migctl migration get to backup any customizations you have made.

161110816: migctl migration create with a source that doesn't exist fails with a non-informative error message: request was denied.

161104564: Creating a Linux migration with wrong os-type specification causes the migration process to get stuck until deleted.

160858543, 160836394, 160844377, 154430477, 154403665, 153241390,153239696, 152408818, 151516642, 132002453: Unstable network in Migrate for Compute Engine infrastructure, or a GKE node restart, might cause migration to get stuck.

Workaround: Delete the migration and re-create it. If recreating the migration does not solve the issue, please contact support.

161787358: In some cases, upgrading from version v1.3 to v1.4 might fail with Failed to convert source message.

Workaround: Re-run the upgrade command.

153811691, 153439420: Migrate for Anthos and GKE support for older Java does not handle OpenJDK 7 and 8 CPU resource calculations.

152974631: Using GKE nodes with CPU and Memory configurations below the recommended values might cause migrations to get stuck.

157890913, 160082702, 161125635, 159693579:A migration might continue to indicate that it is running, while an issue encountered prevents further processing.

Workaround: Check event messages on the migration object using the verbose migctl status command: migctl migration status migration_name -v. You might be able to correct the issue to allow the migration to continue or the migration should be deleted and recreated if an Error event is listed without further retries.

An example is when creating a Windows migration on a cluster with no Windows nodes. In this case the event message will show:

Warning FailedScheduling 10s Pod discover-xyz 0/1 nodes are available: 1 node(s) didn't match node selector.

March 30, 2020

v1.3.0

New migctl CLI for deploying Migrate for Anthos, creating and operating migrations using a structured workflow and a migration processing cluster.

Introducing a unified migration workflow across all supported VM sources -- VMware, AWS EC2, Azure VMs and Compute Engine VMs.

Migrations are defined and operated using a Kubernetes CRD.

Automated generation of a suggested migration plan (specified in a CRD), CI/CD artifacts and deployment specs. The migration process now results in extracting and generating container and deployment artifacts, including a container image and a Dockerfile, extracted data in a persistent volume, deployment/statefulSet, PVC and PV specs in an auto-generated YAML file for easy workload deployment.

The Migrate for Anthos software runtime layer now offers a compatibility feature for older Java versions that are not container aware by reflecting the correct resource allocations in the container's /proc file system.

Migrate for Anthos v1.0 Marketplace deployment is now removed. Migrate for Anthos v1.3 allows installation in v1.0 compatibility mode where needed.

Preview features -- contact your Google Sales representative to enroll.

  • Migrating Windows VMs with IIS ASP.NET web applications to Windows 2019 containers on GKE.
  • Processing migrations in Anthos on-prem.

147211918: In some cases, migration from AWS or Azure as a source can be stuck with no progress. If this happens, run kubectl describe storageclass to view related events. You can also check the status of the matching Cloud Details in Migrate for Compute Engine.

146699220: When the source VM has a systemd service with a NICE config property, the service might not start when running in a container.

Workaround: Remove the NICE property in the source VM before the migration.

144896313: Migration of Security-Enhanced Linux (SELinux) is not supported.

149900626: Some file systems not listed in Compatible VM operating systems may fail to migrate. When running migctl migration logs migration-name, the logs in Cloud Logging may show a message such as:

failed to mount - exit status 32 - mount: /tmp/bootdir: unknown filesystem type 'LVM2\_member'.

152194161: Your migrated workload container fails when running a cluster with GKE nodes of type "COS". When you run the command kubectl logs [podname], you might see the following:

apparmor.go:385] Couldn't set label to lxc-container-default - write /proc/1/attr/current: no such file or directory

This is an indication that the required AppArmor profiles are not installed on the GKE nodes. To solve this, run migctl setup install --cos-runtime.

148334068: When Migrating a physical VM from on-premises connected via Migrate for Compute Engine, Migrate for Anthos attempts to optimize network utilization and discards (rather than stream) blocks that are not in use on the source VM file system. In some cases, this might cause VMware storage sessions to time out. For assistance, please contact support.

GKE on-prem preview: If a source was created with migctl source create using the wrong credentials, migrations will fail. Attempts to delete the migration with migctl migration delete may stop responding in a "Terminating" state, as in the following example:

$ migctl migration list
NAME       PROCESS              STATE                   STATUS        PROGRESS   AGE
my-vm-01   generate-artifacts   createSourceSnapshots   TERMINATING   [2/13]

December 19, 2019

v1.0.1

When specifying a mixed-case value for the clone_vm_disks script's -A <var>app-name</var> argument, the YAML file generated by the script would include a workload name that could not be deployed.

The command now checks for a valid input value.

The Migrate for Compute Engine password could be inadvertently logged in Stackdriver Logging.

Migrate for Anthos failed to recognize a reference to a disk specified as a PARTUUID in the fstab file. PARTUUID is now supported.

Deleting a StatefulSet attached to a persistent volume would leave the volume in an attached state./p>

Using configuration YAML from a version prior to 1.0 caused the pod to enter a crashloop stage.

An error message now is now displayed to request that you update to the latest definition.

When resolving block devices in a multipath device, the operation appeared to succeed even if there was an error with one of the block devices.

Resolving source storage devices would sometimes fail without error if one of the devices has no partitions.

Using kubectl exec on a migration pod would sometimes display superfluous bash warnings about LC_ALL.

Attempting to switch to a non-root user with the su command after connecting to the machine with ssh would fail when you had previously used su to switch to another user.

Migrate for Anthos CSI drive would sometimes fail connecting to the migrated VM.

The kubectl cp command would fail when copying files to the migrated pod.

November 13, 2019

v1.0.0

Migrate for Anthos supports migrating existing VMware, Amazon EC2, Azure, and Compute Engine VMs to containers on Google Kubernetes Engine. For more information, see Benefits of Migrate for Anthos.

You can monitor export of short-term storage to a persistent volume using kubectl. For more information, see Exporting streaming PVs to permanent storage.

Using a ConfigMap, you can have content from application log files you specify written to Stackdriver Logging (a default list is included). For more, see Configuring logging to Stackdriver Logging.

For information on operating systems supported by Migrate for Anthos, see Compatible VM operating systems.

On the Migrate for Compute Engine portlet in VMWare vCenter, VMs will be shown as Managed by Migrate for Compute Engine during migration process. Only the cache and storage migration status are updated in this view. Other functionality, such as Migrate for Compute Engine actions, may not be functional.

For known issues and workarounds, see Troubleshooting Migrate for Anthos.

VMs using EFI configurations are not compatible for migration with this release.

Operating systems running systemd versions lower than 234 are limited to 65536 open files.

When using a private GKE cluster, the GKE control plane might be unable to reach Migrate for Anthos infrastructure (specifically, the admission-controller) by default. This is because the admission-controller Pod listens on port 9443.

To work around this issue, add port 9443 to the firewall rules of the control plane node. For more, see Creating a private cluster.

When specifying a mixed-case value for the clone_vm_disks script's -A <var>app-name</var> argument, the YAML file generated by the script includes a workload name that can not be deployed.

To work around this issue, specify the argument's value in lowercase only.

The Migrate for Compute Engine password can be inadvertently logged in Stackdriver Logging.

Migrate for Anthos fails to recognize a reference to a disk specified as a PARTUUID in the fstab file.

Deleting a StatefulSet attached to a persistent volume will leave the volume in an attached state.

Using configuration YAML from a version prior to 1.0 causes the pod to enter a crashloop stage.

To work around this, update your YAML file to conform to the latest definition.

When resolving block devices in a multipath device, the operation appears to succeed even if there was an error with one of the block devices.

Resolving source storage devices would sometimes fail without error if one of the devices has no partitions.

Using kubectl exec on a migration pod sometimes displays superfluous bash warnings about LC_ALL. These are only cosmetic.

Attempting to switch to a non-root user with the su command after connecting to the machine with ssh fails when you have previously used su to switch to another user.

To work around this issue, use kubectl exec instead of ssh to get a shell to the container.

Migrate for Anthos CSI drive may sometimes fail connecting to the migrated VM.

The kubectl cp command fails when copying files to the migrated pod.