Troubleshooting

Beta

This product or feature is in a pre-release state and might change or have limited support. For more information, see Product launch stages.

Learn about troubleshooting steps that you might find helpful if you run into problems using Migrate for Anthos.

I cannot install Migrate for Anthos

Application took too long to deploy

If you see the error Application took too long to deploy when deploying Migrate for Anthos from GCP Marketplace, re-create your cluster manually.

After you have re-created your cluster, re-install Migrate for Anthos.

My application won't start

What's happening to my app? Where can I see logs?

See Monitoring migrated workloads for more information on logging.

My application pods / workloads have a status of "Unschedulable" for more than 10 minutes

In your Migrate for Anthos deployment setup, check that your:

• Migrate for Compute Engine Manager IP is correct.
• Migrate for Compute Engine Manager is reachable from the GKE cluster's subnet.
• Migrate for Compute Engine API password is correct.

My pods do not change from "Pending" status

During Compute Engine to GKE migrations, Migrate for Anthos may fail to recognize the UUID of the source VM's disk. You can add it manually:

1. Load the logs for the pod using kubectl or Stackdriver.
2. If you see the message [hcrunner] - Failed to find boot partition, continue with the following steps.

   1. Find the UUID for the boot disk printed in one of the messages, which will be a string of hexadecimal values. In the example below, the UUID is e823158e-f290-4f91-9c3d-6f33367ae0da.

      [util] - SHELL OUTPUT: {"name": "/dev/sdb1", "partflags": null, "parttype":
      "0x83", "uuid": "<strong>e823158e-f290-4f91-9c3d-6f33367ae0da</strong>",
      "fstype": "ext4"}
      

   2. Delete the existing workload using its YAML file.

      kubectl delete -f DEPLOYMENT_YAML
      

   3. Open the YAML file in a text editor.

   4. Find the section named env.

   5. Add the following:

            - name: "HC_BOOTDEVICE_UUID"
              value: "[UUID]"
      

• If you see the message touch: cannot touch '/vlsdata/etc/fstab': No such file or directory check the following:

  • Your CSI driver workloads have a status of OK in the GKE console.
  • Your workload is in the same cluster as your Migrate for Anthos deployment.

• If you see one of the following messages:

  • hcutil.Error: Failed mount -o rw None /vlsdata (32) (Output:mount: /vlsdata: special device None does not exist.
  • [hcrunner] - [Errno 30] Read-only file system: '/vlsdata/rootdir/etc/dhcp/dhclient-up-hooks

  Delete the workload's failing PersistentVolumeClaim and recreate it.

Debugging Kubernetes resources

More help is available at the following pages:

• If you are experiencing a problem with Google Kubernetes Engine, see the GKE troubleshooting page.

• If you are experiencing an issue related to your cluster, refer to Troubleshooting Clusters in the Kubernetes documentation.

• If you are having an issue with your application, its pods, or its controller object, refer to Troubleshooting Applications.

I would like personalized support

Paid support is available for customers migrating with Migrate for Anthos. Reach out so we can help.

Providing information to GCP support

The Sysreport provides Migrate for Anthos support with information about your cluster's configuration for faster time to problem resolution.

You can access the script from Cloud Shell.

1. Open Cloud Shell

Next, run the collect_sysreport.sh script.

/google/migrate/anthos/collect_sysreport.sh [NAMESPACE] [DEPLOYMENT_NAME] [--workloads]

Where:

• [NAMESPACE] is the namespace where your Migrate for Anthos components were installed.
• [DEPLOYMENT_NAME] is name given when you created your Migrate for Anthos deployment in the GKE marketplace.
• --workloads collects additional data from your migrated workloads. See below for more information.

The script creates anthos-migrate-logs.TIMESTAMP.tar.xz, which you provide to GCP support.

By default, the script collects:

• Logs from the Migrate for Anthos CSI controller and CSI nodes.
• Syslog from the Migrate for Anthos CSI node hosts.
• The output of:
  • kubectl cluster-info
  • kubectl get nodes; kubectl describe node
  • kubectl version
  • kubectl top node

With the --workloads flag enabled, for every workload the script collects:

• The workload's logs.
• The output of:
  • ps aux
  • netstat -tlnp
  • iptables -t nat -L
  • fstab
  • kubectl get pod
  • kubectl describe pod
  • kubectl top pod --all-namespaces --containers
  • kubectl cluster-info dump
  • kubectl api-resources -o wide
  • kubectl top pod --all-namespaces --containers
  • kubectl api-resources -o wide
  • kubectl get componentstatuses --all-namespaces
  • kubectl get endpoints --all-namespaces
  • kubectl get events --all-namespaces
  • kubectl describe limits --all-namespaces
  • kubectl get namespaces
  • kubectl describe pvc --all-namespaces
  • kubectl describe pv --all-namespaces
  • kubectl describe quota --all-namespaces
  • kubectl describe sa --all-namespaces
  • kubectl describe services --all-namespaces
  • kubectl describe services --all-namespaces
  • kubectl get ingresses --all-namespaces
  • kubectl describe networkpolicies --all-namespaces
  • kubectl get podsecuritypolicies --all-namespaces
  • kubectl get clusterrolebindings --all-namespaces
  • kubectl describe storageclasses --all-namespaces
  • kubectl describe volumeattachments --all-namespaces