Importing virtual disks


If you have virtual disks in your on-premises environment with software and configurations that you need (sometimes referred to as golden disks or golden images), you can save time by importing those virtual disks into Compute Engine and using the resulting image to create virtual machines. The import tool supports most virtual disk file formats, including VMDK and VHD.

If you exported your disk from Compute Engine, you can create images from the disk.

For information about how to create an automated system for migrating several virtual machines (VMs), see Migrating VMs to Compute Engine.

Before you begin

Enable the Cloud Build API

The import tool uses Cloud Build.

In most cases, gcloud compute images import attempts to grant these permissions to the Cloud Build service account. However, you can manually grant these permissions to ensure that the required permissions are in effect.

Console

  1. Enable the Cloud Build API.

    Enable the Cloud Build API

    When you enable the Cloud Build API from the console, Compute Engine grants the Cloud Build service account the following roles so that the Cloud Build service can import instances into Compute Engine:

    • roles/iam.serviceAccountTokenCreator
    • roles/compute.admin
    • roles/iam.serviceAccountUser

    The import tool also uses the default Compute Engine service account. By default, the Compute Engine service account has the IAM project editor role. If this role is removed, the import process might fail. To add the role back to the service account, see Granting access. For more information about the Compute Engine default service account, see Compute Engine default service account.

gcloud

To set up the Cloud Build service using gcloud command-line tool, complete the following steps:

  1. Enable Cloud Build.

    gcloud services enable cloudbuild.googleapis.com

    The import tool also uses the default Compute Engine service account. By default, the Compute Engine service account has the IAM project editor role. If this role is removed, the import process might fail. To add the role back to the service account, see Granting access. For more information about the Compute Engine default service account, see Compute Engine default service account.

  2. Add the compute.admin role to the service account for the Cloud Build API.

    gcloud projects add-iam-policy-binding project-id \
       --member serviceAccount:project-num@cloudbuild.gserviceaccount.com \
       --role roles/compute.admin
    
  3. Add the iam.serviceAccountUser role to the service account for the Cloud Build API.

    gcloud projects add-iam-policy-binding project-id \
       --member serviceAccount:project-num@cloudbuild.gserviceaccount.com \
       --role roles/iam.serviceAccountUser
    
  4. Add the iam.serviceAccountTokenCreator role to the service account for the Cloud Build API.

    gcloud projects add-iam-policy-binding project-id \
       --member serviceAccount:project-num@cloudbuild.gserviceaccount.com \
       --role roles/iam.serviceAccountTokenCreator
    

    Replace the following:

Supported operating systems

You can import both bootable and non-bootable disks. For your virtual disks to be bootable on Compute Engine, they must run one of the supported operating systems.

Limitations

This feature has the following general limitations:

  • If you are importing a virtual disk running RHEL, bring your own license (BYOL) is supported only if the python-boto package is installed on the virtual disk prior to import.

  • Operating systems on virtual disks must support ACPI.

This feature has the following limitations on Linux:

  • Linux virtual disks must use grub as the bootloader.

  • Linux virtual disks must meet the same requirements as custom images, including support for Virtio-SCSI Storage Controller devices.

This feature has the following limitations on Windows:

  • When installed on Windows virtual disks, application-allowlisting software, such as CB Protection by Carbon Black, can cause the import process to fail. You might need to uninstall such software prior to import.

  • On Windows-based VMs, the Microsoft KM-TEST Loopback Adapter prevents network access to the metadata server and you must disable or remove it before import.

Permissions

The image import tool performs several steps when you import a virtual disk file including uploading your file to Cloud Storage, creating a new bucket if necessary, downloading the file to Compute Engine, and then creating an image in Compute Engine from the disk file. This process happens automatically. To enable a seamless experience when using this feature, Google recommends that your account have the following roles:

  • roles/storage.admin
  • roles/viewer
  • roles/resourcemanager.projectIamAdmin

The import process uses the default Compute Engine Service account as part of its workflow. By default, this account has the roles/editor permission, which is sufficient for the process. However, if you have modified the default roles and permissions for the Compute Engine Service account, ensure that the service account still has the following roles applied:

  • roles/compute.storageAdmin
  • roles/storage.objectViewer

Checking for compatibility

Before you attempt to import the disk for your VM, download and run the precheck tool inside your VM. The precheck tool scans for any compatibility issues that might cause the import process to fail or the disk to not work properly on Compute Engine.

Importing virtual disks

Importing a bootable virtual disk

Console

  1. In the Google Cloud Console, upload the virtual disk file to Cloud Storage.
  2. Go to the Create an image page .

    Go to the Create an image page

  3. Specify a Name for your image.

  4. Under Source, select Virtual disk (VMDK, VHD,..).

  5. Browse to or manually input the storage location for the Cloud Storage file.

  6. Select the operating system that is available on the imported disk. You can also make the following changes:

    • You can choose to Install guest packages. Google recommends that you install the guest environment. For more information about the guest environment, see guest environment.

    • For Windows or Red Hat Enterprise Linux (RHEL) operating systems, you can also choose a licensing option. You can either allow Compute Engine to provide a license or you can bring your own license. For more information about bringing your own license on Windows, see Bring your own license.

  7. (Optional) Specify additional properties for your image. For example, you can organize this image as part of an image family.

  8. Click Create to import the image.

gcloud

Use the gcloud compute images import command to create a bootable Compute Engine image. Although Compute Engine can boot most boot disk images, the import command ensures that the disk has the required drivers and latest guest environment packages, which are required to start an instance and connect to it using SSH or RDP.

You can import virtual disk files from either a Cloud Storage bucket or from your local workstation.

If you import the virtual disk file from your workstation, the import tool automatically uploads the file to a Cloud Storage bucket for you.

If you prefer, you can upload the virtual disk file to Cloud Storage yourself before you start the import process, but you must upload the file to a Cloud Storage bucket in the same project that will be used for the import process.

gcloud compute images import image-name \
    --source-file source-file \
    --os os

Replace the following:

  • image-name: The name of your destination image.
  • source-file: Your virtual disk file. It can be a local file or a file stored in Cloud Storage. If your virtual disk is a local file, you can provide an absolute or relative path. If your virtual disk file is already stored in Cloud Storage, the file must exist in a Cloud Storage bucket in the project that is used for the import process, and you must specify the full path of the file in the gs://bucket-name/object-name format.
  • os: The operating system of the --source-file. For a list of supported values, review the --os flag options for the gcloud compute images import command.

    For instructions on how to import images with existing licenses to Google Cloud, see Bringing your own licenses.

If you specify a local file, the upload operation can take a long time depending on the size of your virtual disk and the speed of your network connection. The import operation can take tens of minutes to run depending on the size of the disk.

Sample command

The following example imports a debian-9 virtual disk named my_server.vmdk stored in gs://your_gcs_bucket.

gcloud compute images import my-imported-image \
    --source-file gs://your_gcs_bucket/my_server.vmdk \
    --os debian-9

Optional parameters

By default guest environment packages are added to all imported boot disk images. If you do not want these packages, add the --no-guest-environment flag to your import command.

API

  1. Add the virtual disk to Cloud Storage.

  2. In the API, create a POST request to the Cloud Build API.

    POST https://cloudbuild.googleapis.com/v1/projects/project-id/builds
    {
     "steps":[
       {
         "args":[
           "-image_name=image-name",
           "-source_file=source-file",
           "-os=os",
           "-timeout=7000s",
           "-client_id=api"
         ],
         "name":"gcr.io/compute-image-tools/gce_vm_image_import:release",
         "env":[
           "BUILD_ID=$BUILD_ID"
         ]
       }
     ],
     "timeout":"7200s",
     "tags":[
       "gce-daisy",
       "gce-daisy-image-import"
     ]
    }
    

    Replace the following:

    • project-id: The project ID for the project that you want to import the image into.
    • image-name: The name of the image to be imported.
    • source-file: The URI for the image in Cloud Storage. For example, gs://my-bucket/my-image.vmdk.
    • os: The operating system of the image.

    For additional args values that can be provided, see the optional flags section of the VM image import GitHub page.

    Example response

    The following sample response resembles the output that is returned:

    {
     "name": "operations/build/myproject-12345/operation-1578608233418",
     "metadata": {
      "@type": "type.googleapis.com/google.devtools.cloudbuild.v1.BuildOperationMetadata",
      "build": {
       "id": "3a2055bc-ccbd-4101-9434-d376b88b8940",
       "status": "QUEUED",
       "createTime": "2019-09-20T15:55:29.353258929Z",
       "steps": [
        {
         "name": "gcr.io/compute-image-tools/gce_vm_image_import:release",
         "env": [
          "BUILD_ID=3a2055bc-ccbd-4101-9434-d376b88b8940"
         ],
         "args": [
          "-timeout=7056s",
          "-image_name=my-image",
          "-client_id=api",
          "-data-disk",
          "-source_file=gs://my-bucket/my-image.vmdk"
         ]
        }
       ],
       "timeout": "7200s",
       "projectId": "myproject-12345",
       "logsBucket": "gs://123456.cloudbuild-logs.googleusercontent.com",
       "options": {
        "logging": "LEGACY"
       },
       "logUrl": "https://console.cloud.google.com/gcr/builds/3a2055bc-ccbd-4101-9434-d376b88b8940?project=123456"
      }
    }
    

    There are a couple ways you can monitor your build:

    • Run a projects.builds.get request using the returned build-id.
    • Review the logs hosted at the provided logUrl.

Importing a non-bootable virtual disk

Console

  1. In the Google Cloud Console, upload the virtual disk file to Cloud Storage.
  2. Go to the Create an image page.

    Go to the Create an image page

  3. Specify a Name for your image.

  4. Under Source, select Virtual disk (VMDK, VHD, ...).

  5. Browse to or manually input the storage location for the Cloud Storage file.

  6. Under operating system, select No operating system. Data only.

  7. (Optional) Specify additional properties for your image. For example, you can organize this image as part of an image family.

  8. Click Create to import the image.

gcloud

You can use the gcloud compute images import command to create a non-bootable Compute Engine image. If your virtual disk does not have a bootable operating system installed on it, you can still import it using the --data-disk flag in place of the --os flag. This skips over the step that installs drivers and guest environment packages to make the image bootable on Compute Engine.

gcloud compute images import image-name \
    --source-file source-file \
    --data-disk

Replace the following:

  • image-name: name of your destination image.
  • source-file: your virtual disk file. It can be a local file or a file stored in Cloud Storage. If your virtual disk is a local file, you can use an absolute or relative path. If your virtual disk file is already stored in Cloud Storage, the file must exist in a Cloud Storage bucket in the project that is used for the import process, and you must specify the full path of the file in the gs://bucket-name/object-name format.

Sample command

The following example imports a virtual disk named my-disk.vmdk stored in gs://my-gcs-bucket/.

gcloud compute images import my-imported-image \
    --source-file gs://my-gcs-bucket/my-disk.vmdk \
    --data-disk

API

  1. Add the virtual disk to Cloud Storage.

  2. In the API, create a POST request to the Cloud Build API.

    POST https://cloudbuild.googleapis.com/v1/projects/project-id/builds
    {
      "steps":[
        {
          "args":[
            "-image_name=image-name",
            "-source_file=source-file",
            "-timeout=7000s",
            "-client_id=api",
            "-data_disk"
          ],
          "name":"gcr.io/compute-image-tools/gce_vm_image_import:release",
          "env":[
            "BUILD_ID=$BUILD_ID"
          ]
        }
      ],
      "timeout":"7200s",
      "tags":[
        "gce-daisy",
        "gce-daisy-image-import"
      ]
    }
    

    Replace the following args values:

    • project-id: project ID for the project that you want to import the image into.
    • image-name: name of the image to be imported.
    • source-file: URI for the image in Cloud Storage. For example, gs://my-bucket/my-image.vmdk.

Importing disks using networks that don't allow external IP addresses

You can import virtual disks using a network that does not allow external IPs by using the Cloud Build API and specifying the -no_external_ip argument.

With this method, Cloud Build attempts to import the image without using an external IP.

  1. Add the virtual disk to Cloud Storage.

  2. In the API, create a POST request to the Cloud Build API.

    POST https://cloudbuild.googleapis.com/v1/projects/project-id/builds
    {
    "steps":[
     {
       "args":[
         "-image_name=image-name",
         "-source_file=source-file",
         "-os=os",
         "-no_external_ip",
         "-timeout=7000s",
         "-client_id=api"
       ],
       "name":"gcr.io/compute-image-tools/gce_vm_image_import:release",
       "env":[
         "BUILD_ID=$BUILD_ID"
       ]
     }
    ],
    "timeout":"7200s",
    "tags":[
     "gce-daisy",
     "gce-daisy-image-import"
    ]
    }
    

    Replace the following:

    • project-id: project ID for the project that you want to import the image into.
    • image-name: name of the image to be imported.
    • source-file: URI for the image in Cloud Storage. For example, gs://my-bucket/my-image.vmdk.
    • os: operating system of the image.

Importing a virtual disk with UEFI bootloader

You can import virtual disks with an UEFI bootloader by using the Cloud Build API and specifying the -uefi_compatible argument.

  1. Add the virtual disk to Cloud Storage.

  2. In the API, create a POST request to the Cloud Build API.

    POST https://cloudbuild.googleapis.com/v1/projects/project-id/builds
    {
    "steps":[
     {
       "args":[
         "-image_name=image-name",
         "-source_file=source-file",
         "-os=os",
         "-uefi_compatible",
         "-timeout=7000s",
         "-client_id=api"
       ],
       "name":"gcr.io/compute-image-tools/gce_vm_image_import:release",
       "env":[
         "BUILD_ID=$BUILD_ID"
       ]
     }
    ],
    "timeout":"7200s",
    "tags":[
     "gce-daisy",
     "gce-daisy-image-import"
    ]
    }
    

    Replace the following:

    • project-id: project ID for the project that you want to import the image into.
    • image-name: name of the image to be imported.
    • source-file: URI for the image in Cloud Storage. For example, gs://my-bucket/my-image.vmdk.
    • os: operating system of the image.

Importing a virtual disk using shared VPC

Because the import tool uses Cloud Build API, before you can import a virtual disk by using a shared VPC, you must add the compute.networkUser role to the service account for the Cloud Build API.

gcloud projects add-iam-policy-binding host-project-id \
   --member serviceAccount:service-project-num@cloudbuild.gserviceaccount.com \
   --role roles/compute.networkUser

Replace the following:

  • host-project-id: ID of the project where the shared VPC is located.
  • service-project-num: project number for the project that you want to import the image to.

gcloud

Use the gcloud compute images import command to import your virtual disk. The following command shows the specifications for importing either a non-bootable (--data-disk) or a bootable disk (--os). Choose one option.

gcloud compute images import image-name \
    --source-file source-file \
    [--data-disk|--os os] \
    --project service-project-id \
    --zone zone \
    --network network \
    --subnet subnet

Replace the following:

  • image-name: name of your image to import.
  • source-file: your virtual disk file. It can be a local file or a file stored in Cloud Storage. If your virtual disk is a local file, you can provide an absolute or relative path. If your virtual disk file is already stored in Cloud Storage, the file must exist in a Cloud Storage bucket in the project that is used for the import process, and you must specify the full path of the file in the gs://bucket-name/object-name format.
  • Specify one of the following:
    • --data-disk: use this flag if the disk does not have a bootable operating system installed on it.
    • os: operating system of the --source-file. For a list of supported values, review the --os flag options for the gcloud compute images import command.
  • service-project-id: ID of the project that you want to import the image into.
  • zone: zone that you want to import the image to. This zone must match the region of the subnet. For example, if the subnet is us-west1. The import zone must be one of the following: us-west1-a, us-west1-b or us-west1-c.
  • network: full path to a shared VPC network. For example, projects/host-project-id/global/networks/vpc-network-name.
  • subnet: full path to a shared VPC subnetwork For example, projects/host-project-id/global/networks/vpc-subnet-name.

    Sample command

    gcloud compute images example-image \
     --source-file gs://source-bucket/source-file.vmdk \
     --os centos-7 \
     --project my-image-project \
     --zone us-west1-c \
     --network projects/my-vpc-project/global/networks/my-shared-vpc \
     --subnet projects/my-vpc-project/regions/us-west1/subnetworks/my-shared-subnet
    

API

  1. Add the virtual disk to Cloud Storage.

  2. In the API, create a POST request to the Cloud Build API.

    POST https://cloudbuild.googleapis.com/v1/projects/service-project-id/builds
    {
     "steps":[
       {
         "args":[
           "-image_name=image-name",
           "-source_file=source-file",
           "-os=os",
           "-zone=zone",
           "-network=network",
           "-subnet=subnet",
           "-timeout=7000s",
           "-client_id=api"
         ],
         "name":"gcr.io/compute-image-tools/gce_vm_image_import:release",
         "env":[
           "BUILD_ID=$BUILD_ID"
         ]
       }
     ],
     "timeout":"7200s",
     "tags":[
       "gce-daisy",
       "gce-daisy-image-import"
     ]
    }
    

    Replace the following args values:

    • service-project-id: ID for the project that you want to import the image into.
    • image-name: name of the image to be imported.
    • source-file: URI for the image in Cloud Storage. For example, gs://my-bucket/my-image.vmdk.
    • os: operating system of the --source-file.
    • zone: zone that you want to import the image to. This zone must match the region of the subnet. For example, if the subnet is us-west1. The import zone must be one of the following: us-west1-a, us-west1-b or us-west1-c.
    • network: full path to a shared VPC network. For example, projects/host-project-id/global/networks/vpc-network-name.
    • subnet: full path to a shared VPC subnetwork For example, projects/host-project-id/global/networks/vpc-subnet-name.

Making an image bootable

If you have a Compute Engine custom image that has a bootable operating system on it but does not have the necessary Compute Engine drivers or guest environment packages, you can use the image import tool to make that image bootable on Compute Engine.

Use the --source-image flag to specify a custom image to make bootable, instead of using the --source-file flag that specifies a new disk to import.

gcloud compute images import image-name \
    --source-image source-image-name \
    --os os

Replace the following:

  • image-name: The name of your destination image.
  • source-image-name: The name of your source image.
  • os: The operating system of the --source-image. For a list of supported values, review the --os flag options for the gcloud compute images import command.

    For instructions on how to import images with existing licenses to Google Cloud, see Bringing your own licenses.

Sample command

The following example turns a Compute Engine image named my-image into a bootable image named my-bootable-image. In this example, the operating system installed on the image is Ubuntu 16.04.

gcloud compute images import my-bootable-image \
    --source-image my-image \
    --os ubuntu-1604

Resource cleanup

Files stored on Cloud Storage and images in Compute Engine incur charges. The import tool imports the virtual disk file to Cloud Storage and creates one Compute Engine custom image.

After you verify that the image is imported correctly and that it boots correctly as a Compute Engine instance, you can delete the virtual disk file from Cloud Storage. The tool prints the URI of the file as it uploads it to Cloud Storage. This URI has the following form: gs://bucket-name/tmpimage/image-name.

If you imported an image using the --data-disk flag and then ran the import tool a second time with the --source-image flag to make that image bootable, then the first image still exists. If you have no need for it, consider deleting that image. If you specify the same image name for both the --image and --source-image flags, then the image is automatically overwritten and no further cleanup is required.

What's next