Importing Boot Disk Images to Compute Engine

This guide shows you how to import boot disks from your existing systems to Google Compute Engine. You can import boot disk images from your physical datacenters, from virtual machines on your local workstation, or from virtual machines that run on another cloud platform. The image import process can import only one disk at a time, and this guide focuses on how to import boot disk images.

Import your existing boot disks only if you are unable to build or migrate your applications to run on top of Compute Engine public images. Public images are already configured to run in the Compute Engine environment, so you can run applications on those images without having to worry about bootloader and operating system configurations. However, you might need to import your own boot disk images in the following scenarios:

  • Your applications require an operating system that is not provided as a public image.
  • You already have a set of basic images that you use to create virtual machines in another cloud platform.
  • The work required to migrate application code to one of the public images is greater than the work required to complete the boot disk image import process.

Alternatively, you can get assistance with migration for your virtual machines, from several partners. See Migrating VMs to Google Cloud Platform for details.

Overview

To import a boot disk image to Compute Engine, use the following process:

  1. Plan your import path. You must identify where you are going to prepare your boot disk image before you upload it, and how you are going to connect to that image after it boots in the Compute Engine environment.
  2. Prepare your boot disk so it can boot within the Compute Engine environment and so you can access it after it boots.
  3. Create and compress the boot disk image file.
  4. Upload the image file to Google Cloud Storage and import the image to Compute Engine as a new custom image.
  5. Use the imported image to create a virtual machine instance and make sure it boots properly.
  6. If the image does not successfully boot, you can troubleshoot the issue by attaching the boot disk image to another instance and reconfiguring it.
  7. Optimize the image and install the Linux Guest Environment so that it can your imported operating system image can communicate with the metadata server and use additional Compute Engine features.

Requirements

To import your boot disks to Compute Engine, your existing boot disks must meet the following requirements:

  • If you built a custom operating system kernel, it must meet the hardware and kernel configuration requirements. Most stock Linux distributions already meet these requirements, so this requirement is only for advanced users who build their own custom operating systems to run on Compute Engine.
  • The boot disk must be no larger than 2048GB (2TB).
  • The boot disk that you import must have a functional MBR partition table or a hybrid configuration of a GPT partition table with an MBR bootloader.
  • The primary partition on the boot disk can be in any format that you like as long as it boots properly from the MBR bootloader.
  • The bootloader on the boot disk must not have quiet, rhgb, or splashimage= kernel command line arguments. Compute Engine does not support splash screens on boot. You can remove these values from the GRUB config during the bootloader configuration step.

The image file that you import must meet the following requirements:

  • The disk image file name must be disk.raw.
  • The RAW image file must have a size in an increment of 1 GB. For example, the file must be either 10 GB or 11 GB but not 10.5 GB.
  • The compressed file must be a .tar.gz file that uses gzip compression and the GNU tar format.

You can import boot disk images for any operating system that you like, but some systems are easier to configure for the Compute Engine environment than others. The following operating systems are tested for this guide:

  • Debian 7 and Debian 8
  • RHEL/CentOS 6
  • openSUSE 13.x
  • Ubuntu 12.04, 14.04, and 16.04

The following operating systems might require additional configuration steps that are not covered in this guide:

  • Debian 6 and earlier
  • RHEL/CentOS 7
  • RHEL/CentOS 5 and earlier
  • Ubuntu 10.04 and earlier

Image import costs

Before you begin, understand the costs for the import process. There is no cost for the network ingress to upload your boot disk image file to Google Cloud Storage and there is no cost to import that image as a Compute Engine custom image. However, there are costs for some specific steps in the import process:

  • The cost to temporarily store your compressed image files in a Google Cloud Storage Standard bucket. You must use a temporary Google Cloud Storage bucket to hold your files before you can import them as custom Compute Engine images. You can remove the bucket after you complete the import process.
  • The cost for storing custom images after you finish importing them to Compute Engine.
  • The potential cost for data egress on your existing datacenter, network provider, or your current cloud service. Image files can be very large even after you compress them, so copying those files to Compute Engine could incur significant egress charges on some platforms.
  • The cost for Compute Engine persistent disks and virtual machine instances where you can configure your image after you import it to Compute Engine.

Plan and prepare your import path

Your method for importing your disk depends on the current configuration of the system that you want to move to Compute Engine. You need a system where you can create and compress the boot disk image file as well as a system where you can upload the image file to Google Cloud Storage. Consider the following items when you plan your import path:

  • The image import path requires you to configure your boot disk in a working operating system environment. This process can cause the boot disk to not boot anywhere outside of a Compute Engine environment. It is your responsibility to ensure that you do not lose data on your disks or disrupt your functional business applications while you import your system to Compute Engine.
  • Identify what your existing system access configuration is, and plan how you want to access the system after you import it to Compute Engine.
    • If your system has existing user login or SSH configurations, you can configure only the bootloader and then later configure the image to run optimally on Compute Engine. You will be able to access the instance through your existing SSH configuration or through a direct user login in the interactive serial console.
    • If your system does not have existing user login or SSH configurations, you must configure the boot disk so that you are able to access it after it boots on Compute Engine.
  • The length of the import process can take several hours or days depending on the size of your boot disk and the speed of your network connection.
  • The system where you create and compress your boot disk image must have enough storage space to create the image files on a storage device other than the boot disk itself. Typically your image and tar.gz files will use 2-3 times as much space as the boot disk itself.
  • Understand the file system structure for the existing system that you want to import.
    • If your operating system and application files are spread across multiple disks, import each of those disks individually and use each image to create a unique persistent disk for your Compute Engine virtual machine instance.
    • If your systems have boot volumes in a RAID configuration where multiple disks act as a single logical volume, create a single image from the entire array rather than creating one image for each disk in the array. Compute Engine persistent disks eliminate the need for RAID configurations.
  • If your system encrypts the contents of your boot disk with a Trusted Platform Module or with software-level encryption, decrypt your boot disk before you create the boot disk image file. Google cannot read your images if they are encrypted. Google encrypts your images for you at rest after you after you upload them, and allows you to provide your own encryption keys for your persistent disks, and Cloud Storage buckets.

After you identify or create a system where you can complete the import process, connect to that system and configure the bootloader.

Prepare the boot disk image

On a running system, prepare the boot disk image so that it can function in a Compute Engine environment.

  • Configure the bootloader on the boot disk so that the image can boot on Compute Engine.
  • Configure SSH or user login access on the boot disk so that you will be able to access it after you import it to Compute Engine and start it as a virtual machine instance.

This process can make the system unbootable outside of Compute Engine, so the best practice is to complete this step on an isolated system using a copy of the boot disk that you want to import.

Configure the bootloader

Configure the bootloader on the system so that it can boot on Compute Engine.

  1. Connect to the terminal on the system with the boot disk that you plan to import.

  2. Edit the GRUB config file. Usually this file is at /etc/default/grub, but on some older distributions it might be located in a non-standard directory.

  3. Make the following changes to the GRUB config file:

    • Remove any line that has splashimage=. Compute Engine does not support splash screens on boot.
    • Remove the rhgb and quiet kernel command line arguments.
    • Add console=ttyS0,38400n8d to the kernel command line arguments so that the instance can function with the Interactive Serial Console.
  4. Regenerate the grub.cfg file. Use one of the following commands depending on your distribution.

    • Debian and Ubuntu: sudo update-grub
    • RHEL, CentOS, SUSE, openSUSE: sudo grub-mkconfig -o /boot/grub/grub.cfg
  5. Edit the /etc/fstab file and remove references to all disks and partitions other than the boot disk itself and partitions on that boot disk. Invalid entries in /etc/fstab can cause your system startup process to halt.

After you configure the bootloader create and compress the disk image file.

Configure SSH or user login access on the image

After your image is running in Compute Engine as a virtual machine instance, you must have a way to access that instance. You can either connect to the instance using an existing SSH configuration or you can log in using a username and password by connecting to the Interactive Serial Console.

Complete the SSH or user login configuration before you create and compress the disk image file.

Create and compress the disk image file

Create and compress the boot disk image file for the system that you want to import to Compute Engine. The process to create and compress the image file is different depending on the platform where your systems operate.

Generic

On almost any system, you can use this process to create a RAW image file that you can import to Compute Engine. You can complete this process on the running system that you are importing or you can attach your boot disk as a secondary disk on another system and create the boot disk image from the stopped disk. Ensure that you have enough available storage space to temporarily hold the disk image files. This example takes an image from a running system.

  1. Connect to the terminal on the system that has the boot disk that you plan to import.

  2. Use the lsblk command to identify the source boot disk from which you want to create an image and the location where you have sufficient space to write the image files. For this example, /dev/sda is the source boot disk and /dev/sdb is a large secondary disk mounted at the /tmp directory. Although /dev/sda is running, you can still create an image from it. It is best to do this on a quiet system that is not actively processing data or running your applications.

    $ lsblk
    
    NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
    sda      8:0    0  100G  0 disk
    ├─sda1   8:1    0   96G  0 part /
    ├─sda2   8:2    0    1K  0 part
    └─sda3   8:5    0    4G  0 part [SWAP]
    sdb      8:16   0  500G  0 disk /tmp
    sr0     11:0    1 1024M  0 rom
    

  3. Create the image file from your boot disk.

    $ sudo dd if=/dev/sda of=/tmp/disk.raw bs=4M conv=sparse
    

  4. Change to the directory where you wrote the disk.raw file.

    $ cd /tmp
    

  5. Compress the raw disk into tar.gz format. This step compresses the image file so that you can more quickly upload it to Google Cloud Storage. On OSX, install gtar and use it for this step instead of tar.

    $ sudo tar -Sczf /tmp/compressed-image.tar.gz disk.raw
    

AWS EC2

If you are importing an Amazon EC2 instance, bundle up the Amazon EBS volume using the AMI tools:

  1. Find and note your Amazon account Id on the Account Settings console.

  2. On your Amazon EC2 instance, download and install the EC2 AMI tools.

  3. Run the ec2-bundle-vol as root with the following arguments, replacing cert-<hash>.pem with the certificate file, pk-<hash>.pem with the private key, and AWS_ACCOUNT_ID with your Amazon account ID. This command creates the image file.

    ec2-bundle-vol -c /tmp/build/cert-<hash>.pem \
        -k /tmp/build/pk-<hash>.pem -u AWS_ACCOUNT_ID \
        -r x86_64 --no-filter --exclude /tmp/build \
        --grub-config /tmp/build/menu.lst \
        --fstab /tmp/build/fstab
    
  4. Compress the raw disk into tar.gz format. This step compresses the image file so that you can more quickly upload it to Google Cloud Storage. On OSX, install gtar and use it for this step instead of tar.

    $ sudo tar -Sczf /tmp/compressed-image.tar.gz disk.raw
    

Virtual Box

If you prepared your system in a VirtualBox environment, you can use the VBoxManage tool to convert a .vdi or .qcow2 disk image to disk.raw format.

  1. Shut down the VirtualBox guest machine that you want to import. You can shut down the guest machine with the VirtualBox interface or the VBoxManage utility.

    VBoxManage controlvm [GUEST_NAME] acpipowerbutton
    

    where [GUEST_NAME] is the name of the guest machine.

  2. Convert the guest image to RAW format with the VBoxManage utility:

    VBoxManage clonehd [GUEST_IMAGE] \
        ~/disk.raw --format RAW
    

    where [GUEST_IMAGE] is the path to the guest image .vdi or .qcow2 file.

  3. Compress the raw disk into tar.gz format. This step compresses the image file so that you can more quickly upload it to Google Cloud Storage. On OSX, install gtar and use it for this step instead of tar.

    $ sudo tar -Sczf /tmp/compressed-image.tar.gz disk.raw
    

The image file is compressed and ready to upload to Google Cloud Storage.

Import the image to your custom images list

Upload the file to Google Cloud Storage and import the image into your custom images list. Optionally, you can encrypt the image during the image import step.

Import the image with either the console or the Cloud SDK tools:

Console

Copy the compressed-image.tar.gz file to your local workstation and use the Cloud Platform Console to create a bucket and upload the file.

  1. In the Cloud Platform Console, go to the Cloud Storage Browser page.

    Go to the Browser page

  2. At the top of the page, click Create bucket.
  3. Specify a unique bucket name, the Standard storage class, and a location where you want to store your image files.
  4. Click Create to create the bucket. The Browser page navigates to the new bucket.
  5. At the top of the page, click Upload files.
  6. In the file dialog, select the compressed-image.tar.gz file that you downloaded from your system. The file uploads from your local workstation. This step can take several hours depending on the size of your compressed image file and the speed of your network connection.

After you upload the image to Google Cloud Storage, import the image file to your custom images list.

  1. In the Cloud Platform Console, go to the Images page.

    Go to the Instances page

  2. At the top of the page, click Create image.
  3. In the Name field, specify a unique name for the image.
  4. Optionally, specify an image family for your new image, or configure specific encryption settings for the image.
  5. Click the Source menu and select Cloud Storage file.
  6. Enter the path to the compressed-image.tar.gz file that you uploaded to Cloud Storage.

    [BUCKET_NAME]/compressed-image.tar.gz
    
  7. Click the Create button to import the image. The process can take several minutes depending on the size of the boot disk image.

The image is now included on the Images page, but you must configure the bootloader before you can use the image to create a functional virtual machine instance.

gcloud and gsutil

Use the gsutil tool and the gcloud tool to upload the compressed boot disk image file. You can complete this process on the system where you created the boot disk image, or you can copy that file to another system and complete the upload process there instead.

  1. Install and initialize the Cloud SDK on the system from which you plan to upload the compressed-image.tar.gz.

  2. Use the gsutil tool to create a new Cloud Storage bucket.

     $ gsutil mb gs://[BUCKET_NAME]
     

  3. Upload the compressed-image.tar.gz file to the new bucket.

     $ gsutil cp compressed-image.tar.gz gs://[BUCKET_NAME]
     

  4. Import the image file as a new custom image.

     $ gcloud compute images create [IMAGE_NAME] --source-uri gs://[BUCKET_NAME]/compressed-image.tar.gz
     

The image is now included in the list of custom images, but you must configure the bootloader before you can use the image to create a functional virtual machine instance.

  $ gcloud compute images list --no-standard-images
  
  NAME                                            PROJECT                  FAMILY                    DEPRECATED  STATUS
  [IMAGE_NAME]                                    [PROJECT_ID]                                                   READY
  

Test the imported image to ensure it works

Confirm that the imported image works as expected. Create an instance with a boot disk that uses the imported image.

Console

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the Instances page

  2. Click the Create instance button.
  3. In the Boot disk section, click Change to begin configuring your boot disk.
  4. In the Your images tab, click the image that you imported.
  5. Click Select to confirm the boot disk configuration.
  6. Click the Create button to create the instance.

gcloud

gcloud compute instances create [INSTANCE_NAME] --zone [ZONE] --image [IMAGE_NAME]

where:

  • [INSTANCE_NAME] is a unique name for your instance.
  • [ZONE] is the zone where you created the standalone disk.
  • [IMAGE_NAME] is the name of the image that you imported.

After you create the instance, confirm that it booted properly. Check the serial port output:

Console

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the Instances page

  2. On the list of instances, click on the name of the instance that you created from the imported image. The instance details page opens.
  3. At the bottom of the instance details page, click View serial port to see the serial port output for this instance.

If the instance stopped at Booting from Hard Disk 0..., you must troubleshoot the issues from within the Compute Engine environment or you can reconfigure the boot disk on your original system and repeat the import process.

gcloud

gcloud compute instances get-serial-port-output [INSTANCE_NAME]

If the instance stopped at Booting from Hard Disk 0..., you must troubleshoot the issues from within the Compute Engine environment or you can reconfigure the boot disk on your original system and repeat the import process.

You can also test the instance by connecting to it. Connect to the instance through one of the following options:

  • SSH: If the instance had a functional SSH configuration, you can connect to the instance using SSH and your private key. You can find the instance IP address on the Instances page.
  • Interactive Serial Console: If you need to log in to the instance directly without SSH, you can enable the Interactive Serial Console and log in with a username and password.

Troubleshooting boot disk issues

If your instance does not boot and you are unable to connect to it or log in through the interactive serial console, identify the reason why the boot disk is not completing the boot and startup process.

After you identify where the boot and startup process is failing, you can correct the issue using one of two options:

Troubleshooting boot disks within Compute Engine

Mount your imported image as on a secondary disk that is attached to a temporary virtual machine instance. Use the Cloud Platform Console or the gcloud tool to create a standalone disk from the image that you uploaded and also create a temporary virtual machine with the standalone disk attached. You can use this instance to modify files on the standalone disk and fix issues that are causing that image to fail the boot process.

Console

Create a standalone disk from the boot disk image that you imported. Alternatively, you can detach a boot disk from and instance and create the instance using that detached boot disk instead.

  1. In the Cloud Platform Console, go to the Disks page.

    Go to the Disks page

  2. Click Create disk.
  3. On the Create a new disk page, specify the following attributes:
    • Zone: Select a zone near you. You must use this same zone when you create your temporary instance.
    • Source Type: Image
    • Source Image: Specify the name of the boot disk image that you imported.
  4. Click the Create button to create the disk.

Create a temporary instance where you can attach the standalone disk and configure the bootloader to function in a Cloud Platform Console environment.

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the Instances page

  2. Click the Create instance button.
  3. On the Create a new instance page, specify an instance name and a zone where the instance will be located. The zone must be the same zone where you created your standalone disk.
  4. Expand the Management, disk, networking, SSH keys section.
  5. Under the Disks tab in the Additional disks section, click Add item and select the standalone disk that you created. This attaches the standalone disk to the instance so you can mount it and modify the disk contents later.
  6. Click the Create button to create the instance.

gcloud

Create a standalone disk from the boot disk image that you imported. Alternatively, you can detach a boot disk from and instance and create the instance using that detached boot disk instead.

gcloud compute disks create [DISK_NAME] --zone [ZONE] --image [IMAGE_NAME]

where:

  • [DISK_NAME] is the name for the new standalone disk.
  • [ZONE] is a zone near you. You must use this same zone when you create the temporary instance.
  • [IMAGE_NAME] is the name of the boot disk image that you imported.

Create a temporary instance where you can attach the standalone disk and configure the bootloader to function in a Cloud Platform Console environment.

gcloud compute instances create [INSTANCE_NAME] --zone [ZONE] --disk name=[DISK_NAME]

where:

  • [INSTANCE_NAME] is a unique name for your instance.
  • [ZONE] is the zone where you created the standalone disk.
  • [DISK_NAME] is the name of the standalone disk that you created from the imported boot disk image.

After you create the instance with the attached standalone disk, you have a virtual environment where you can modify the bootloader from your original boot disk image.

Connect to the instance, mount the standalone disk, and configure the bootloader so that it boots properly on Compute Engine.

  1. Connect to the temporary instance using SSH from the browser or the gcloud compute ssh command.
  2. Use the blkid command to identify the disk that you want to modify and the partitions that you need to mount. In this example, /dev/sdb is the disk that you imported.

    $ lsblk
    
    NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
    sda      8:0    0   10G  0 disk
    └─sda1   8:1    0   10G  0 part /
    sdb      8:16   0  100G  0 disk
    ├─sdb1   8:17   0   96G  0 part
    ├─sdb2   8:18   0    1K  0 part
    └─sdb5   8:21   0    4G  0 part
    

  3. Mount the root partition from the standalone disk to the /tmp directory. In this example /dev/sdb1 is the root partition and the other partitions do not require any modifications. Your partition scheme might require you to mount multiple partitions in order to access all of the files that you need to change.

    $ sudo mount /dev/sdb1 /tmp
    

  4. Edit files that might cause the disk to fail the boot process. See the bootloader configuration instructions for details.

  5. Unmount the boot disk from the temporary instance.

    $ sudo umount /tmp
    

When you are done configuring this disk, detach it and use it as the boot disk for new instance.

Console

Detach the standalone disk from the temporary instance.

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the Instances page

  2. On the list of instances, click the name of the temporary instance where you modified the standalone boot disk. The instance details page opens.
  3. At the top of the instance details page, click Edit.
  4. Under the Additional disks section, click the X next to the standalone disk to indicate that you want to detach it from the temporary instance.
  5. At the bottom of the instance details page, click Save to save your changes.

Use the detached standalone disk to create a new instance.

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the Instances page

  2. Click the Create instance button.
  3. On the Create a new instance page, specify an instance name and a zone where the instance will be located. The zone must be the same zone where you created your standalone disk.
  4. In the Boot disk section, click Change to begin configuring your boot disk.
  5. In the Existing disks tab, choose the standalone boot disk to use it as the boot disk for this new instance.
  6. Click the Create button to create the instance.

gcloud

Detach the standalone disk from the temporary instance.

gcloud compute instances detach-disk [INSTANCE_NAME] --disk [DISK_NAME]

where:

  • [INSTANCE_NAME] is a unique name for your instance.
  • [DISK_NAME] is the name for the new standalone disk.

Use the detached standalone disk to create a new instance.

gcloud compute instances create [INSTANCE_NAME] --zone [ZONE] --disk name=[DISK_NAME],boot=yes

where:

  • [INSTANCE_NAME] is a unique name for your instance.
  • [ZONE] is the zone where the standalone disk is located.
  • [DISK_NAME] is the name of the standalone disk that you created from the imported boot disk image.

Test the instance that you created using the modified boot disk. If you are still unable to connect to the instance, read the serial console output again to identify where the boot process is failing. Repeat the troubleshooting process until you correct the issues with the boot disk image.

What's next

Send feedback about...

Compute Engine Documentation