If you need to move your Compute Engine boot disk data
outside of your Compute Engine project, you can export a boot disk
image to Cloud Storage as a tar.gz
file. If you need to
create a persistent disk image to use when you create new persistent disks on
Compute Engine, read
Creating a custom image.
You can backup or share a custom image by exporting the image to Cloud Storage. This method is ideal for sharing individual images with projects that do not have access to your images. Alternatively, you can share images by granting the Compute Engine image user role on the image or on the project that contains it.
The following diagram shows some typical workflows for the creation and reuse of a custom image.
Before you begin
- If you want to use the command-line examples in this guide:
- Install or update to the latest version of the gcloud command-line tool.
- Set a default region and zone.
- If you want to use the API examples in this guide, set up API access.
- Read the images page.
- If the project that you want to export the image from has a trusted image
policy defined, add
projects/compute-image-tools
to the allowed list of publishers.
Enable the Cloud Build API
The virtual appliance export tool uses Cloud Build.
In most cases, gcloud compute images export
attempts to grant these
permissions to the Cloud Build service account. However, you can
manually grant these permissions to ensure that the required permissions
are in effect.
Console
In the Google Cloud Console, enable the Cloud Build API.
When you enable the Cloud Build API from the console, Compute Engine grants the Cloud Build service account the following roles so that the Cloud Build service can export instances from Compute Engine:
roles/iam.serviceAccountTokenCreator
roles/compute.admin
roles/iam.serviceAccountUser
The export tool also uses the default Compute Engine service account. By default, the Compute Engine service account has the IAM project editor role. If this role is removed, the export process might fail. To add the role back to the service account, see Granting access. For more information about the Compute Engine default service account, see Compute Engine default service account.
gcloud
To set up the Cloud Build service using gcloud
command-line tool, complete the
following steps:
Using the
gcloud
command-line tool, enable Cloud Build.gcloud services enable cloudbuild.googleapis.com
The export tool also uses the default Compute Engine service account. By default, the Compute Engine service account has the IAM project editor role. If this role is removed, the export process might fail. To add the role back to the service account, see Granting access. For more information about the Compute Engine default service account, see Compute Engine default service account.
Add the
compute.admin
role to the service account for the Cloud Build API.gcloud projects add-iam-policy-binding project-id \ --member serviceAccount:project-num@cloudbuild.gserviceaccount.com \ --role roles/compute.admin
Add the
iam.serviceAccountUser
role to the service account for the Cloud Build API.gcloud projects add-iam-policy-binding project-id \ --member serviceAccount:project-num@cloudbuild.gserviceaccount.com \ --role roles/iam.serviceAccountUser
Add the
iam.serviceAccountTokenCreator
role to the service account for the Cloud Build API.gcloud projects add-iam-policy-binding project-id \ --member serviceAccount:project-num@cloudbuild.gserviceaccount.com \ --role roles/iam.serviceAccountTokenCreator
Replace the following:
project-id
: The projectID for your project.project-num
: The project number for your project.
Exporting an image with a single command
You can export your virtual disks using either the
Google Cloud Console, the
gcloud
command-line tool, or the
Cloud Build API.
Console
In the Google Cloud Console, go to the Images page.
Click the name of the image that you want to export to go to the image details page. You can't export public images provided by Google. You can only export images that you previously created or imported.
From the image details page, click Export to open the Export Image page.
From the Export image page, choose the Export format of the image.
Choose the Cloud Storage location to export your image to by clicking Browse.
Choose an existing Cloud Storage location to export your image. Or, follow the directions to create a new Cloud Storage bucket, and then enter a name for the new Cloud Storage bucket.
Once you choose a Cloud Storage, choose a filename for the exported image. You can use the default filename, or you can choose your own filename.
After choosing a Cloud Storage, and entering a filename for the image, click Select.
From the Export image page, click Export. After choosing Export, the Cloud Console displays the Image export history, where you can view the image export process. For additional details about the image export process, click the Cloud Build ID to go to the Image export details page where you can view and download the image export log.
Go to the Storage page to access your exported image.
gcloud
The preferred way to export an image to Cloud Storage is to
use the
gcloud compute images export
command. This command uses
Daisy
to chain together the multiple steps that are required to export an image.
It assumes that you have already
created an image,
for example, with the
gcloud compute images create
command.
Using the gcloud
command-line tool, run:
gcloud compute images export \ --destination-uri destination-uri \ --image image-name
Replace the following:
destination-uri
: The Cloud Storage URI destination for the exported virtual disk file.image-name
: The name of the disk image to export.
For example, the following command exports an image named my-image
from
my-project
to a Cloud Storage bucket named my-bucket
. By
default, the image is exported as a disk.raw file and is compressed into
the tar.gz
file format.
gcloud compute images export \ --destination-uri gs://my-bucket/my-image.tar.gz \ --image my-image \ --project my-project
For a list of all available flags see the
gcloud compute images export
reference documentation.
API
In the API, create a POST
request to the Cloud Build API.
POST https://cloudbuild.googleapis.com/v1/projects/project-id/builds { "timeout": "7200s", "steps":[ { "args":[ "-timeout=7000s", "-source_image=source-image", "-client_id=api", "-format=image-format", "-destination_uri=destination-uri" ], "name":"gcr.io/compute-image-tools/gce_vm_image_export:release", "env":[ "BUILD_ID=$BUILD_ID" ] } ], "tags":[ "gce-daisy", "gce-daisy-image-export" ] }
Replace the following:
project-id
: The project ID for the project that contains the image that you want to export.source-image
: Name of the image to be exported.image-format
: Format of the exported image. Valid formats includevmdk
,vhdx
,vpc
,vdi
, andqcow2
.destination-uri
: The Cloud Storage URI location that you want to export the virtual disk file to. For example,gs://my-bucket/my-exported-image.vmdk
.
For additional args
values that can be provided, see the optional
flags section of the
VM image export GitHub page.
Example response
The following sample response resembles the output that is returned:
{ "name": "operations/build/myproject-12345/operation-1578608233418", "metadata": { "@type": "type.googleapis.com/google.devtools.cloudbuild.v1.BuildOperationMetadata", "build": { "id": "3a2055bc-ccbd-4101-9434-d376b88b8940", "status": "QUEUED", "createTime": "2019-10-02T18:59:13.393492020Z", "steps": [ { "name": "gcr.io/compute-image-tools/gce_vm_image_export:release", "env": [ "BUILD_ID=3a2055bc-ccbd-4101-9434-d376b88b8940" ], "args": [ "-timeout=7056s", "-source_image=my-image", "-client_id=api", "-format=vmdk", "-destination_uri=gs://my-bucket/my-exported-image.vmdk" ] } ], "timeout": "7200s", "projectId": "myproject-12345", "logsBucket": "gs://123456.cloudbuild-logs.googleusercontent.com", "options": { "logging": "LEGACY" }, "logUrl": "https://console.cloud.google.com/gcr/builds/3a2055bc-ccbd-4101-9434-d376b88b8940?project=123456" } }
There are a couple ways you can monitor your build:
- Run a
projects.builds.get
request using the returned
build-id
. - Review the logs hosted at the provided
logUrl
.
Creating and exporting an image manually
If the gcloud compute images create
and gcloud compute images export
commands do not meet your requirements, you can create and export an image
manually from a Compute Engine instance. This process has discrete
steps to first create an image and then export an image.
In the following example, note the created disk is called image-disk.
To create and export an image:
Optionally, stop the instance that the disk is attached to before you create the snapshot. Stopping the instance ensures the integrity of the disk contents in the snapshot. Replace
disk-name
with the name of the disk that you want to use to create the snapshot.Create a snapshot of the disk. Name the snapshot
image-snapshot
.gcloud compute disks snapshot disk-name \ --snapshot-names image-snapshot
Use the
image-snapshot
snapshot to create a new disk namedimage-disk
by running the following command:gcloud compute disks create image-disk \ --source-snapshot image-snapshot
Create a temporary disk named
temporary-disk
to hold your tar file, and specify thesize
of the disk to be at least 50% larger than the image disk.You can detach and delete the disk afterwards.
gcloud compute disks create temporary-disk \ --size size
where
size
is the size, in gigabytes or terabytes, of the temporary disk. For example, specify100GB
to create a 100-gigabyte disk.Create an instance and enable
storage-rw
scope on the instance. Also, attach theimage-disk
and thetemporary-disk
to the instance as secondary disks with specificdevice-name
attributes. Replaceinstance-name
with the name of the instance to create.gcloud compute instances create instance-name \ --scopes storage-rw \ --disk name=image-disk,device-name=image-disk \ --disk name=temporary-disk,device-name=temporary-disk
Note that you're passing in service account scopes so that you can upload your file to Cloud Storage in later steps.
Review the details about starting a new instance if necessary.
Connect to your instance. Replace
instance-name
with the name of the instance to connect to.gcloud compute ssh instance-name
Format and mount the temporary disk. Formatting the disk deletes the contents of the temporary disk.
sudo mkdir /mnt/tmp
sudo mkfs.ext4 -F /dev/disk/by-id/google-temporary-disk
sudo mount -o discard,defaults /dev/disk/by-id/google-temporary-disk /mnt/tmp
Optionally, you can mount the image disk and make additional changes before you create the tar file. For example, you might want to delete any existing files from the
/home
directory if you do not want them to be part of your image. Mount the disk partitions that you need to modify, modify the files on the disk that you need to change, and then unmount the disk when you are done.Create a directory where you can mount your disk or partition.
sudo mkdir /mnt/image-disk
Use the
ls
command to determine which disk or disk partition you need to mount.ls /dev/disk/by-id/
The command prints a list of disk IDs and partitions. For example, the following disk has a partition table with one partition. The
google-image-disk
ID points to the full disk from which you want to create an image. Thegoogle-image-disk-part1
ID points to the first partition on this disk. Mount the partition if you need to make changes to the disk, then create the image from the full disk.google-image-disk google-image-disk-part1
Mount the disk or the partition. If your disk has a partition table, mount the individual partitions for your disk. For example, mount
google-image-disk-part1
.sudo mount /dev/disk/by-id/google-image-disk-part1 /mnt/image-disk
Alternatively, if your disk is raw formatted with no partition table, mount the full
google-image-disk
disk.sudo mount /dev/disk/by-id/google-image-disk /mnt/image-disk
Modify the files in the
/mnt/image-disk
directory to configure the files on the disk. As an example, you might remove the/mnt/image-disk/home/[USER]/.ssh/authorized_keys
file to protect your SSH keys from being shared.After you have finished modifying files, unmount the disk.
sudo umount /mnt/image-disk/
Create a tar file of your image.
When you finish customizing the files on the image disk, create a raw disk file on your temporary disk. The name of the raw disk image must be 'disk.raw':
sudo dd if=/dev/disk/by-id/google-image-disk of=/mnt/tmp/disk.raw bs=4096
Then tar and gzip this file:
cd /mnt/tmp
sudo tar czvf myimage.tar.gz disk.raw
This command creates an image of the instance in the following location:
/mnt/tmp/myimage.tar.gz
Upload the image into Cloud Storage.
To upload the tar file to Cloud Storage, use the gsutil command line tool that comes preinstalled on your instance.
Create a bucket using
gsutil
.Make sure to review the bucket and object naming guidelines before you create your bucket. Then, create your bucket using the following command. Replace
bucket-name
with the name of the bucket to create.me@example-instance:~$ gsutil mb gs://bucket-name
Copy your file to your new bucket. Replace
bucket-name
with the name of the bucket to copy the file to.me@example-instance:~$ gsutil cp /mnt/tmp/myimage.tar.gz gs://bucket-name
You have exported your file into Cloud Storage. You can now share the image with other people, or use the tar file to add a new image to a Google Cloud Console project.
What's next
- Share images using the image user role.
- Learn about the import methods available for Compute Engine.