Copying images from Container Registry

As a part of the transition from Container Registry to Artifact Registry, you can copy Container Registry images that you want to keep in Artifact Registry.

Overview

There are several options for copying images from Container Registry that you want to store and reuse in Artifact Registry. We recommend copying files with the gcrane tool since it supports copying sets of images with a single command.

To reduce costs, consider removing old, untagged images before copying containers to Artifact Registry.

Removing untagged images from Container Registry

Removing untagged images reduces the cost of storage in Container Registry as well as the cost to copy images from Container Registry to Artifact Registry.

A number of tools are available to identify and automate removal of images that you no longer need. For example, the gcr-cleaner tool helps you to find and remove old images based on different criteria. The gcr-cleaner tool is not an official Google product.

For more information about setting up and using the tool, see the gcr-cleaner documentation.

Copying images with gcrane

For copying images to Artifact Registry, we recommend that you use the gcrane tool.

gcrane offers several benefits, including:

  • Copy sets of images with a single command, including all images under a specified path or all images stored on multi-regional host in your project.
  • Skipping image layers that are already uploaded.
  • A fast copy option when copying images between Container Registry and Artifact Registry within the same Google Cloud multi-region.

    The fast copy option is also the least expensive copying method. Since this option does not rely on pulling and pushing images, there are no charges for network egress from Container Registry during the copy operation.

Requirements

Verify the following requirements:

  1. You have created a Docker repository in Artifact Registry for the images you are copying.

  2. You have the required permissions:

  3. To use the fast copy option, verify the following requirements:

    • You must use gcrane version 0.10.0 or higher. The gcrane setup instructions in this documentation include steps to download the latest version.

      To check the version of an existing gcrane installation, run the command:

      gcrane version
      
    • The Artifact Registry service account must have the Storage Object Viewer (roles/storage.objectViewer) role or a role with equivalent permissions. Run the following command to grant the Storage Object Viewer role to the service account:

      gcloud projects add-iam-policy-binding PROJECT \
          --member='serviceAccount:cloud-cicd-artifact-registry-copier@system.gserviceaccount.com' \
          --role='roles/storage.objectViewer'
      

      Replace PROJECT with ID of the Google Cloud project with your Container Registry hosts.

Environment options and costs

You can run gcrane from several environments:

  • Compute Engine instance - Use this option if you have a larger number of containers to copy.

    Costs:

    • Instance uptime for the Compute Engine VM.
    • Network egress for the images you copy, unless you are using the fast copy option. To minimize costs:
    • Use a VM instance in the same location as the Container Registry storage bucket.
    • To minimize cost and maximize copying speed, use an Artifact Registry repository in the same multi-region as Container Registry. Egress within the same multi-region does not incur charges for egress and is much faster than copying across locations.

      There is also no charge for egress from a multi-region to a region on the same continent until upcoming Cloud Storage pricing changes take effect.

  • Cloud Shell - An option for copying small sets of about 40GB or less. Since gcrane skips uploading image layers that are already uploaded, this limit is for new data that you are copying.

    Copying larger repositories might cause Cloud Shell to disconnect after the request timeout period of 10 minutes.

    Costs: Network egress for the images you copy, unless you are using the fast copy option.

    • You cannot chose the location of a Cloud Shell session. To check the location of the current session, run the command:
    curl metadata/computeMetadata/v1/instance/zone
    
    • To minimize cost and maximize copying speed, use an Artifact Registry repository in the same multi-region as Container Registry. Egress within the same multi-region does not incur charges for egress and is much faster than copying across locations.

      There is also no charge for egress from a multi-region to a region on the same continent until upcoming Cloud Storage pricing changes take effect.

  • Local machine - If you cannot use the other options, you can run gcrane from a local machine.

    Costs: Network egress for the images you copy, unless you are using the fast copy option

    To minimize cost and maximize copying speed, use an Artifact Registry repository in the same multi-region as Container Registry. Egress within the same multi-region does not incur charges for egress and is much faster than copying across locations.

    There is also no charge for egress from a multi-region to a region on the same continent until upcoming Cloud Storage pricing changes take effect.

Setting up Compute Engine

To copy images with gcrane from a Compute Engine VM instance:

  1. Create a VM instance. To minimize costs, create the instance in the same location as the registry you are copying from.
  2. By default, the VM instance is associated with the default service account and has permissions to pull images. You must change the access scope so that the VM instance can push images.

    1. Stop the VM instance. See Stopping an instance.

    2. Change the access scope with the command:

      gcloud compute instances set-service-account INSTANCE --scopes=storage-rw
      

      Replace INSTANCE with the name of the VM instance.

    3. Restart the VM instance. See Starting a stopped instance.

  3. Connect to the VM instance using SSH.

  4. Run the following command to download gcrane.

    curl -L \
    https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz \
    -o go-containerregistry.tar.gz
    
  5. Run the following commands to make the gcrane command executable.

    tar -zxvf go-containerregistry.tar.gz
    chmod +x gcrane
    sudo mv gcrane /usr/local/bin/
    
  6. Run the command gcrane --help to verify the installation.

You are now ready to copy images. To continue:

Setting up Cloud Shell

  1. Open a Cloud Shell window.

    Open Cloud Shell

  2. Set the default project. Replace PROJECT with the ID of project where Container Registry and Artifact Registry are installed

    gcloud config set project PROJECT.
    
  3. Run the following command to download the gcrane tool.

    curl -L \
    https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz \
    -o go-containerregistry.tar.gz
    
  4. Run the following commands to make the gcrane command executable.

    tar -zxvf go-containerregistry.tar.gz
    chmod +x gcrane
    sudo mv gcrane /usr/local/bin/
    
  5. Run the command gcrane --help to verify the installation.

You are now ready to copy images. To continue:

Setting up a local machine

  1. Download gcrane from the GitHub repository. For example, the following command downloads the Linux x86-64 distribution from the command line.

    curl -L \
    https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz \
    -o go-containerregistry.tar.gz
    
  2. Run the following commands to make the gcrane command executable. The commands assume that the downloaded file is named go-containerregistry.tar.gz.

    tar -zxvf go-containerregistry.tar.gz
    chmod +x gcrane
    sudo mv gcrane /usr/local/bin/
    
  3. Run the command gcrane --help to verify the installation.

You are now ready to copy images. To continue:

Identifying images to copy

After you have installed gcrane, you can list the existing images in Container Registry to find the ones you want to copy.

To list existing images, run the command:

gcrane ls LOCATION.gcr.io/PROJECT

To list tags an image has, run the command:

gcrane ls LOCATION.gcr.io/PROJECT/IMAGE

To list images recursively under a specific path, run the command:

gcrane ls -r LOCATION.gcr.io/PROJECT/PATH

For each of the commands:

  • Replace LOCATION with the multi-region of the registry: asia, eu, or us.
  • Replace PROJECT with the project ID.

See Copying images for the commands to copy your images.

Copying images

You can copy individual images from Container Registry, all images under a specified path in a location, or all images stored in a location.

Copying a single image

To copy a single tagged image, run the command:

gcrane cp GCR-LOCATION.gcr.io/PROJECT/IMAGE \
AR-LOCATION.pkg.dev/PROJECT/REPOSITORY/IMAGE

Where

  • GCR-LOCATION is the multi-region of the Container Registry host: asia, eu, or us.
  • AR-LOCATION is the region or multi-region of the repository.
  • PROJECT is the project ID.
  • REPOSITORY is the name of the Artifact Registry repository.
  • GCR-IMAGE is the image that you want to copy from Container Registry.
  • AR-IMAGE is the name for the image in Artifact Registry.

For example, consider the following source and destination image:

  • Container Registry image: eu.gcr.io/my-project/my-image:tag1
  • Artifact Registry image: europe-docker.pkg.dev/my-project/my-repo/my-image:tag1

Run the following command to copy the image:

gcrane cp eu.gcr.io/my-project/my-image:tag1 \
europe-docker.pkg.dev/my-project/my-repo/my-image:tag1

Copying all images under a path

To copy images recursively under a specific path in Container Registry, run the command:

gcrane cp -r GCR-LOCATION.gcr.io/PROJECT/GCR-PATH \
AR-LOCATION.pkg.dev/PROJECT/REPOSITORY/AR-PATH

Where

  • GCR-LOCATION is the multi-region of the Container Registry host: asia, eu, or us.
  • AR-LOCATION is the region or multi-region of the repository.
  • PROJECT is the project ID.
  • REPOSITORY is the name of the Artifact Registry repository.
  • GCR-PATH is the path of the files you want to copy.
  • AR-PATH is the path for the images in your Artifact Registry repository.

For example, consider the following source and destination paths:

  • Container Registry path: eu.gcr.io/my-project/test-images/testing
  • Artifact Registry path: europe-docker.pkg.dev/my-project/my-repo/test-images/testing

The following command recursively copies all images under test-images/testing to the my-repo repository:

gcrane cp -r eu.gcr.io/my-project/test-images/testing \
europe-docker.pkg.dev/my-project/my-repo/test-images/testing

Copying all images from a Container Registry location

To copy all images from a Container Registry multi-region, run the command:

gcrane cp -r GCR-LOCATION.gcr.io/PROJECT \
AR-LOCATION.pkg.dev/PROJECT/REPOSITORY

Where

  • GCR-LOCATION is the multi-region of the Container Registry host: asia, eu, or us.
  • AR-LOCATION is the region or multi-region of the repository.
  • PROJECT is the project ID.
  • REPOSITORY is the name of the Artifact Registry repository.

If you want to copy images for more than one Container Registry location in your project, run the command once for each host.

For example, consider the following source registry and target Artifact Registry repository. In this example, Container Registry and Artifact Registry are in different projects.

  • Container Registry host: eu.gcr.io/my-project
  • Artifact Registry image: europe-docker.pkg.dev/new-project/my-repo

The following command copies all images from the eu multi-region in the project my-project to the my-repo repository in the project new-project.

gcrane cp -r eu.gcr.io/my-project \
europe-docker.pkg.dev/new-project/my-repo

Copying images with Docker

Unlike the gcrane and gcloud copying methods, this approach is a two-step process.

  1. Pull an image from Container Registry
  2. Push the image to your Artifact Registry repository.

Requirements

Verify the following requirements:

  1. You have created a Docker repository in Artifact Registry for the images you are copying.

  2. You have the required permissions:

  3. You have configured Docker to authenticate to both Container Registry and Artifact Registry.

Costs

When you pull an image, you are billed for network egress. The specific pricing depends on the destination of your pull command. For example, if you pull an image from a Cloud Shell session, pricing is for network egress within Google Cloud. If you pull an image to a machine outside Google Cloud, pricing for general network egress.

The Docker commands that you use to tag, push, and pull images in Artifact Registry are similar to the ones you use in Container Registry. There are two key differences:

  • The hostname for Artifact Registry Docker repositories include a location prefix, followed by -docker.pkg.dev. Examples include australia-southeast1-docker.pkg.dev, europe-north1-docker.pkg.dev, and europe-docker.pkg.dev.
  • Since Artifact Registry supports multiple Docker repositories in a single project, you must specify the repository name in commands.

To copy an image to Artifact Registry with Docker, pull the image from Container Registry and then push it to your Artifact Registry repository.

For example, consider a Container Registry image in the eu multi-region and that you want to copy to an Artifact Registry repository in the europe region.

  • Container Registry image: eu.gcr.io/my-project/my-image:tag1
  • Artifact Registry image: europe-docker.pkg.dev/my-project/my-repo/my-image:tag1

The following command pulls the image from Container Registry.

docker pull eu.gcr.io/my-project/my-image:tag1

The following command pushes the image to the Artifact Registry repository named my-repo.

docker push europe-north1-docker.pkg.dev/my-project/my-repo/my-image

You can also push images to an Artifact Registry repository that you set up in a different project. This example pushes the same source image to the repository my-repo in the project new-project.

docker push europe-north1-docker.pkg.dev/new-project/my-repo/my-image

For details about pushing, and pulling images in Artifact Registry, see Pushing and pulling images.

Copying images with gcloud

Use the gcloud container images add-tag command to copy an image from Container Registry to your Artifact Registry repository.

Requirements

Verify the following requirements:

  1. You have created a Docker repository in Artifact Registry for the images you are copying.

  2. You have the required permissions:

Costs

If Container Registry is in a different location than your Artifact Registry repository, network egress charges apply for images you copy. If both services are in the same location, network egress is free.

Run the following command to copy an image from Container Registry to Artifact Registry:

gcloud container images add-tag GCR-IMAGE AR-IMAGE

Where

  • GCR-IMAGE is the full path to the Container Registry image.
  • AR-IMAGE is the full path for the image in the Artifact Registry repository.

For example, consider the following source and destination images:

  • Container Registry image: eu.gcr.io/my-project/my-image:tag1
  • Artifact Registry image: europe-docker.pkg.dev/my-project/my-repo/my-image:tag1

This command copies the image from Container Registry in the multi-region eu to the repository my-repo in the multi-region europe.

gcloud container images add-tag eu.gcr.io/my-project/my-image:tag1 \
europe-docker.pkg.dev/my-project/my-repo/my-image:tag1

You can also copy images to an Artifact Registry repository that you set up in a different project. This example copies the same source image to the repository my-repo in the project new-project.

gcloud container images add-tag eu.gcr.io/my-project/my-image:tag1 \
europe-docker.pkg.dev/new-project/my-repo/my-image:tag1

What's next