Scan Python packages automatically

This document explains how to enable the Container Scanning API, push an image to Artifact Registry, and see the list of vulnerabilities found in the image.

Artifact Analysis stores vulnerability information as notes. An occurrence is created for each instance of a note associated with an image. See the overview and pricing documents for more information.

Note: Artifact Analysis only supports automatic scanning of Python packages in Docker containers uploaded to Artifact Registry. Container Registry is not supported.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Artifact Registry and Container Scanning APIs.

   Enable the APIs

5. Install the Google Cloud CLI.

6. To initialize the gcloud CLI, run the following command:

   gcloud init

7. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

8. Make sure that billing is enabled for your Google Cloud project.

9. Enable the Artifact Registry and Container Scanning APIs.

   Enable the APIs

10. Install the Google Cloud CLI.

11. To initialize the gcloud CLI, run the following command:

    gcloud init

12. Create a Docker repository in Artifact Registry and push a container image with your Python code to the repository. If you are not familiar with Artifact Registry, see the Docker quickstart.
    
    For information on how you can use Cloud Build to build and containerize your Python applications, see Build, test and containerize Python applications.
    

Important: When you enable the Container Scanning API, billing begins immediately. Once you enable the API for a project, Artifact Analysis automatically scans each newly pushed image to Artifact Registry in that project.

Artifact Analysis does not automatically scan existing images. To scan an existing image, you must push it again.

View the image vulnerabilities

Artifact Analysis scans new images when they're uploaded to Artifact Registry. This scan extracts information about the packages in the container.

You can view vulnerability occurrences for your images in Artifact Registry using the Google Cloud console, Google Cloud CLI, or the Container Analysis API. If an image has vulnerabilities, you can then obtain the details.

Artifact Analysis only updates the metadata for images that were pushed or pulled in the last 30 days. Artifact Analysis archives metadata that is older than 30 days.To re-scan an image with archived metadata, pull that image. Refreshing metadata can take up to 24 hours.

View occurrences in the Google Cloud console

To see the vulnerabilities in an image:

1. Get the list of repositories.

   Open the Repositories page

2. In the repositories list, click a repository.

3. In the images list, click an image name.

   Vulnerability totals for each image digest are displayed in the Vulnerabilities column.

   

4. To view the list of vulnerabilities for an image, click the link in the Vulnerabilities column.

   The Scan results section displays a summary of the package types scanned, total vulnerabilities, vulnerabilities with fixes available, vulnerabilities without fixes, and effective severity.

   

   The table of vulnerabilities lists the Common Vulnerabilities and Exposures (CVE) name for each vulnerability found, the effective severity, Common Vulnerability Scoring System (CVSS) score, fixes (when available), the name of the package that contains the vulnerability, and the package type.

   You can filter and sort these files to check a specific file, directory, or type of file by file extension.

   Google Cloud console displays up to 1200 vulnerabilities in this table. If your image has more than 1200 vulnerabilities, you must use gcloud or the API to view the full list.

5. For details about a specific CVE, click the CVE name.

6. To view vulnerability occurrence details such as version number and affected location, click View or View Fixed in the row with the name of the vulnerability. The link text is View for vulnerabilities without a fix, and View Fixed for vulnerabilities where a fix has been applied.

View occurrences using gcloud

To view occurrences for an image in Artifact Registry, run the following command:

  gcloud artifacts docker images list --show-occurrences \
      LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID

Where:

• LOCATION is the regional or multi-regional location of the repository.
• PROJECT_ID is your Google Cloud console project ID.
• REPOSITORY is the name of the repository where the image is stored.

• IMAGE_ID is the name of the image in the repository. You cannot specify an image tag with this command.

  By default, the command returns the 10 most recent images. To show a different number of images, use the --show-occurrences-from flag. For example, the following command returns the 25 most recent images.

  gcloud artifacts docker images list --show-occurrences-from=25 \
      us-central1-docker.pkg.dev/my-project/my-repo/my-image
  

To view vulnerabilities for an image tag or a layer:

  gcloud artifacts docker images describe \
      LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID:TAG \
      --show-package-vulnerability

or

  gcloud artifacts docker images describe \
      LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID@sha256:HASH \
      --show-package-vulnerability

Where:

• LOCATION is the regional or multi-regional location of the repository.
• PROJECT_ID is your Google Cloud console project ID.
• REPOSITORY is the name of the repository where the image is stored.
• IMAGE_ID is the name of the image in the repository.
• TAG is the image tag about which you want to get information.

• HASH is the image digest.

  Artifact Analysis returns results including the packageType.

To filter the vulnerability occurrences:

  gcloud artifacts docker images list --show-occurrences \
      LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID \
      --occurrence-filter=FILTER_EXPRESSION

Where:

• LOCATION is the regional or multi-regional location of the repository.
• PROJECT_ID is your Google Cloud console project ID.
• REPOSITORY is the name of the repository where the image is stored.
• IMAGE_ID is the name of the image in the repository.
• FILTER_EXPRESSION is a sample filter expression in the format explained in Filtering vulnerability occurrences.

View occurrences using the API

To get a list of occurrences in your project:

 curl -X GET -H "Content-Type: application/json" -H \
    "Authorization: Bearer $(gcloud auth print-access-token)" \
    https://containeranalysis.googleapis.com/v1/projects/PROJECT_ID/occurrences

To get a summary of vulnerabilities in your project:

 curl -X GET -H "Content-Type: application/json" -H \
    "Authorization: Bearer $(gcloud auth print-access-token)" \
    https://containeranalysis.googleapis.com/v1/projects/PROJECT_ID/occurrences:vulnerabilitySummary

To get details on a specific occurrence:

 curl -X GET -H "Content-Type: application/json" -H \
    "Authorization: Bearer $(gcloud auth print-access-token)" \
    https://containeranalysis.googleapis.com/v1/projects/PROJECT_ID/occurrences/OCCURRENCE_ID

View vulnerabilities in Cloud Build

If you're using Cloud Build, you can also view image vulnerabilities in the Security insights side panel within the Google Cloud console.

The Security insights side panel provides a high-level overview of build security information for artifacts stored in Artifact Registry. To learn more about the side panel and how you can use Cloud Build to help protect your software supply chain, see View build security insights.

Filter occurrences

You can use filter strings in the gcloud commands and the Artifact Analysis API to filter occurrences before viewing them. The following sections describe the supported search filters.

View occurrences of a specific type

You can use the kind value to filter by occurrence type. See available kinds.

The following examples demonstrate how to filter to deployment occurrences for an image:

gcloud

In the gcloud CLI, run the following:

gcloud artifacts docker images list --show-occurrences \
    --occurrence-filter='kind="DEPLOYMENT"' --format=json \
    LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID

Where:

• DEPLOYMENT is the kind of occurrence.
• LOCATION is the regional or multi-regional location of the repository.
• PROJECT_ID is your Google Cloud console project ID.
• REPOSITORY is the name of the repository where the image is stored.
• IMAGE_ID is the name of the image in the repository.

API

In your API query, use the following filter expression:

kind="DEPLOYMENT" AND resourceUrl="RESOURCE_URL"

Where:

• DEPLOYMENT is the kind of occurrence.
• RESOURCE_URL is the complete URL of the image in the format https://LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID@sha256:HASH.

You can use the hasPrefix function to filter with a wider scope.

For example, the following filters to occurrences of a specific kind across many images:

kind="NOTE_KIND" AND has_prefix(resourceUrl, "RESOURCE_URL_PREFIX")

Where:

• RESOURCE_URL_PREFIX specifies a substring from a resource URL.
  • To filter for all versions of an image, omit the image digest. Use the format: https://LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID@
  • To filter for all images in a project, only specify the resource location and project. Use the format: https://LOCATION-docker.pkg.dev/PROJECT_ID/

View vulnerability occurrences

To retrieve a list of vulnerability occurrences for an image, you can filter by the VULNERABILITY occurrence type.

gcloud

In the gcloud CLI, run the following:

gcloud artifacts docker images list \
    --show-occurrences \
    --occurrence-filter='kind="VULNERABILITY"' \
    --format=json \
    LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID

Where:

• VULNERABILITY is the kind of occurrence.
• LOCATION is the regional or multi-regional location of the repository.
• PROJECT_ID is your Google Cloud console project ID.
• REPOSITORY is the name of the repository where the image is stored.
• IMAGE_ID is the name of the image in the repository.

API

In your API query, use the following filter expression:

GET https://containeranalysis.googleapis.com/v1/projects/PROJECT_ID/occurrences?filter=kind%3D%22VULNERABILITY%22%20AND%20resourceUrl%3D%22ENCODED_RESOURCE_URL%22

Where:

• ENCODED_RESOURCE_URL is the encoded path to your image. To learn about encoding, see URL encoding.

For more information on using the API, see projects.occurrences.get.

Filter by package type

To limit vulnerability scanning results to one package type, run the following gcloud CLI command:

  gcloud artifacts docker images list /
      --show-occurrences /
      --occurrence-filter='kind="VULNERABILITY" AND packageType="PACKAGE_TYPE"' /
      LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID

Where:

• VULNERABILITY is the kind of occurrence.
• PACKAGE_TYPE is the application language package type. Available types are: COMPOSER, GO, MAVEN, NPM, NUGET, PYTHON, RUBYGEMS, and RUST.
• LOCATION is the regional or multi-regional location of the repository.
• PROJECT_ID is your Google Cloud console project ID.
• REPOSITORY is the name of the repository where the image is stored.
• IMAGE_ID is the name of the image in the repository. You cannot specify an image tag with this command.

View images associated with a specific note

You can retrieve a list of resources that are associated with a specific note ID. For example, you can list images with a specific CVE vulnerability.

To list all images within a project that are associated with a particular note, use the following filter expression:

gcloud

gcloud artifacts docker images list /
    --show-occurrences /
    --occurrence-filter='noteProjectId="goog-vulnz" AND noteId="NOTE_ID"' /
    LOCATION-docker.pkg.dev/PROJECT_ID

Where:

• goog-vulnz is the PROVIDER_PROJECT_ID for Artifact Analysis vulnerability analysis. If you are using Artifact Analysis in a custom project, you can replace this value with your own provider project ID.
• LOCATION is the regional or multi-regional location of the repository.
• PROJECT_ID is your Google Cloud console project ID.
• NOTE_ID is the ID of the note. For example, when you see a vulnerability in Artifact Analysis scanning results, it will often use CVE ID formatting similar to CVE-2019-12345.

API

GET https://containeranalysis.googleapis.com/v1/projects/PROJECT_ID/occurrences?filter=noteProjectId%3D%22goog-vulnz%22%20AND%20resourceUrl%3D%22ENCODED_RESOURCE_URL%22%20AND%20noteId%3D%22NOTE_ID%22

Where:

• goog-vulnz is the PROVIDER_PROJECT_ID for Artifact Analysis vulnerability analysis. If you are using Artifact Analysis in a custom project, you can replace this value with your own provider project ID.
• ENCODED_RESOURCE_URL is the encoded path to your image. To learn about encoding, see URL encoding.
• NOTE_ID is the ID of the note. For example, when you see a vulnerability in Artifact Analysis scanning results, it will often use CVE ID formatting similar to CVE-2019-12345.

To check a specific image for a specific note, use the following filter expression:

gcloud

gcloud artifacts docker images list /
    --show-occurrences /
    --occurrence-filter='noteProjectId="goog-vulnz" AND noteId="NOTE_ID"' /
    LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID

Where

• LOCATION is the regional or multi-regional location of the repository.
• PROJECT_ID is your Google Cloud console project ID.
• REPOSITORY is the name of the repository where the image is stored.
• IMAGE_ID is the name of the image in the repository. You cannot specify an image tag with this command.

API

In your API query, add the following filter expression:

resourceUrl="RESOURCE_URL" AND noteProjectId="goog-vulnz" \
AND noteId="NOTE_ID"

Where:

• RESOURCE_URL is the complete URL of the image in the format https://LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY/IMAGE_ID@sha256:HASH.
• goog-vulnz is the PROVIDER_PROJECT_ID for Artifact Analysis vulnerability analysis. If you are using Artifact Analysis in a custom project, you can replace this value with your own provider project ID.
• NOTE_ID is the ID of the note. Security related notes are often formatted as CVE-2019-12345.

What's next

• Use Pub/Sub notifications to get notifications about vulnerabilities and other metadata.

• Kritis Signer and Voucher allow you create Binary Authorization attestations as part of your build pipeline. These tools can create Binary Authorization attestations based on vulnerability scanning results. For more information, see Creating attestations with Kritis Signer or Creating attestations with Voucher.