Access security metadata using API

Stay organized with collections Save and categorize content based on your preferences.

Assured OSS provides security metadata for every package available on their Artifact Registry repository. Each version of the package has its own metadata. This page explains what information is provided as part of the metadata and how you can access this metadata.

The security metadata is stored in Google Cloud Container Analysis which provides metadata storage as per the open source Grafeas model . The following information is provided for each version of the package:

  1. A PackageNote for each version released of the package by Assured OSS. The package note contains:

    1. Distributions where each distribution corresponds to one artifact being provided by Assured OSS. It includes the location and digest URL(from where the signature and hash can be downloaded) for binary and reproducible sources.
    2. Related URL that contains links to other pieces or metadata like build provenance, test attestation, vulnerabilities.
  2. A BuildOccurrence. The link to the build occurrence is given in the package note's related URL section against the label BUILD_OCCURRENCE. The intotoStatement in the Build Occurrence contains the SLSA v0.2 provenance.

  3. An AttestationOccurrence. The link to the attestation occurrence is given in the package note's related URL section against the label _TESTOCCURRENCE. The serialized payload in the attestation contains JSON representing all security testing done on the package-version.

  4. A second BuildOccurrenance. The link to this build occurrence is give in the package note's related URL section against the label ORIGINAL_SOURCE_PROVENANCE. This file contains the original source (GitHub) details in the form of Source.

  5. A VulnerabilityNote and VulnerabilityOccurrence for every vulnerability associated with the package. The note and occurrence combined provide information about vulnerabilities such as the vulnerability summary or description, severity, CVSSV3 score, affected and fixed version information, remediation and so on.

Access metadata using API

You can access the security metadata using the Google Cloud Container Analysis API. You can use the Container Analysis API clients which are available in different languages. Install the required Container Analysis client libraries.

Set up Authentication

Authenticate to the API using the same service account that you use to access Assured OSS. To set up authentication for programmatic access, Application Default Credentials (ADC) must be set up. Provide authentication credentials to the application code by setting the environment variable GOOGLE_APPLICATION_CREDENTIALS using the following command:

export GOOGLE_APPLICATION_CREDENTIALS=KEY_FILE_LOCATION

Where KEY_FILE_LOCATION is the path to the service account json key file.

Fetch Security Metadata

To fetch security metadata, you require a package note_name which is constructed using the following information:

  • Project_ID - The alpha-numeric identifier for your Google Cloud project
  • Package Language - JAVA or PYTHON
  • Package_ID - For Java, it's groupId_artifactId and for Python it's packageName
  • Version - Version of the package

The note_name should have the following structure: projects/<project_id>/notes/<language>-<package_id>-<version>.

For example: projects/cloud-aoss/notes/JAVA-org.apache.commons_commons-compress-1.21

Option 1:

If you're using the Container Analysis API, use getNotes with the note_name to query the PackageNote. The PackageNote has links to other notes and occurrences.

Option 2:

If you're using the CURL command, use the command - curl -H "Content-Type: application/json" -H "Authorization: Bearer $(gcloud auth print-access-token)" $URL.

  • For Java, use URL: https://containeranalysis.googleapis.com/v1/projects/cloud-aoss/notes/JAVA-org.jsoup_jsoup-1.15.2
  • For Python, use URL: https://containeranalysis.googleapis.com/v1/projects/cloud-aoss/notes/PYTHON-flask-cors-3.0.10

Sample Metadata

{
  "name": "projects/cloud-aoss/notes/JAVA-org.jsoup_jsoup-1.15.2",
  "kind": "PACKAGE",
  "relatedUrl": [
    {
      "url": "projects/cloud-aoss/occurrences/621d7e7b-8e6c-4313-b24b-eae3a1d1b884",
      "label": "ORIGINAL_SOURCE_PROVENANCE"
    },
    {
      "url": "https://containeranalysis.googleapis.com/v1/projects/cloud-aoss/occurrences/a0967d97-634c-4066-b6da-0a723273a165",
      "label": "BUILD_OCCURRENCE"
    },
    {
      "url": "projects/cloud-aoss/occurrences/6005db73-7310-44f9-9a36-6ca0174732a1",
      "label": "TEST_OCCURRENCE"
    },
    {
      "url": "projects/cloud-aoss/notes/GHSA-gp7f-rwcx-9369",
      "label": "VULNERABILITY_GHSA-gp7f-rwcx-9369"
    }
  ],
  "createTime": "2022-10-12T06:48:19.162016Z",
  "updateTime": "2022-10-18T05:31:15.081736Z",
  "package": {
    "distribution": [
      {
        "cpeUri": "cpe:2.3:a:JAVA::org.jsoup:jsoup:1.15.2:*:*:*:*:*:*:*",
        "url": "https://us-maven.pkg.dev/…/org/jsoup/jsoup/1.15.2/jsoup-1.15.2.jar",
        "description": "..."
      },
      {
        "cpeUri": "cpe:2.3:a:JAVA::org.jsoup:jsoup:1.15.2:*:*:*:*:*:*:*",
        "url": "https://us-maven.pkg.dev/…/org/jsoup/jsoup/1.15.2/jsoup-1.15.2-sources.jar",
        "description": "..."
      }
]}}

Use the note and occurrence names provided in relatedURLs map to fetch additional data related to source, build, test, and vulnerability. To do this, use the GetNote and GetOccurrence APIs or curl commands. For BuildOccurrence, remove the initial prefix of https://containeranalysis.googleapis.com/v1/ when you use the API.

Sample Python script

A sample Python script that prints the entire downloaded metadata is available at the following Cloud Storage location along with its README file:

gs://cloud-aoss/utils/python-download-metadata/v1.0

Follow these steps to download the script using the gsutil command line tool:

  1. Authenticate with the service account to access the Cloud Storage bucket using the following command:
gcloud auth activate-service-account --key-file KEY-FILE

Where KEY-FILE is the path to the file containing the service account credentials.

  1. Download the download_metadata.py to your machine using the following command:
gsutil cp -r gs://cloud-aoss/utils/python-download-metadata/v1.0/download_metadata.py
PATH_TO_LOCAL_STORE

Where PATH_TO_LOCAL_STORE is the local path where you want to save the file.

  1. Download the README.md and instructions to use the script using the following command:
gsutil cp -r gs://cloud-aoss/utils/python-download-metadata/v1.0/README.md PATH_TO_LOCAL_STORE

Where PATH_TO_LOCAL_STORE is the local path where you want to save the file.

Before executing this script, ensure that the client libraries are installed in the local environment and the Application Development Credentials are set up.

Sample Command

python3 ./download_metadata.py -l 'JAVA' -p 'org.apache.logging.log4j:log4j-core' -v '2.17.1'</th>

Learn more

Assured Open Source Software is part of the Software Delivery Shield solution. Software Delivery Shield is a fully-managed, end-to-end software supply chain security solution that helps you to improve the security posture of developer workflows and tools, software dependencies, CI/CD systems used to build and deploy your software, and runtime environments such as Google Kubernetes Engine and Cloud Run. To learn how you can use Assured Open Source Software with other components of Software Delivery Shield to improve the security posture of your software supply chain, see Software Delivery Shield overview.

What's next?