Access metadata using Cloud Storage

Assured OSS provides security metadata for every available package. Each version of the package has its own metadata. This page explains what information is provided as part of the metadata and how you can access this metadata.

You can access the security metadata using one of the following options:

  • Artifact Analysis API
  • Cloud Storage

This page applies to the free tier only.

Security metadata files

The security metadata for each Assured OSS package version is distributed across the following four files in a Cloud Storage bucket:

  • buildinfo.zip: This file contains metadata about the build information of a package. This metadata would change in very rare scenarios like re-building of package, certificate rotation, and addition of new fields.
  • vexinfo.zip: This file contains the vulnerability information for a package in CycloneDX 1.4 format. This is expected to change whenever there is any information about a vulnerability for the package.
  • healthinfo.zip: This file contains details about the testing performed on a package. This can change if the testing status of a package gets modified. For example, if a new security testing is performed or if coverage result is changed.
  • licenseinfo.zip: This file contains metadata pertaining to the license information of a package. The license information of a package can change if the publisher of the package modifies the information.

Each zip file contains the following two files:

  • A JSON file containing data.
  • A signature.zip file containing the signature for that data.

JSON file structure

The structure for the JSON files is described in the following section:

buildInfo.json

  • creationTime: the time of creation for this document in RFC 3339 string format.
  • creator: the owner of the document. The following details are provided:
    • name: the name of the organization that created this document.
    • email: the email address of the organization in case of any query or complaint.
  • buildDetails: the details related to building the binary. The following details are provided:
    • packageFileName: the name of the file to which the build details apply.
    • buildProvenance: a string representing build provenance of the package in SLSA v0.2 format. This is generated by Cloud Build.
    • envelope: a string representing a DSSE envelope that can be used to verify the integrity of the provenance document. This is also generated by Cloud Build.
    • slsaLevel: This indicates the SLSA level adhered by the build system.
    • buildTool: Name of the tool being used for building package, which is Cloud Build. For more information, see Cloud Build.
    • transitiveClosureState: This is an ENUM indicating if all the build dependencies for the package (direct or indirect) are also present in Assured OSS's portfolio or not. This can have two values:
      • OPEN: If zero or only partial transitive dependencies are supported by Assured OSS.
      • CLOSED: If all transitive dependencies are supported by Assured OSS.
  • sourceInfo: the information about the source code that was used to build the package. The following details are provided:
    • sourceUrl: the GitHub URL string.
    • commitHash: commit hash string attached to the release.
    • tag: the release tag associated with the package version.
    • host: name of the system that hosts the source code in GitHub.
    • commitTime: the time of a commit in RFC 3339 string format.
  • sbom - SBOM string in SPDX 2.3 format.

vexInfo.json

  • creationTime: the time of creation for this document in RFC 3339 string format.
  • creator: the owner of the document. The following details are provided:
    • name: the name of the organization that created this document.
    • email: the email address of the organization in case of any query or complaint.
  • vexData: Vulnerability Exploitability eXchange (VEX) string in CycloneDX 1.4 format.

healthInfo.json

  • creationTime: the time of creation for this document in RFC 3339 string format.
  • creator: the owner of the document. The following details are provided:
    • name: the name of the organization that created this document.
    • email: the email address of the organization in case of any query or complaint.
  • testingData: the details about the security testing done on a package. The following details are provided:
    • testType: the type of test that was done. For example, FUZZ.
    • tool: the name of the tool that was used to perform the test.
    • testStatus: the status of the test. The status may be one of the following:
      • TESTED: testing was executed.
      • NOT_REQUIRED: testing was not required for the package. For example, fuzz testing is not required on a package that contains only interfaces.
      • UNTESTED: package was not tested.

licenseInfo.json

  • package_name: The name of the package in string format.
  • package_version: The version of the package.
  • license_info: The license associated with the package.

Access security metadata from Cloud Storage

To access the security metadata from Cloud Storage, follow these steps:

Step 1 - Set up authentication

For information about setting up authentication, see Set up authentication.

Step 2 - Construct the URL

You can use either gcloud storage or curl commands to download the metadata. Construct the URL for both using the following information:

  • Language: java or python. The value must be in lowercase.
  • Package_ID: for Java, it's groupId:artifactId and for Python it's packageName. The value must be in lower case.
  • Version: the version of the package.
  • Metadata_Type: choose between buildinfo.zip, vexinfo.zip, and healthinfo.zip.

The URL must have the following format:

gcloud

gs://cloud-aoss-metadata/<language>/<package_id>/<version>/<metadata_type>

Note that the URL must be in lowercase.

Sample Python URL: gs://cloud-aoss-metadata/python/cryptography/37.0.4/buildinfo.zip

Sample Java URL: gs://cloud-aoss-metadata/java/com.google.errorprone:error_prone_annotations/2.15.0/buildinfo.zip

curl

https://storage.googleapis.com/cloud-aoss-metadata/<language>/<package_id>/<version>/<metadata_type>

Note that the URL must be in lowercase.

Sample Python URL: https://storage.googleapis.com/cloud-aoss-metadata/python/cryptography/37.0.4/buildinfo.zip

Sample Java URL: https://storage.googleapis.com/cloud-aoss-metadata/java/com.google.errorprone:error_prone_annotations/2.15.0/buildinfo.zip

Step 3 - Download the metadata

Use the following commands to download the metadata:

gcloud

gcloud storage cp  "gs://cloud-aoss-metadata/<language>/<package_id>/<version>/<metadata_type>" outputFolderLocation

To download all the metadata types such as the build information, health information, and VEX information together for a given package and version, use the following command:

gcloud storage cp "gs://cloud-aoss-metadata/<language>/<package_id>/<version>" outputFolderLocation --recursive

The metadata is distributed across three zip files (buildinfo.zip, healthinfo.zip, vexinfo.zip) and downloaded into the output folder.

curl

curl -H "Authorization: Bearer $(gcloud auth print-access-token)" -L https://storage.googleapis.com/cloud-aoss-metadata/<language>/<package_id>/<version>/<metadata_type> -o output.zip

Step 4 - Extract the metadata

Extract the zip file. On macOS or Linux, use the unzip command: unzip input.zip -d outputFolder. Each zip file contains a JSON file and a signature.zip file. The JSON file contains the actual security metadata and the signature.zip file contains files to verify the Google signature on the JSON.

Download license information

The license information for all Assured OSS packages is available in a single JSON file. To download this file, run the following command:

gcloud

gcloud storage cp  "gs://cloud-aoss/info/LicenseInfo.json" outputFolderLocation

curl

curl -H "Authorization: Bearer $(gcloud auth print-access-token)" -L https://storage.googleapis.com/cloud-aoss/info/LicenseInfo.json

To download the signature for the license metadata file, run the following command:

gcloud

gcloud storage cp "gs://cloud-aoss/info/LicenseInfo.json-sig.zip" outputFolderLocation

curl

curl -H "Authorization: Bearer $(gcloud auth print-access-token)" -L https://storage.googleapis.com/cloud-aoss/info/LicenseInfo.json-sig.zip -o output.zip

What's next