In the free tier, Assured Open Source Software packages are hosted in a Google-managed Artifact Registry repository.
This page explains how you can connect to the Artifact Registry repository for Assured OSS to directly access and download the Python packages.
This document applies to the free tier only. For the premium tier, see Download Python packages using direct repository access.
Before you begin
Configure Assured OSS. For the free tier, submit the customer enablement form to enable access to Assured OSS.
Validate connectivity to Assured OSS for the requested service accounts.
Enable the Artifact Registry API for the parent Google Cloud project of the service accounts used to access Assured OSS.
Install the latest version of the Google Cloud CLI.
If you have installed the Google Cloud CLI previously, make sure you have the latest version by running the command:
gcloud components update
Set up authentication
You can authenticate to the Assured OSS Python package repository using one of the following methods:
- Authenticate with a keyring
- Authenticate with a service account key
The following sections describe how to set up these authentication methods.
Authenticate with keyring
To use the Python keyring to authenticate to the Artifact Registry, see authenticating with keyring and the information on the credential search order. We recommend that you use the Python keyring for authentication.
To set up the keyring for authentication, do the following:
Install the
keyring
library:pip install keyring
Install the Artifact Registry backend:
pip install keyrings.google-artifactregistry-auth
List backends to confirm the installation:
keyring --list-backends
The list should include the following:
ChainerBackend(priority:10)
GooglePythonAuth(priority:9)
For information about setting up Application Default Credentials, see Set up authentication.
This step ensures that the Assured OSS credential helper obtains your key when connecting with the repositories.
Authenticate with a service account key
Authenticate with a service account key when the application requires authentication with a username and a password.
Replace the package repository URL
https://us-python.pkg.dev/cloud-aoss/cloud-aoss-python/simple
with the URLhttps://_json_key_base64:BASE64_KEY@us-python.pkg.dev/cloud-aoss/cloud-aoss-python/simple
Replace BASE64_KEY with the base64-encoded service account JSON key file. To convert the entire service account JSON key file to base64 encoding, use the following command:
BASE64_KEY=$(cat KEY_FILE_LOCATION | base64 -w 0)
Replace the following:
- KEY_FILE_LOCATION: location of service account JSON key file
Install packages
The following instructions assume that you're using PyPI as the repository to download your dependencies. If you're using another repository, you must use different steps to download dependencies.
Install packages existing in Assured OSS
To specify the Assured OSS Python packages that you want to download, create two requirements.txt
files. The sample files are the following:
requirements-google.txt
# Packages present in Google's Artifact Registry urllib3==1.26.11 --hash=sha256:1cffe1aa066363a75c856f261c8fce62d87f7c40ce0f46453ea12bf652b12a13 jsonschema==4.13.0 --hash=sha256:29895bfe55b93b75552fbdd1e09aa0c82b7c1c9395d4f267e10c7d43cd31a74e
requirements-pypi.txt
# Packages present in Google's Artifact Registry are mentioned here so that pip # downloads their dependencies from PyPI. urllib3==1.26.11 jsonschema==4.13.0 # Below this comment, add any package version which you need but is NOT # present in Google's Artifact Registry and therefore needs to be downloaded from # PyPI.
To download the packages, run the following commands:
To download the required packages which are present in the Artifact Registry for Assured OSS, run the following:
pip install --require-hashes --requirement=requirements-google.txt \ --index-url https://_json_key_base64:BASE64_KEY@us-python.pkg.dev/cloud-aoss/cloud-aoss-python/simple -v \ --no-deps
Consider the following:
--require-hashes
is optional. If included, hashes are specified for all packages and for all package versions in therequirements.txt
file.- Replace
BASE64_KEY
with the base64-encoded service account JSON file. -v
is optional. If specified, the command gives more output.
To download the required packages which aren't present in the Artifact Registry for Assured OSS, run the following:
pip install --requirement=requirements-pypi.txt --index-url https://pypi.org/simple -v
This command also downloads the missing dependencies of the packages that you have downloaded using the previous command.
Access packages not available in Assured OSS
If you want access to packages that aren't available in the Artifact Registry repository for Assured OSS, you can do the following:
Assured OSS is also pre-configured with Assured OSS as the preferred repository and canonical public repositories, such as Maven Central or PyPI, as secondary repositories.
To use this feature (preview), you can point to a single URL:
https://us-python.pkg.dev/cloud-aoss/python/simple
List all Python packages available in Assured OSS
To use an API to get a list of all the Python packages in the Artifact Registry repository, see List all Python packages available in Assured OSS.
Generating the requirements-google.txt
file
This section provides additional information about generating the
requirements-google.txt
file that you need to download the Python
packages. To download the hash and generate the requirements-google.txt
file
for your environment, you can use one of the following two options:
Use a script to generate a single requirements file for all artifacts.
Download a separate requirements file for each artifact.
The following sections provide more information about these options.
Option 1: Use a script to generate a single requirements file for all artifacts
Use the generator.sh
script to generate a single
requirements-google.txt
file for all the Python packages
(along with their hashes) available with Assured OSS for linux operating system. The
packages must satisfy constraints such as Python version, machine architecture,
and operating system. You can then delete the package versions which you don't
require and use the resulting file.
The generator.sh
script helps in the following two ways:
- It generates the most recent list of Python package versions available with Assured OSS that can be successfully installed on your system.
- It generates the
requirements-google.txt
file along with all the hashes.
The required script and its README.md
file are available in a Cloud Storage bucket
(gs://cloud-aoss/utils/python-requirements-txt/v1.0
) which can be downloaded
using the Google Cloud CLI.
To download the script and the README.md
file, follow these steps:
Authenticate with the service account to access the Cloud Storage bucket using the following command:
gcloud auth activate-service-account --key-file KEY_FILE
Replace KEY_FILE with the path to the file containing the service account credentials.
Download the
generator.sh
to your machine using the following command:gcloud storage cp gs://cloud-aoss/utils/python-requirements-txt/v1.0/generator.sh PATH_TO_LOCAL_STORE --recursive
Replace PATH_TO_LOCAL_STORE with the local path where you want to save the downloaded file.
Download the
README.md
file using the following command:gcloud storage cp gs://cloud-aoss/utils/python-requirements-txt/v1.0/README.md PATH_TO_LOCAL_STORE --recursive
Replace PATH_TO_LOCAL_STORE with the local path where you want to save the downloaded file. The
README.md
file contains instructions on how to use the script.
To run the script, use the following commands:
To generate the
requirements-google.txt
file, run the following command:chmod +x generator.sh ./generator.sh
To get the information of the package in a CSV file, run the following command:
chmod +x generator.sh ./generator.sh -i
Option 2: Download requirements.txt
for each required artifact
You can also download a separate requirements.txt
(containing the hash) file
for each Python artifact and then combine them into a single requirements.txt
file.
Artifact hashes are available in a Cloud Storage bucket which can be downloaded
using the gcloud CLI. Hashes for each
package and a version are located at the gs://cloud-aoss/python/PACKAGE_NAME/VERSION
Cloud Storage bucket location.
To download the requirements.txt
file, follow these steps:
Authenticate with the service account to access the Cloud Storage bucket using the following command:
gcloud auth activate-service-account --key-file KEY_FILE
Replace KEY_FILE with the path to the file containing the service account credentials.
Download the
requirements.txt
of a specific package and version to your local machine using the following command:gcloud storage cp gs://cloud-aoss/python/PACKAGE_NAME/VERSION PATH_TO_LOCAL_STORE --recursive
Replace the following:
- PACKAGE_NAME: the package name
- VERSION: the version of the package
- PATH_TO_LOCAL_STORE: the local path where you want to download the file
Sample command:
gcloud storage cp gs://cloud-aoss/python/bleach/5.0.0 /tmp/bleach
--recursiveSample
requirements.txt
file:bleach==5.0.0 \ --hash=sha256:6d286e765bfd3e309209cfa1d063e4d46afa966dea8cb97431c02b1e3067d812
The contents of each such
requirements.txt
file can be combined into a singlerequirements-google.txt
file.
What's next
- Set up virtual repository access
- Supported Java and Python packages
- Access security metadata using Cloud Storage
- Access security metadata using Artifact Analysis API
- Subscribe to notifications