After you integrate Assured OSS with Security Command Center, Assured Open Source Software packages are hosted in an Artifact Registry repository that is created in a project that you control.
This page explains how you can connect to the Artifact Registry repository for Assured OSS to directly access and download the Python packages.
This document applies to the Assured OSS premium tier only. For the free tier, see Download Python packages using direct repository access for the free tier.
Before you begin
Validate connectivity to Assured OSS for the requested service accounts.
Install the latest version of the Google Cloud CLI.
If you have installed the Google Cloud CLI previously, make sure you have the latest version by running the command:
gcloud components update
Set up authentication
You can authenticate to the Assured OSS Python package repository using one of the following methods:
- Authenticate with a keyring
- Authenticate with a service account key
The following sections describe how to set up these authentication methods.
Authenticate with keyring
To use the Python keyring to authenticate to the Artifact Registry, see authenticating with keyring and the information on the credential search order. We recommend that you use the Python keyring for authentication.
To set up the keyring for authentication, do the following:
Install the
keyring
library:pip install keyring
Install the Artifact Registry backend:
pip install keyrings.google-artifactregistry-auth
List backends to confirm the installation:
keyring --list-backends
The list should include the following:
ChainerBackend(priority:10)
GooglePythonAuth(priority:9)
For information about setting up Application Default Credentials, see Set up authentication.
This step ensures that the Assured OSS credential helper obtains your key when connecting with the repositories.
Authenticate with a service account key
Authenticate with a service account key when the application requires authentication with a username and a password.
Replace the package repository URL
https://us-python.pkg.dev/PROJECT_ID/assuredoss-python/simple
with
the URL
https://_json_key_base64:BASE64_KEY@us-python.pkg.dev/PROJECT_ID/assuredoss-python/simple
Replace the following:
- Replace
PROJECT_ID
with the ID of the project that you selected when you set up Assured Open Source Software. Replace
BASE64_KEY
with the base64-encoding of entire service account JSON key file. To convert the entire service account JSON key file to base64-encoding use the following command:BASE64_KEY=$(cat KEY_FILE_LOCATION | base64)
Replace
KEY_FILE_LOCATION
with location of service account JSON key file.
Install packages
The following instructions assume that you're using PyPI as the repository to download your dependencies. If you're using another repository, you must use different steps to download dependencies.
Install packages existing in Assured OSS
To specify the Assured OSS Python packages that you want to download, create two requirements.txt
files. The sample files are the following:
requirements-google.txt
# Packages present in Google's Artifact Registry urllib3==1.26.11 --hash=sha256:1cffe1aa066363a75c856f261c8fce62d87f7c40ce0f46453ea12bf652b12a13 jsonschema==4.13.0 --hash=sha256:29895bfe55b93b75552fbdd1e09aa0c82b7c1c9395d4f267e10c7d43cd31a74e
requirements-pypi.txt
# Packages present in Google's Artifact Registry are mentioned here so that pip # downloads their dependencies from PyPI. urllib3==1.26.11 jsonschema==4.13.0 # Below this comment, add any package version which you need but is NOT # present in Google's Artifact Registry and therefore needs to be downloaded from # PyPI.
To download the packages, run the following commands:
To download the required packages which are present in the Artifact Registry for Assured OSS, run the following:
pip install --require-hashes --requirement=requirements-google.txt --index-url https://_json_key_base64:BASE64_KEY@us-python.pkg.dev/PROJECT_ID/assuredoss-python repo/simple -v --no-deps
Consider the following:
--require-hashes
is optional. If included, hashes are specified for all packages and for all package versions in therequirements.txt
file.-v
is optional. If specified, the command gives more output.
To download the required packages which aren't present in the Artifact Registry for Assured OSS, run the following:
pip install --requirement=requirements-pypi.txt --index-url https://pypi.org/simple -v
This command also downloads the missing dependencies of the packages that you have downloaded using the previous command.
List all Python packages available in Assured OSS
To use an API to get a list of all the Python packages in the Artifact Registry repository, see List all Python packages available in Assured OSS.
Generating the requirements-google.txt
file
This section provides additional information about generating the
requirements-google.txt
file that you need to download the Python
packages. To download the hash and generate the requirements-google.txt
file
for your environment, you can use one of the following two options:
Use a script to generate a single requirements file for all artifacts.
Download a separate requirements file for each artifact.
The following sections provide more information about these options.
Option 1: Use a script to generate a single requirements file for all artifacts
Use the generator.sh
script to generate a single
requirements-google.txt
file for all the Python packages
(along with their hashes) available with Assured OSS for linux operating system. The
packages must satisfy constraints such as Python version, machine architecture,
and operating system. You can then delete the package versions which you don't
require and use the resulting file.
The generator.sh
script helps in the following two ways:
- It generates the most recent list of Python package versions available with Assured OSS that can be successfully installed on your system.
- It generates the
requirements-google.txt
file along with all the hashes.
The required script and its README.md
file are available in a Cloud Storage bucket
(gs://cloud-aoss/utils/python-requirements-txt/v1.0
) which can be downloaded
using the Google Cloud CLI.
To download the script and the README.md
file, follow these steps:
Authenticate with the service account to access the Cloud Storage bucket using the following command:
gcloud auth activate-service-account --key-file KEY_FILE
Replace
KEY_FILE
with the path to the file containing the credentials of the service account provided during the set up.Download the
generator.sh
to your machine using the following command:gcloud storage cp gs://cloud-aoss/utils/python-requirements-txt/v1.0/generator.sh PATH_TO_LOCAL_STORE --recursive
Replace
PATH_TO_LOCAL_STORE
with the local path where you want to save the downloaded file.Download the
README.md
file using the following command:gcloud storage cp gs://cloud-aoss/utils/python-requirements-txt/v1.0/README.md PATH_TO_LOCAL_STORE --recursive
Replace
PATH_TO_LOCAL_STORE
with the local path where you want to save the downloaded file. TheREADME.md
file contains instructions on how to use the script.
To run the script, use the following commands:
To generate the
requirements-google.txt
file, run the following command:chmod +x generator.sh ./generator.sh
To get the information of the package in a CSV file, run the following command:
chmod +x generator.sh ./generator.sh -i
Option 2: Download requirements.txt
for each required artifact
You can also download a separate requirements.txt
(containing the hash) file
for each Python artifact and then combine them into a single requirements.txt
file.
Artifact hashes are available in a Cloud Storage bucket which can be downloaded
using the gcloud CLI. Hashes for each
package and a version are located at the gs://cloud-aoss/python/PACKAGE_NAME/VERSION
Cloud Storage bucket location.
To download the requirements.txt
file, follow these steps:
Authenticate with the service account to access the Cloud Storage bucket using the following command:
gcloud auth activate-service-account --key-file KEY_FILE
Replace
KEY_FILE
with the path to the file containing the service account credentials.Download the
requirements.txt
of a specific package and version to your local machine using the following command:gcloud storage cp gs://cloud-aoss/python/PACKAGE_NAME/VERSION PATH_TO_LOCAL_STORE --recursive
Replace the following:
PACKAGE_NAME
: the package nameVERSION
: the version of the packagePATH_TO_LOCAL_STORE
: the local path where you want to download the file
Sample command:
gcloud storage cp gs://cloud-aoss/python/bleach/5.0.0 /tmp/bleach
--recursiveSample
requirements.txt
file:bleach==5.0.0 \ --hash=sha256:6d286e765bfd3e309209cfa1d063e4d46afa966dea8cb97431c02b1e3067d812
The contents of each such
requirements.txt
file can be combined into a singlerequirements-google.txt
file.
What's next
- Download Java packages using direct repository access in the premium tier
- Access security metadata and verify packages in the premium tier