Store Go modules in Artifact Registry

Store Go modules in Artifact Registry

Set up a private Artifact Registry Go repository, upload a module to it, and use the module as a dependency.

Before you begin

1. Sign in to your Google Account.

   If you don't already have one, sign up for a new account.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Artifact Registry API.

   Enable the API

5. Install the Google Cloud CLI.

6. To initialize the gcloud CLI, run the following command:

   gcloud init

   Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.

7. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

8. Make sure that billing is enabled for your Google Cloud project.

9. Enable the Artifact Registry API.

   Enable the API

10. Install the Google Cloud CLI.

11. To initialize the gcloud CLI, run the following command:

    gcloud init

    Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.
12. Install Go1.15 or later.
13. Install the package-go-module gcloud CLI add-on:
    

    gcloud components install package-go-module

Create a repository

To create and configure a new repository:

1. Run the following command to create a new repository.

   gcloud artifacts repositories create REPOSITORY \
       --repository-format=go \
       --location=LOCATION \
       --description=DESCRIPTION
   

   Replace the following:

   • REPOSITORY is the name of the repository. For each repository location in a project, repository names must be unique.
   • LOCATION is the regional or multi-regional location for the repository. You can omit this flag if you set a default. To view a list of supported locations, run the command gcloud artifacts locations list.
   • DESCRIPTION is an optional description of the repository. Don't include sensitive data, since repository descriptions are not encrypted.

2. Run the following command to view the repository details.

     gcloud artifacts repositories describe --location=LOCATION REPOSITORY
   

   The output resembles the following:

     Encryption: Google-managed key
     Repository Size: 0.000MB
     createTime: '2022-06-03T20:20:01.644813Z'
     format: GO
     mode: STANDARD_REPOSITORY
     name: projects/my-project/locations/us-west1/repositories/my-repo
     updateTime: '2022-06-03T20:20:01.644813Z'
   

Set defaults for gcloud commands

You can simplify your gcloud CLI commands by configuring the defaults for project, repository and location values. Once defaults are configured, --project, --location, and --repository tags aren't needed.

Package and upload a Go module

The package-go-module gcloud CLI add-on packages your Go modules, allowing you to version and upload to Artifact Registry using the gcloud command.

Create a Go module

First, create a simple Go module to upload to your repository.

1. In your home directory, create a directory named "foo" for your module

   mkdir foo
   

2. Change directories into your module's directory and run go mod init to create a go.mod file for your module.

     cd foo \
     go mod init example.com/foo
   

   Replace example.com/foo with the module path. See the Go Modules Reference for more information.

3. Create a foo.go file in your foo directory with the following contents:

   
   package foo
   
   const HelloWorld = "Hello World!"
   

Package and upload your module

Package and upload the module to your repository:

  gcloud artifacts go upload --project=PROJECT \
      --repository=REPOSITORY \
      --location=LOCATION \
      --module-path=example.com/foo \
      --version=VERSION \
      --source=SOURCE_LOCATION

Replace the following:

• PROJECT with your Google Cloud project ID.
• REPOSITORY with the name of the repository where the package is stored.
• LOCATION with the regional or multi-regional location of the repository.
• example.com/foo with the module path. See the Go Modules Reference for more information.
• VERSION with the semantic version of the module in the form vX.Y.Z where X is the major version, Y is the minor version, and Z is the patch version.
• SOURCE_LOCATION with the path to the root directory of your Go module. If you omit the --source flag, the default is the current directory.

The module is uploaded to Artifact Registry.

For more information on creating Go modules, see this tutorial.

List modules

Run the following command to inspect the uploaded Go module in the default project, repository, and location when the default values are configured:

  gcloud artifacts packages list

The output resembles the following:

  Listing items under project my-project, location us-west1, repository my-repo.

  PACKAGE                   CREATE_TIME          UPDATE_TIME
  example.com/foo           2022-06-03T20:43:39  2022-06-20T20:37:40

View module version details

Run the following command to view the versions of your module in the default project, repository, and location when the default values are configured:

  gcloud artifacts versions list --package=MODULE_PATH

The output resembles the following:

  Listing items under project my-project, location us-west1, repository my-repo, package example.com/foo.

  VERSION  DESCRIPTION  CREATE_TIME          UPDATE_TIME
  v0.1.0                2022-06-03T20:43:39  2022-06-03T20:43:39
  v0.1.1                2022-06-20T20:37:40  2022-06-20T20:37:40

Download modules

To import modules stored in Artifact Registry, you need to instruct Go to look for dependencies from Artifact Registry, and bypass the checksum database.

Set up the Go environment

1. Instruct Go to download modules from Artifact Registry, the public Go module proxy, and then source in that order:

     export GOPROXY=https://LOCATION-go.pkg.dev/PROJECT/REPOSITORY,https://proxy.golang.org,direct
   

   Replace the following:

   • LOCATION is the regional or multi-regional location of the repository.
   • PROJECT is your Google Cloud project ID.
   • REPOSITORY is the name of the repository where the package is stored.

2. Exclude your module from being checked using the public checksum database:

     export GONOSUMDB=MODULE_PATH_REGEX
   

   Replace MODULE_PATH_REGEX with your module path or a regex if you want to exclude multiple modules.

   To exclude your module example.com/foo from being checked using the public checksum database, run the following command:

     export GONOSUMDB=example.com/foo
   

   If you want all modules with module paths beginning in example.com to be excluded from being checked using the public checksum database, run the following command:

     export GONOSUMDB=example.com/*
   

Authenticate to Artifact Registry

When downloading packaged Go modules to use as dependencies from Artifact Registry, the Go binary uses the credentials in your netrc file to authenticate to Artifact Registry. To simplify the authentication process, you can use the Go credential helper to refresh the tokens in your netrc file for authentication to Artifact Registry.

The location of your netrc file can be set with the netrc environment variable. If the NETRC variable is not set, then the go command will read $HOME/.netrc on UNIX-like platforms or %USERPROFILE%\_netrc on Windows.

Artifact Registry supports the following authentication methods.

Short-lived credentials (recommended)

Use the Artifact Registry Go credential helper tool to update the authentication tokens in your netrc file.

Use a service account key

Use this option when you can't use credentials in your environment for authentication. Add the unencrypted service account key to your netrc file.

Note: Service account keys are a security risk if not managed correctly. You should choose a more secure alternative to service account keys whenever possible. If you must authenticate with a service account key, you are responsible for the security of the private key and for other operations described by Best practices for managing service account keys. If you are prevented from creating a service account key, service account key creation might be disabled for your organization. For more information, see Managing secure-by-default organization resources.

Add the Go credential helper to GONOPROXY

Before using the Go credential helper you need to add it to the GONOPROXY list to force Go to download it directly from GitHub. If you have other modules you want to be downloaded directly from source you can add them in a comma separated list as shown in the following example:

  export GONOPROXY=MODULE_PATH1, MODULE_PATH2

Where MODULE_PATH1 and MODULE_PATH2 are module paths of modules to be downloaded from source.

To add the Go credential helper to your GONOPROXY list and run it to set up your credentials:

1. Add the Go credential helper to your GONOPROXY

     export GONOPROXY=github.com/GoogleCloudPlatform/artifact-registry-go-tools
   

2. Run the following command to add your Artifact Registry credentials to your netrc file with the Go module package tool:

     GOPROXY=proxy.golang.org \
         go run github.com/GoogleCloudPlatform/artifact-registry-go-tools/cmd/auth@v0.1.0 \
         add-locations --locations=LOCATION \
         [--json_key=path/to/service/account/key.json]
   

   Where LOCATION is the regional or multi-regional location of your repository. To add multiple locations, enter them as a comma separated list.

   The Go credential helper adds settings to your netrc file for authenticating to Artifact Registry. If you pass the --json_key flag the key is added to your netrc file for password authentication.

Use your module as a dependency

1. If you are using short-lived credentials for authenticating to Artifact Registry, you will need to refresh your OAuth token by running the following command:

     GOPROXY=proxy.golang.org \
     go run github.com/GoogleCloudPlatform/artifact-registry-go-tools/cmd/auth@v0.1.0 refresh
   

2. In your home directory, create a directory named "bar"

     mkdir bar
   

3. Change directories into your module's directory and run go mod init to create a go.mod file for your package.

     cd bar \
     go mod init example.com/bar
   

   Replace example.com/bar with the module path. See the Go Modules Reference for more information.

4. To require the version of foo stored in Artifact Registry, edit your go.mod file to resemble the following:

   
   module example.com/bar
   
   go 1.19
   
   require example.com/foo v0.1.0
   

   Replace the following:

   • example.com/foo is the module path of the required module
   • v0.1.0 is the version stored in Artifact Registry

5. Create a main.go file in your bar directory with the following contents:

     
     package main
   
     import (
       "fmt"
   
       foo "example.com/foo"
     )
   
     func main() {
       fmt.Println(foo.HelloWorld)
     }
   
     
   

6. Run go mod tidy to download dependencies, including the foo package:

     go mod tidy
   

7. Run the bar module:

     go run .
   

   The output resembles the following:

     Hello World!
   

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps. Before you remove the repository, ensure that any modules you want to keep are available in another location.

1. To delete the repository:

     gcloud artifacts repositories delete \
         --location=LOCATION \
         --project=PROJECT \
         REPOSITORY
   

   Replace the following:

   • LOCATION with the regional or multi-regional location of the repository.
   • PROJECT with your Google Cloud project ID.
   • REPOSITORY with the name of the repository.

2. If you want to remove the default repository and location settings that you configured for the active gcloud configuration, run the following commands:

     gcloud config unset artifacts/repository
     gcloud config unset artifacts/location
   

What's next

• Learn more about configuring authentication.
• Learn about managing repositories.
• Learn about managing Go modules.
• Read our resources about DevOps and explore the DevOps Research and Assessment research program.