Edit on GitHub
Report issue
Page history

Using Google Cloud Storage with a JFrog Artifactory filestore

Author(s): @jfrogtw ,   Published: 2018-09-24


This tutorial describes how to use Google Cloud Storage for a filestore managed by the JFrog Artifactory universal repository manager. When the procedure is complete, all artifacts pushed to Artifactory will be stored in Cloud Storage, and all artifacts retrieved through Artifactory will be drawn from Cloud Storage.

Using Cloud Storage for your Artifactory filestore provides a secure, scalable binary repository that is stored remotely and with redundancy, aiding swift data recovery from a local disaster.

Objectives

To move your Artifactory filestore to Cloud Storage, execute the following steps:

  • Ready Google Cloud Storage
  • Ready Artifactory
  • Configure Artifactory to use Cloud Storage
  • Migrate your filestore to Cloud Storage

Before You Begin

This tutorial assumes you already have a Google Cloud Platform (GCP) account and have completed the Google Cloud Storage quickstart documentation.

You also need to have a licensed copy of JFrog Artifactory Enterprise Edition. v4.6 or later, and have completed the Artifactory installation procedure.

Costs

This tutorial uses billable components of GCP, including:

  • Google Cloud Storage

You can use the Pricing Calculator to generate a cost estimate based on your projected production usage.

Setting up Artifactory to Use Cloud Storage

Step 1: Ready Google Cloud Storage

Acquire the keys necessary for Cloud Storage and Artifactory interoperability, and create a Cloud Storage Bucket to hold your binary repository data.

Acquire interoperable storage access keys

The API for Google Cloud Storage Interoperability uses HMAC authentication and lets Cloud Storage interoperate with tools written for cloud storage systems.

To use Cloud Storage with Artifactory, turn on the Google Cloud Storage Interoperability API. This API is enabled per project member, not per project. Each member can set a default project and maintain their own access keys.

Once the API is enabled, use the Cloud Storage console to obtain your access key and access secret. You will use these values for the corresponding parameters in Artifactory when you configure it to use Cloud Storage:

Cloud Storage Value Description
access key This Cloud Storage account value will be used as the Artifactory parameter identity.
access secret This Cloud Storage account value will be used as the Artifactory parameter credential.

Create a storage bucket in Cloud Storage

Use the Cloud Storage Browser to create a storage bucket in Cloud Storage, as the place where artifacts will be stored.

Step 2: Ready Artifactory

You must prepare Artifactory before assigning Cloud Storage as your filestore. The procedure you use depends on whether it is a new or existing installation of Artifactory.

Option A: Install Artifactory

If you have not yet installed Artifactory Enterprise Edition v4.6 or newer, do so now. Perform a standard installation using the default settings.

Option B: Existing Artifactory installation

If you have an installed version of Artifactory, perform these steps:

  • Backup your current filestore: Just in case something goes wrong, we recommend backing up your current filestore. To preserve your current filestore as a safety measure, perform a complete system backup of Artifactory.
  • Make sure your version of Artifactory supports Cloud Storage: If your version of Artifactory is not Enterprise Edition v4.6 or newer, you must perform a standard upgrade using your current settings.
  • Stop Artifactory: Shut down Artifactory. It should not be running for the remaining steps.

Confirm your enterprise license key

To use Artifactory's support for Cloud Storage, you must have an enterprise license for your Artifactory installation. Make sure your $ARTIFACTORY_HOME/etc/artifactory.lic file contains your enterprise license key.

Step 3: Configure Artifactory to use Cloud Storage

Artifactory offers flexible filestore management through the binarystore.xml configuration file located in the $ARTIFACTORY_HOME/etc folder. By modifying this file you can implement a variety of different binary storage configurations.

To configure Artifactory to use a Cloud Storage object storage provider, you need to specify google-storage as the binary provider in the configuration file. You will also need to provde the name of the bucket to use, and the interoperability storage keys.

Add this example XML into the binarystore.xml file to setup a default chain that uses google-storage as the binary provider:

You should use the interoperability values from your Cloud Storage account (as described in Step 1) for identity and credential. The name of the bucket you created in Cloud Storage should be used as the bucketName.

Other parameters

The full set of parameters available for the google-storage binary provider offer other options that may be useful.

Artifactory uses the JetS3t framework to access Cloud Storage. Some of the parameters below are used to set the corresponding value in the framework. For more details, please refer to the JetS3t Configuration Guide.

Parameter Description
testConnection Default: true. When true, the Artifactory uploads and downloads a file when starting up to verify that the connection to the cloud storage provider is fully functional.
multiPartLimit Default: 100,000,000 bytes. File size threshold over which file uploads are chunked and multi-threaded.
identity Your cloud storage provider identity.
credential Your cloud storage provider authentication credential.
bucketName Your globally unique bucket name on Cloud Storage.
path The relative path to your files within the bucket
proxyIdentity proxyCredential proxyPort proxyHost Corresponding parameters if you are accessing the cloud storage provider through a proxy server.
port Default: 80. The port number through which you want to access Cloud Storage. You should only use the default value unless you need to contact a different endpoint for testing purposes.
endpoint Default: commondatastorage.googleapis.com. The Cloud Storage hostname. You should only use the default value unless you need to contact a different endpoint for testing purposes.
httpsOnly Default: false. Set to true if you only want to access Cloud Storage through a secure https connection.
httpsPort Default: 443. The port number through which you want to access Cloud Storage securely through https. You should only use the default value unless you need to contact a different endpoint for testing purposes.
bucketExists Default: false. Only available on google-storage. When true, it indicates to the binary provider that a bucket already exists in Google Cloud Storage and therefore does not need to be created.

For more information, see the Artifactory User's Guide topic Configuring the Filestore.

Step 4: Migrate your filestore to Cloud Storage

To migrate your filestore, execute the following steps:

  • Copy the $ARTIFACTORY_HOME/data/filestore directory to your Cloud Storage bucket.
  • Start Artifactory

Submit a Tutorial

Share step-by-step guides

SUBMIT A TUTORIAL

Request a Tutorial

Ask for community help

SUBMIT A REQUEST

GCP Tutorials

Tutorials published by GCP

VIEW TUTORIALS

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.