Using Cloud Storage with a JFrog Artifactory filestore
Contributed by the Google Cloud community. Not official Google documentation.
This tutorial describes how to use Cloud Storage for a filestore managed by the JFrog Artifactory universal repository manager. When the procedure is complete, all artifacts pushed to Artifactory will be stored in Cloud Storage, and all artifacts retrieved through Artifactory will be drawn from Cloud Storage.
Using Cloud Storage for your Artifactory filestore provides a secure, scalable binary repository that is stored remotely and with redundancy, aiding swift data recovery from a local disaster.
Objectives
To move your Artifactory filestore to Cloud Storage, execute the following steps:
- Ready Cloud Storage
- Ready Artifactory
- Configure Artifactory to use Cloud Storage
- Migrate your filestore to Cloud Storage
Before you begin
This tutorial assumes you already have a Google Cloud account and have completed the Cloud Storage quickstart documentation.
You also need to have a licensed copy of JFrog Artifactory Enterprise Edition. v4.6 or later, and have completed the Artifactory installation procedure.
Costs
This tutorial uses billable components of Google Cloud, including Cloud Storage.
You can use the Pricing Calculator to generate a cost estimate based on your projected production usage.
Setting up Artifactory to use Cloud Storage
Step 1: Ready Cloud Storage
Acquire the keys necessary for Cloud Storage and Artifactory interoperability, and create a Cloud Storage Bucket to hold your binary repository data.
Acquire interoperable storage access keys
The API for Cloud Storage interoperability uses HMAC authentication and lets Cloud Storage interoperate with tools written for cloud storage systems.
To use Cloud Storage with Artifactory, turn on the Cloud Storage Interoperability API. This API is enabled per project member, not per project. Each member can set a default project and maintain their own access keys.
Once the API is enabled, use the Cloud Storage console to obtain your access key and access secret. You will use these values for the corresponding parameters in Artifactory when you configure it to use Cloud Storage:
| Cloud Storage Value | Description |
|---|---|
| access key | This Cloud Storage account value will be used as the Artifactory parameter identity. |
| access secret | This Cloud Storage account value will be used as the Artifactory parameter credential. |
Create a storage bucket in Cloud Storage
Use the Cloud Storage Browser to create a storage bucket in Cloud Storage, as the place where artifacts will be stored.
Step 2: Ready Artifactory
You must prepare Artifactory before assigning Cloud Storage as your filestore. The procedure you use depends on whether it is a new or existing installation of Artifactory.
Option A: Install Artifactory
If you have not yet installed Artifactory Enterprise Edition v4.6 or newer, do so now. Perform a standard installation using the default settings.
Option B: Existing Artifactory installation
If you have an installed version of Artifactory, perform these steps:
- Backup your current filestore: Just in case something goes wrong, we recommend backing up your current filestore. To preserve your current filestore as a safety measure, perform a complete system backup of Artifactory.
- Make sure your version of Artifactory supports Cloud Storage: If your version of Artifactory is not Enterprise Edition v4.6 or newer, you must perform a standard upgrade using your current settings.
- Stop Artifactory: Shut down Artifactory. It should not be running for the remaining steps.
Confirm your enterprise license key
To use Artifactory's support for Cloud Storage, you must have an enterprise
license for your Artifactory installation. Make sure your
$ARTIFACTORY_HOME/etc/artifactory.lic file contains your enterprise license
key.
Step 3: Configure Artifactory to use Cloud Storage
Artifactory offers flexible filestore management through the binarystore.xml
configuration file located in the $ARTIFACTORY_HOME/etc folder. By modifying
this file you can implement a variety of different binary storage
configurations.
To configure Artifactory to use a Cloud Storage object storage provider, you
need to specify google-storage as the binary provider in the configuration
file. You will also need to provde the name of the bucket to use, and the
interoperability storage keys.
Add this example XML
into the binarystore.xml file to setup a default chain that uses
google-storage as the binary provider:
You should use the interoperability values from your Cloud Storage account (as described in Step 1) for identity and credential. The name of the bucket you created in Cloud Storage should be used as the bucketName.
Other parameters
The full set of parameters available for the google-storage binary
provider offer other options that may be useful.
Artifactory uses the JetS3t framework to access Cloud Storage. Some of the parameters below are used to set the corresponding value in the framework. For more details, please refer to the JetS3t Configuration Guide.
| Parameter | Description |
|---|---|
| testConnection | Default: true. When true, the Artifactory uploads and downloads a file when starting up to verify that the connection to the cloud storage provider is fully functional. |
| multiPartLimit | Default: 100,000,000 bytes. File size threshold over which file uploads are chunked and multi-threaded. |
| identity | Your cloud storage provider identity. |
| credential | Your cloud storage provider authentication credential. |
| bucketName | Your globally unique bucket name on Cloud Storage. |
| path | The relative path to your files within the bucket |
| proxyIdentity proxyCredential proxyPort proxyHost | Corresponding parameters if you are accessing the cloud storage provider through a proxy server. |
| port | Default: 80. The port number through which you want to access Cloud Storage. You should only use the default value unless you need to contact a different endpoint for testing purposes. |
| endpoint | Default: commondatastorage.googleapis.com. The Cloud Storage hostname. You should only use the default value unless you need to contact a different endpoint for testing purposes. |
| httpsOnly | Default: false. Set to true if you only want to access Cloud Storage through a secure https connection. |
| httpsPort | Default: 443. The port number through which you want to access Cloud Storage securely through https. You should only use the default value unless you need to contact a different endpoint for testing purposes. |
| bucketExists | Default: false. Only available on google-storage. When true, it indicates to the binary provider that a bucket already exists in Google Cloud Storage and therefore does not need to be created. |
For more information, see the Artifactory User's Guide topic Configuring the Filestore.
Step 4: Migrate your filestore to Cloud Storage
To migrate your filestore, execute the following steps:
- Copy the
$ARTIFACTORY_HOME/data/filestoredirectory to your Cloud Storage bucket. - Start Artifactory
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.