Have a question? Want to report an issue? Contact JFrog support

Skip to end of metadata
Go to start of metadata

Overview

Artifactory fully supports S3 object storage for distributed file systems so your Artifactory filestore can reside on the cloud. This presents several benefits:

  1. Unlimited scalability
    Since your files are now stored on the cloud, this means that your Artifactory filestore is scalable and effectively unlimited (to the extent offered by your storage provider). You may freely continue to upload files without having to install or maintain any file storage devices. You can even upload files larger than 5 GB using multi-part upload.

  2. Security
    Enjoy the same security and authentication mechanisms provided by your S3 provider.

  3. Disaster recovery
    Since your files are replicated and stored with redundancy, this offers the capability for disaster recovery.

  4.  Support any S3 compatible distributed file system
     
    Arifactory's support is based on the S3 protocol. Any provider that uses S3, such as Ceph, Swift (through the S3 API) and others , will also be supported by Artifactory. With support for AWS S3 version 4, you can sign AWS requests using Signature Version 4.

Support for S3 object storage is included with an Artifactory Enterprise license.

In order to use S3 object storage with Artifactory, make sure you first install or upgrade to Artifactory V3.6 or later.

Backup your system. Your current filestore will be deleted.

Setting up Artifactory to use S3 will delete all files in your current filestore.

If you already have a running installation of Artifactory, then before you setup Artifactory to use S3 and migrate your filestore to the cloud, we strongly recommend that you do a complete system backup.

Page Contents


Setting up Artifactory to Use S3

First time installation or upgrade

If you are moving your filestore to S3 in the context of upgrading Artifactory, or a first time installation, we recommend that you first do a standard installation of Artifactory using the default settings, or a standard upgrade using your current settings.

In order to move your Artifactory filestore to the cloud, you need to execute the following steps:

Setting Your License

Artifactory supports S3 filestore with an Enterprise license. To use an S3 file store, make sure your $ARTIFACTORY_HOME/etc/artifactory.lic file is populated with an Enterprise license.

Configuring Artifactory to Use S3

From version 4.6, Artifactory's filestore is configured through the binarystore.xml file. For details, please refer to Configuring the Filestore

Migrating Your Filestore

The way to migrate your filestore over to your S3 provider depends on whether the version of Artifactory you are running is 5.0 and above or below 5.0:

For Artifactory 5.0 and above

For an Artifactory HA cluster running version 5.0 and above, to migrate your filestore so an S3 provider, please refer to Migrating Data from NFS.

For Artifactory Below Version 5.0

There are two ways to migrate your filestore over to your S3 provider. 

Automatic Filestore Migration (Recommended)

To make sure your filestore migration completes successfully without corrupting files, we recommend configuring Artifactory to do this migration for you automatically. To do so, you need to create the following links in $ARTIFACTORY_HOME/data/eventual/ (create it if the eventual folder does not exist):

  • A link with the name _add  that points to the $ARTIFACTORY_HOME/data/filestore directory (or to the  $CLUSTER_HOME/ha-data/filestore directory for an HA installation).
  • A link with the name _pre that points to the $ARTIFACTORY_HOME/data/_pre directory (or to the  $CLUSTER_HOME/ha-data/_pre directory for an HA installation)

With this setting, as soon as Artifactory starts up, it will automatically move your complete filestore over to your S3 provider.

Your current filestore will be deleted

The process of moving your filestore to your S3 provider will delete your current filestore. We strongly recommend you do a complete system backup before doing this migration.

Once the migration is complete, you may delete the _pre link and the $ARTIFACTORY_HOME/data/_pre directory (or the  $CLUSTER_HOME/ha-data/_pre directory for an HA installation)

Manual Filestore Migration

To migrate your filestore manually, you need to execute the following steps:

  • Stop Artifactory
  • Copy the $ARTIFACTORY_HOME/data/filestore directory to your S3 object storage to the bucket name and path specified when you configured Artifactory to use S3.
  • Start Artifactory

Using S3 with Older Versions

From version 4.6, Artifactory's filestore is configured through the binarystore.xml file. For details, please refer to Configuring the Filestore

To configure earlier versions of Artifactory to use your S3 object storage provider, you need to add or modify the following parameters to your  $ARTIFACTORY_HOME/etc/storage.properties file:

ParameterDescription
binary.provider.type

This must be set to S3.

Were you using fulldb?

If you were storing your binaries as blobs in the database (i.e. this was set to fulldb), make sure you remove all parameters connected to using fulldb.

binary.provider.s3.identity
Your username or IAM user key for your S3 provider.
binary.provider.s3.credential
Your password or IAM secret for your S3 provider.
binary.provider.s3.endpoint

The URL of your provider's endpoint for access.

Connect to the nearest endpoint

If your S3 provider supports connections to different endpoints, we recommend connecting to the geographically nearest endpoint for best performance. For example, if you are using AWS-S3, you should configure your endpoint according to the regions specified in the AWS documentation.

binary.provider.s3.bucket.name

The name of the bucket at your S3 provider.

This parameter must be globally unique.

binary.provider.s3.bucket.path

(Optional) Path within your bucket

Default: filestore

binary.provider.s3.https.only

If true, all communication with S3 will be via encrypted HTTPS connections, otherwise communications will be sent unencrypted via HTTP.

Default: True.
binary.provider.s3.env.s3service.server-side-encryption

Set to AES256 to use server-side encryption with the encryption key managed by Amazon S3. Omit this parameter if you are not using server-side encryption.

binary.provider.s3.proxy.hostExplicitly sets the host name of a HTTP proxy server (specify it without the "http://" prefix)
binary.provider.s3.proxy.portExplicitly sets the port number of a HTTP proxy server
binary.provider.s3.regionThe region offered by your cloud storage provider with which you want to work.

Keep your database settings

Make sure you don't change your database settings in your storage.properties file.

For example, a storage.properties file that uses an S3 object storage provider should look like this:

## Database settings - these should not be modified when migrating your filestore to S3
type=derby
url=jdbc:derby:{db.home};create=true
driver=org.apache.derby.jdbc.EmbeddedDriver
 
## Specify object storage provider
binary.provider.type=S3

## S3 identity
binary.provider.s3.identity=<Your S3 provider username>

## S3 credential
binary.provider.s3.credential=<Your S3 provider password>

## S3 endpoint  - in this example, this is the endpoint for AWS
binary.provider.s3.endpoint= http://s3.amazonaws.com