Cloud customer?
Start for Free >
Upgrade in MyJFrog >
What's New in Cloud >





Filestore Overview

The filestore is where Artifactory physically stores the binaries. Artifactory uses the filestore in conjunction with the database to manage binary storage. For more information on database, see Common Resources.

You can manage filestore through the binarystore.xml configuration file located in the $JFROG_HOME/artifactory/var/etc/artifactory folder. You can modify this file to implement a variety of storage configurations including basic and customizable local and network based file-systems, cloud storage, and redundant storage. 

In addition to using different storage providers, you can also chain a series of storage providers together to build complex and scalable binary storage setups.

Review Best Practices For Managing Your Artifactory Filestore before you proceed to configure the filestore.

Take care when you modify binarystore.xml

Making changes to this file may result in losing binaries stored in Artifactory!

If you are not sure of what you are doing, contact JFrog Support for assistance.

Binary Provider

A binary provider is a storage element that is based on checksum-based storage system. You can use one or more binary providers to build your storage setup.

The  binarystore.xml  file can include a chain with a set of binary providers. You can embed binary providers into one another to form chains that represent a coherent filestore.



Page contents

 


Getting Started

Artifactory includes the following default binarystore.xml configuration.

<config version="v1">
	<chain template="file-system"/>
</config>
  • <config> The wrapper element containing a chain element. The binarystore.xml must start and end with this element.
  • <chain template> The structure of the filestore. To use one of the built-in filestores, the chain element needs to include the corresponding template attribute.

The <config> tag is a marker element for versioning configurations. It does not affect Artifactory's settings.


Chain Templates

The following sections describe the basic chain templates come built-in with Artifactory and are ready for you to use out-of-the-box, as well as other binary providers that are included in the default chains.

Click on the template name to navigate to the corresponding binary provider template.

file-system

The most basic filestore configuration for Artifactory used for a local or mounted filestore.

cache-fs

Works the same way as filesystem but also caches download requests that are cleaned up using an LRU (Least Recently Used) protocol. Improves performance of instances with high IOPS (I/O Operations) or slow NFS access.

full-db

All the metadata and the binaries are stored as blobs in the Artifactory database with an additional layer of caching.

full-db-direct
All the metadata and the binaries are stored as blobs in the Artifactory database without caching.
s3-storage-v3

A file store configuration to use S3 Cloud Storage using the official Amazon SDK.

You can optimize the JFrog Platform for the download of large binaries, such as Docker images, from cloud storage, by delegating its function as a registry so it responds to download requests with a link. For more information, see Direct Cloud Storage Download.

cluster-s3-storage-v3

A file store configuration where data is temporarily stored on the file system of each node using the Eventual-Cluster Binary Provider , and is then passed on to your S3 object storage for persistent storage

You can optimize the JFrog Platform for the download of large binaries, such as Docker images, from cloud storage, by delegating its function as a registry so it responds to download requests with a link. For more information, see Direct Cloud Storage Download.

s3-sharding
A file store configuration to use multiple S3 buckets in a sharded configuration. 
s3-direct
A file store configuration that allows directly uploading to S3, bypassing the eventual upload mechanism.
s3

This is the setting used to store data in S3 using the JetS3t library. 

You can optimize the JFrog Platform for the download of large binaries, such as Docker images, from cloud storage, by delegating its function as a registry so it responds to download requests with a link. For more information, see Direct Cloud Storage Download.

Important

Because the JetS3t library is no longer maintained; therefore, this template is being deprecated in Artifactory in the second quarter of 2022. You should use the s3-storage-v3 instead, which uses the official, highly-maintained AWS S3 SDK.

The transition should be seamless between s3 to s3-storage-v3, as most parameters are the same between the two providers. To learn more, see Amazon S3 Official SDK Template.

google-storage-v2
The google-storage-v2 template is used for configuring Google Cloud Storage as the remote filestore using the Google native client.
google-storage

This is the setting used for Google Cloud Storage as the remote filestore.

Important

Because the JetS3t library is no longer maintained; therefore, this template is being deprecated in Artifactory in the second quarter of 2022. You should use the google-storage-v2 instead, which uses the official SDK.

The transition should be seamless between google-storage to google-storage-v2, as most parameters are the same between the two providers. 

azure-blob-storage-v2
This is the setting used for Azure Blob Storage v2 as the remote filestore.
azure-blob-storage
This is the setting used for Azure Blob Storage as the remote filestore.
double-shards

A pure sharding configuration that uses two physical mounts with one copy, which results in each artifact saved only once.

redundant-shards
A pure sharding configuration that uses two physical mounts with two copies, which results in each shard storing a copy of each artifact.
cluster-filesystem
A filestore configuration where each node has its own local filestore (just like the file-system chain) and is connected to all other nodes through dynamically allocated Remote Binary Providers using the Sharding-Cluster provider.
cluster-s3

This is the setting used for Amazon S3 using the JetS3t library. It is based on the sharding and dynamic provider logic that synchronizes the cluster-file-system.

The JetS3t library is no longer maintained and hence this template will be deprecated in the future.

cluster-google-storage-v2
This is the setting used for Google Cloud Storage in a cluster configuration as the remote filestore.
cluster-google-storage

This is the setting used for Google Cloud Storage using the JetS3t library. It is based on the sharding and dynamic provider logic that synchronizes the cluster-file-system.

The JetS3t library is no longer maintained and hence this template will be deprecated in the future.

Important

Because the JetS3t library is no longer maintained; therefore, this template is being deprecated in Artifactory in the second quarter of 2022. You should use the cluster-google-storage-v2 instead, which uses the official SDK.

The transition should be seamless between google-storage to google-storage-v2, as most parameters are the same between the two providers.

cluster-azure-blobstorage

This is the setting used for Azure Blob Storage. It is based on the sharding and dynamic provider logic that synchronizes the cluster-file-system.


Modifying an Existing Chain Template

To accommodate any specific requirements you may have for your filestore, modify one of the existing chain templates either by extending it with additional binary providers or by overriding one of its attributes.

For example, the built-in filesystem chain template stores binaries under the $JFROG_HOME/artifactory/var/data/artifactory/filestore directory. To modify the template so that it stores binaries under $FILESTORE/binaries you could extend it as follows:

<!-- file-system chain template structure  -->  
<config version="v1">   
	<chain template="file-system"/>
	<provider id="file-system" type="file-system"> 				<!-- Modify the "file-system" binary provider -->
		<baseDataDir>$FILESTORE/binaries</baseDataDir>		<!-- Override the <baseDataDir> attribute -->
	</provider>
</config>



Configuring a Custom Filestore From Scratch


In addition to the built-in filestore chain templates below, you may construct custom chain template to accommodate any filestore structure you need.
Since the different binary providers in the filestore must be compatible with each other, misconfiguration might lead to data loss.  For configuring a custom filestore, contact JFrog Support

Copyright © 2022 JFrog Ltd.