View Source

Overview

Azure blob storage requires JFrog Container Registry, Pro, Enterprise X , or an Enterprise+ license.

Artifactory supports managing your Artifactory filestore on the cloud with Azure Blob Storage providing you with:

Massive scalability
On the cloud, your Artifactory filestore is massively scalable. You may freely continue to upload files without having to install or maintain any file storage devices. You can even upload files larger than 5 GB using multi-part upload with the blob size limit currently at 4.75 TB.
Security
An Azure Blob Storage account offers a variety of security capabilities such as role-based access control, Azure Active Directory, in-transit security, Storage Service encryption and more. For full details on the security capabilities provided by Azure Blob Storage, please refer to.
Disaster recovery
Since your files are replicated and stored with redundancy, using Azure Blob Storage offers the capability for disaster recovery.

Setting up Artifactory to use Azure Blob Storage will delete all files in your current filestore.

If you already have a running installation of Artifactory, then before you setup Artifactory to use Azure Blob Storage and migrate your filestore to the cloud, we strongly recommend that you do a complete system backup.

Setting up Azure Blob Storage

Artifactory requires the use of Azure block blob storage, a storage system optimized for unstructured text and binary data. Block blobs are made up of blocks of data that can be managed individually, which is appropriate as an Artifactory filestore.

Before setting up an Artifactory for filestore, you must.

To learn more, see in the Microsoft Azure documentation.

Setting up Artifactory to Use Azure Blob Storage

Whether you are installing Artifactory for the first time, or are moving your filestore to Azure Blob Storage in the context of upgrading Artifactory, we recommend that you first do a standard installation of Artifactory using the default settings, or a standard upgrade using your current settings.

To move your Artifactory filestore to Azure Blob Storage, execute the following steps:

Shut down Artifactory.
Set your license
Configure Artifactory to use Azure Blob Storage
To configure Artifactory to use Azure Blob Storage, you need to use the Azure Blob Storage binary provider described in Configuring the Filestore.
Migrate your filestore to the cloud
For an Artifactory HA cluster running version 5.0 or 6.0, to migrate your filestore, see Migrating Data from NFS.
Start up Artifactory

Configuring Artifactory to Use Azure

Artifactory's filestore is configured through the binarystore.xml file. The binarystore.xml configuration file is located in the $JFROG_HOME/artifactory/var/etc/artifactory folder.

Direct (Eventual-less) versus Eventual Upload Mechanism

If you are migrating from any eventual mechanism to the direct upload mechanism, make sure your eventual directory is empty or you could experience data loss.

The default Azure chain templates rely on an eventual upload mechanism, whereby an upload from a client is considered successful when the full binary has been uploaded to Artifactory. The direct upload mechanism serves as an alternative mechanism whereby an upload is not considered successful until it reaches Azure. There are advantages to both which we cover below.

Direct Upload Mechanism (Recommended)

The Direct Upload mechanism enables you to upload without the need to maintain persistent storage for the eventual directory. This mechanism may also allow for faster uploads, since there is less contention for disk IO, particularly when Artifactory is hosted on Azure.

Azure supports direct upload but not direct download.

Client uploads an artifact to Artifactory.
Artifactory receives and simultaneously uploads to Azure.
Artifactory finishes uploading the binary to Azure.
1. Artifactory returns 201 success to the client.
2. A database entry for the artifact is created.

Eventual Upload Mechanism

If you are uploading on a system where the Azure upload speed is slow (for example, when Artifactory is hosted on-prem), you may want to use the Eventual Upload mechanism. The Eventual Upload mechanism also allows you to upload when Azure is down or experiencing network issues.

Client uploads an artifact to Artifactory.
Artifactory receives the full upload.
1. Artifactory returns a 201 success message to the client.
2. A database entry for the artifact is created.
3. The binary is stored in an eventual directory on the local disk.
Artifactory uploads the binary to Azure.
The binary is deleted from the eventual directory.

Azure Blob Storage V2 Binary Provider

The azure-blob-storage-v2 template is used for configuring Azure Blob Storage as the remote filestore.

The snippets below show the basic template configuration and an examples that use the Azure Blob Storage binary provider.

This binary provider uses the following set of parameters.

testConnection

Default: true

When true, Artifactory uploads and downloads a file when starting up to verify that the connection to the cloud storage provider is fully functional.

accountName

The storage account can be a g eneral-purpose storage account or a blob storage account which is specialized for storing objects/blobs.

Your cloud storage provider identity.

accountKey

Your cloud storage provider authentication credential.

containerName

Your globally unique container name on Azure Blob Storage.

endpoint

The hostname. You should only use the default value unless you need to contact a different endpoint for testing or production purposes.
Note: if the endpoint is not defined in the xml file it will be automatically generated with the default https://<ACCOUNT_NAME>.blob.core.windows.net value.

The Azure blob.core.windows.net endpoint is supported for all JFrog subscriptions. To use additional endpoints, requires Enterprise and Enterprise+ subscriptions are required

httpsOnly

Default: true.

When true, the cloud binary provider can be accessed using a secure HTTPS connection.

binariesDir

The directory to store the artifacts.

Applicable for direct upload only.

The default is data/filestore.

tempDir

The temporary directory to store the artifacts during upload.

Applicable for direct upload only.

Because you must configure the Azure Blob Storage provider with parameters specific to your account (but can leave all other parameters with the recommended values).

Azure Direct Upload V2 Template (Recommended)

The Azure direct (or eventual-less) template allows directly uploading to Azure, bypassing the eventual upload mechanism. For more information, see Direct (Eventual-less) versus Eventual Upload Mechanism.

The following example shows a template modified to support Azure direct upload.

<config version="3">
    <chain template="azure-blob-storage-direct-v2"/>
    <provider type="cache-fs" id="cache-fs">
        <cacheProviderDir>/var/opt/jfrog/artifactory/data/cache</cacheProviderDir> 
        <maxCacheSize>10000000000</maxCacheSize>
    </provider>
    <provider id="azure-blob-storage-v2" type="azure-blob-storage-v2">
        <accountName>myaccount</accountName>
        <accountKey>somekey</accountKey>
        <endpoint>https://artifactoryhaprod.blob.core.windows.net/</endpoint>;
        <containerName>mycontainer</containerName> 
        <multiPartLimit>256000000</multiPartLimit>
        <multipartElementSize>100000000</multipartElementSize>
        <binariesDir>data/filestore</binariesDir>
        <tempDir>temp</tempDir>
    </provider>   
</config>

What's in the template?

The following snippet shows the default chain that uses azure-blob-storage-direct-v2 as the binary provider:

<config version="1">
    <chain template="azure-blob-storage-direct-v2">
        <provider id="cache-fs" type="cache-fs">
            <provider id="azure-blob-storage-v2" type="azure-blob-storage-v2"/>
        </provider>
    </chain>
</config>

For details about the cache-fs provider, see Cached Filesystem Binary Provider.

Azure Eventual Upload V2 Template

The following example shows a template modified to support Azure eventual upload.

 <config version="1">
    <chain template="azure-blob-storage-v2"/>
    <provider id="azure-blob-storage-v2" type="azure-blob-storage-v2">
        <accountName>XXXXXXXX</accountName>
        <accountKey>XXXXXXXX</accountKey>
        <endpoint>https://<ACCOUNT_NAME>.blob.core.windows.net/</endpoint>
        <containerName><NAME></containerName>
    </provider>
</config>

What's in the template?

The following snippet shows the default chain that uses azure-blob-storage as the binary provider:

<config version="1">
	<chain template="azure-blob-storage-v2">
	    <provider id="cache-fs" type="cache-fs">
	        <provider id="eventual" type="eventual">
	            <provider id="retry" type="retry">
	                <provider id="azure-blob-storage-v2" type="azure-blob-storage-v2"/>
	            </provider>
	        </provider>
	    </provider>
	</chain>
</config>

For details about the cache-fs provider, see Cached Filesystem Binary Provider.
For details about the eventual provider, see Eventual Binary Provider.
For details about the retry provider, see Retry Binary Provider.

Azure Blob Storage Cluster V2 Binary Provider

This is the setting used for Azure Blob Storage. It is based on the sharding and dynamic provider logic that synchronizes the cluster-file-system.
When using the cluster-azure-blob-storage template, data is temporarily stored on the file system of each node using the Eventual Binary Provider, and is then passed on to your Azure Blob Storage for persistent storage.
Each node has its own local filestore (just like in the file-system binary provider) and is connected to all other cluster nodes via dynamically allocated Remote Binary Providers using the Sharding Cluster Binary Provider.

cluster-azure-blob-storage-v2 template configuration

Because you must configure the azure-blob-storage-v2 provider with parameters specific to your account (but can leave all other parameters with the recommended values), if you choose to use the cluster-azure-blob-storage-v2 template, your binarystore.xml configuration file should look like this:

<config version="2">
	<chain template="cluster-azure-blob-storage-v2"/>
    <provider id="azure-blob-storage-v2" type="azure-blob-storage-v2">
        <accountName>XXXXXX</accountName>
        <accountKey>XXXXXX</accountKey>
        <endpoint>https://<ACCOUNT_NAME>.blob.core.windows.net/</endpoint>
        <containerName><NAME></containerName>
    </provider>
</config>

What's in the template?

While you don't need to configure anything else in your binarystore.xml, this is what the cluster-azure-blob-storage-v2 template looks like under the hood:

<config version="2">
	<chain template="cluster-azure-blob-storage-v2">
       <provider id="cache-fs-eventual-azure-blob-storage" type="cache-fs">
           <provider id="sharding-cluster-eventual-azure-blob-storage-v2" type="sharding-cluster">
               <sub-provider id="eventual-cluster-azure-blob-storage-v2" type="eventual-cluster">
                   <provider id="retry-azure-blob-storage-v2" type="retry">
                       <provider id="azure-blob-storage-v2" type="azure-blob-storage-v2"/>
                   </provider>
               </sub-provider>
               <dynamic-provider id="remote-azure-blob-storage-v2" type="remote"/>
           </provider>
       </provider>
   </chain>

	<!-- cluster eventual Azure Blob Storage Service default chain -->
	<provider id="sharding-cluster-eventual-azure-blob-storage-v2" type="sharding-cluster">
		<readBehavior>crossNetworkStrategy</readBehavior>
		<writeBehavior>crossNetworkStrategy</writeBehavior>
		<redundancy>2</redundancy>
		<lenientLimit>1</lenientLimit>
		<property name="zones" value="local,remote"/>
	</provider>

	<provider id="remote-azure-blob-storage-v2" type="remote">
		<zone>remote</zone>
	</provider>

	<provider id="eventual-cluster-azure-blob-storage-v2" type="eventual-cluster">
		<zone>local</zone>
	</provider>

	<!--cluster eventual template-->
	<provider id="azure-blob-storage-v2" type="azure-blob-storage-v2">
		<accountName>XXXXXX</accountName>
		<accountKey>XXXXXX</accountKey>
		<endpoint>https://<ACCOUNT_NAME>.blob.core.windows.net/</endpoint>
		<containerName><NAME></containerName>
	</provider>
</config>

For details about the cache-fs provider, see Cached Filesystem Binary Provider.

For details about the eventual provider, see Eventual Binary Provider.

For details about the retry provider, see Retry Binary Provider.

For details about the remote binary provider, see Remote Binary Provider.

For details about the sharding-cluster, see Sharding Cluster Binary Provider.

Azure Blob Storage V1 Binary Provider

The azure-blob-storage template is used for configuring Azure Blob Storage as the remote filestore.

The snippets below show the basic template configuration and an examples that use the Azure Blob Storage binary provider.

This binary provider uses the following set of parameters.

testConnection

Default: true

When true, Artifactory uploads and downloads a file when starting up to verify that the connection to the cloud storage provider is fully functional.

accountName

The storage account can be a General-purpose storage account or a Blob storage account which is specialized for storing objects/blobs.

Your cloud storage provider identity.

accountKey

Your cloud storage provider authentication credential.

containerName

Your globally unique container name on Azure Blob Storage.

endpoint

The hostname. You should only use the default value unless you need to contact a different endpoint for testing or production purposes.
Note: if the endpoint is not defined in the xml file it will be automatically generated with the default https://<ACCOUNT_NAME>.blob.core.windows.net value.

The Azure blob.core.windows.net endpoint is supported for all JFrog subscriptions. To use additional endpoints, requires Enterprise and Enterprise+ subscriptions are required

httpsOnly

Default: true.

When true, the cloud binary provider can be accessed using a secure HTTPS connection.

binariesDir

The directory to store the artifacts.

Applicable for direct upload only.

The default is data/filestore.

tempDir

The temporary directory to store the artifacts during upload.

Applicable for direct upload only.

Because you must configure the Azure Blob Storage provider with parameters specific to your account (but can leave all other parameters with the recommended values).

Azure Direct Upload V1 Template (Recommended)

The Azure direct (or eventual-less) template allows directly uploading to Azure, bypassing the eventual upload mechanism. For more information, see Direct (Eventual-less) versus Eventual Upload Mechanism.

The following example shows a template modified to support Azure direct upload.

<config version="3">
    <chain template="azure-blob-storage-direct"/>
    <provider type="cache-fs" id="cache-fs">
        <cacheProviderDir>/var/opt/jfrog/artifactory/data/cache</cacheProviderDir> 
        <maxCacheSize>10000000000</maxCacheSize>
    </provider>
    <provider id="azure-blob-storage" type="azure-blob-storage">
        <accountName>myaccount</accountName>
        <accountKey>somekey</accountKey>
        <endpoint>https://artifactoryhaprod.blob.core.windows.net/</endpoint>;
        <containerName>mycontainer</containerName> 
        <multiPartLimit>256000000</multiPartLimit>
        <multipartElementSize>100000000</multipartElementSize>
        <binariesDir>data/filestore</binariesDir>
        <tempDir>temp</tempDir>
    </provider>   
</config>

What's in the template?

The following snippet shows the default chain that uses azure-blob-storage-direct as the binary provider:

<config version="1">
    <chain template="azure-blob-storage-direct">
        <provider id="cache-fs" type="cache-fs">
            <provider id="azure-blob-storage" type="azure-blob-storage"/>
        </provider>
    </chain>
</config>

For details about the cache-fs provider, see Cached Filesystem Binary Provider.

Azure Eventual Upload Template

The following example shows a template modified to support Azure eventual upload.

 <config version="1">
    <chain template="azure-blob-storage"/>
    <provider id="azure-blob-storage" type="azure-blob-storage">
        <accountName>XXXXXXXX</accountName>
        <accountKey>XXXXXXXX</accountKey>
        <endpoint>https://<ACCOUNT_NAME>.blob.core.windows.net/</endpoint>
        <containerName><NAME></containerName>
    </provider>
</config>

What's in the template?

The following snippet shows the default chain that uses azure-blob-storage as the binary provider:

<config version="1">
	<chain template="azure-blob-storage">
	    <provider id="cache-fs" type="cache-fs">
	        <provider id="eventual" type="eventual">
	            <provider id="retry" type="retry">
	                <provider id="azure-blob-storage" type="azure-blob-storage"/>
	            </provider>
	        </provider>
	    </provider>
	</chain>
</config>

For details about the cache-fs provider, see Cached Filesystem Binary Provider.
For details about the eventual provider, see Eventual Binary Provider.
For details about the retry provider, see Retry Binary Provider.

Azure Blob Storage Cluster V1 Binary Provider

This is the setting used for Azure Blob Storage. It is based on the sharding and dynamic provider logic that synchronizes the cluster-file-system.
When using the cluster-azure-blob-storage template, data is temporarily stored on the file system of each node using the Eventual Binary Provider, and is then passed on to your Azure Blob Storage for persistent storage.
Each node has its own local filestore (just like in the file-system binary provider) and is connected to all other cluster nodes via dynamically allocated Remote Binary Providers using the Sharding Cluster Binary Provider..

cluster-azure-blob-storage template configuration

Because you must configure the azure-blob-storage provider with parameters specific to your account (but can leave all other parameters with the recommended values), if you choose to use the cluster-azure-blob-storage template, your binarystore.xml configuration file should look like this:

<config version="2">
	<chain template="cluster-azure-blob-storage"/>
    <provider id="azure-blob-storage" type="azure-blob-storage">
        <accountName>XXXXXX</accountName>
        <accountKey>XXXXXX</accountKey>
        <endpoint>https://<ACCOUNT_NAME>.blob.core.windows.net/</endpoint>
        <containerName><NAME></containerName>
    </provider>
</config>

What's in the template?

While you don't need to configure anything else in your binarystore.xml, this is what the cluster-azure-blob-storage template looks like under the hood:

<config version="2">
	<chain template="cluster-azure-blob-storage">
       <provider id="cache-fs-eventual-azure-blob-storage" type="cache-fs">
           <provider id="sharding-cluster-eventual-azure-blob-storage" type="sharding-cluster">
               <sub-provider id="eventual-cluster-azure-blob-storage" type="eventual-cluster">
                   <provider id="retry-azure-blob-storage" type="retry">
                       <provider id="azure-blob-storage" type="azure-blob-storage"/>
                   </provider>
               </sub-provider>
               <dynamic-provider id="remote-azure-blob-storage" type="remote"/>
           </provider>
       </provider>
   </chain>

	<!-- cluster eventual Azure Blob Storage Service default chain -->
	<provider id="sharding-cluster-eventual-azure-blob-storage" type="sharding-cluster">
		<readBehavior>crossNetworkStrategy</readBehavior>
		<writeBehavior>crossNetworkStrategy</writeBehavior>
		<redundancy>2</redundancy>
		<lenientLimit>1</lenientLimit>
		<property name="zones" value="local,remote"/>
	</provider>

	<provider id="remote-azure-blob-storage" type="remote">
		<zone>remote</zone>
	</provider>

	<provider id="eventual-cluster-azure-blob-storage" type="eventual-cluster">
		<zone>local</zone>
	</provider>

	<!--cluster eventual template-->
	<provider id="azure-blob-storage" type="azure-blob-storage">
		<accountName>XXXXXX</accountName>
		<accountKey>XXXXXX</accountKey>
		<endpoint>https://<ACCOUNT_NAME>.blob.core.windows.net/</endpoint>
		<containerName><NAME></containerName>
	</provider>
</config>

For details about the cache-fs provider, see Cached Filesystem Binary Provider.

For details about the eventual provider, see Eventual Binary Provider.

For details about the retry provider, see Retry Binary Provider.

For details about the remote binary provider, see Remote Binary Provider.

For details about the sharding-cluster, see Sharding Cluster Binary Provider.