Cloud customer?
Start for Free >
Upgrade in MyJFrog >
What's New in Cloud >





Sharding-Cluster Binary Provider Overview

The sharding-cluster binary provider can be used together with other binary providers for both local or cloud-native storage. 

It adds a crossNetworkStrategy parameter to be used as read and write behaviors for validation of the redundancy values and the balance mechanism. It must include a Remote Binary Provider in its dynamic-provider setting to allow synchronizing providers across the cluster.

The Sharding-Cluster provider listens to cluster topology events and creates or removes dynamic providers based on the current state of nodes in the cluster.


Page Contents

Sharding-Cluster Binary Provider Template

type
sharding-cluster
zones

The zones defined in the sharding mechanism. Read/write strategies take providers based on zones.

lenientLimit

Default: 1 (From version 5.4. Note that for filestores configured with a custom chain and not using the built-in templates, the default value of the lenientLimit parameter is 0 to maintain consistency with previous versions.)

The minimum number of successful writes that must be maintained for an upload to be successful. The next balance cycle (triggered with the GC mechanism) will eventually transfer the binary to enough nodes such that the redundancy commitment is preserved.
In other words, Leniency governs the minimal allowed redundancy in cases where the redundancy commitment was not kept temporarily.

For example, if lenientLimit is set to 3, my setup includes 4 filestores, and 1 of them goes down, writing will continue. If a 2nd filestore goes down, writing will stop.

The amount of currently active nodes must always be greater or equal than the configured lenientLimit. If set to 0, the redundancy value has to be kept.

dynamic-provider
The type of provider that can be added and removed dynamically based on cluster topology changes. Currently only the Remote Binary Provider is supported as a dynamic provider.

Sharding-Cluster Binary Provider Example

<config version="v1">
    <chain>
        <provider id="cache-fs" type="cache-fs">   
            <provider id="sharding-cluster" type="sharding-cluster">
                <sub-provider id="state-aware" type="state-aware"/>
                <dynamic-provider id="remote" type="remote"/>
                <property name="zones" value="remote"/>
            </provider>
        </provider>
    </chain>
  
    <provider id="sharding-cluster" type="sharding-cluster">
        <readBehavior>crossNetworkStrategy</readBehavior>
        <writeBehavior>crossNetworkStrategy</writeBehavior>
        <redundancy>2</redundancy>
        <lenientLimit>1</lenientLimit>
    </provider>
  
    <provider id="state-aware" type="state-aware">
       <fileStoreDir>filestore1</fileStoreDir>
    </provider>
  
    <provider id="remote" type="remote">
        <checkPeriod>15000</checkPeriod>
        <connectionTimeout>5000</connectionTimeout>
        <socketTimeout>15000</socketTimeout>
        <maxConnections>200</maxConnections>
        <connectionRetry>2</connectionRetry>
        <zone>remote</zone>
    </provider>
</config
  • No labels
Copyright © 2022 JFrog Ltd.