Working with Federated Repositories

Setup Prerequisites

Artifactory Versions Must be Identical (prior to 7.49.6)

When using a version of Artifactory older than 7.49.6, you must verify that the same Artifactory version is installed in all the Artifactory instances hosting the members to be included in the Federation.

After all instances have been upgraded to Artifactory release 7.49.6, Artifactory includes multi-version support, which enables the members of a Federation to run different versions of Artifactory, even if the version at one site includes configuration features and values that are not supported on the versions running at other sites. Thanks to multi-version support, future upgrades after 7.49.6 can be performed on one site at a time, eliminating the need for simultaneous upgrades across all locations.

For more information, see Multi-Version Support.

JFrog Platform Deployments (JPDs) Clocks Must be Synchronized

If the federated repository artifacts are simultaneously updated on two (or more) member federated repositories, the update that is registered last will overwrite the other update(s). Therefore, to ensure consistent, predictable and trackable behavior of the system as a whole, the server clocks of all the machines running a federated members must be synchronized.

Configure the Custom Base URL in the General System of the Administration module. The Base URL supports detecting member JFrog Deployments (JPDs) in your organization.

Establish Trust Between the JPDs

You will need to establish trust between the JPDs in your Federated Repository. The options for enabling trust depend on whether you have a Cloud or a Self-hosted deployment.

Cloud Customers

JFrog Enterprise/ Enterprise+ SaaS customers: 

• Set up a binding token to create bi-directional trust between the JPDs.

Self-hosted Customers

Choose from one of the following options:

• Set up bindings between relevant JPDs: Using binding tokens enable admins to create trust between managed JFrog Platform Deployments (JPDs) once the JPDs have been added to the Mission Control instance, thus simplifying the setup across JPDs
• Enable a Circle of Trust: This option is intended for customers who are interested in providing full access to the other JPDs, access which is granted automatically once the Circle of Trust is implemented.

Setting Up a Federated Repository

Using the UI

1. From the Administration module, navigate to Repositories | Repositories.
   

   JFrog Cloud New Interface (Beta)

   Go to Artifactory, and select Repositories. To learn more, click here. 

2. Click Add Repository and select Federated Repository.
   
   
3. Configure the Basic and Advanced repository settings similarly to configuring a Local Repository.
   Note that it is mandatory to assign a Repository Key that will be added as the prefix to the Federated repository and displayed on all the sites.
   
   
4. In the Federation tab, proceed to add the repositories located on other JFrog Platform Deployments (JPDs) to the Federation.
   
   
5. Click Add Repository.
   
   
6. In the Add Repositories dialog, add the repositories to the Federation using one of these methods:
   
   • Deployments: If you have JFrog Mission Control installed, the repositories on the remote JFrog Platform Deployments will automatically be populated. If the repository with the identical name doesn't exist on the remote JFrog Platform Deployment, proceed to click Create New to duplicate this repository.

   • URL: Manually add a predefined URL path to the repository. If the repository does not exist on the remote JFrog Platform Deployment, it will be created automatically according to the following syntax.
     

     <BASE_URL>/artifactory/<repository_name> //For example, http://<ip address>:8082/artifactory/fed117

     JFrog Platform Deployments will appear only if you have set up bindings or established a Circle of Trust. See Establish Trust between the JPDs.

7. Click Save.
   You will be routed back to the Basic tab.
8. Click Create Federated Repository.
9. Click Save.

Using the REST API

The dedicated Federated Repository Configuration JSON file is application/vnd.org.jfrog.artifactory.repositories.FederatedRepositoryConfiguration+json

The following repository REST APIs support working with Federated repositories:

• Get Repositories
• Create Repository
• Update Repository Replication Configuration
• Delete Repository

Pausing/ Resuming Federated Synchronization

You can pause and resume synchronization of the artifacts from your source Federated repository to the other member repositories.

Removing Repositories from the Federation

Admins on any of the federated members can remove themselves and other members from the Federation. Note that you can delete the initial repository and the Federation will continue to function between the remaining federated members.

1. From the Administration module, click Repositories | Repositories and click the Federated tab to view a list of Federations.
   

   JFrog Cloud New Interface (Beta)

   Go to Artifactory, and select Repositories. To learn more, click here. 

2. Select the Federation repository from the list and click the Federation tab.
3. Click the x located on the top right of the repository.

Converting a Local Repository to a Federated Repository

You can convert an existing local repository to a Federated repository directly from its listing in the Local tab.

1. In the Administration module, select Repositories > Local.
   

   JFrog Cloud New Interface (Beta)

   Go to Artifactory, and select Repositories. To learn more, click here. 

2. At the end of the row for the relevant local repository, open the Actions menu and select Convert to Federated. 
   
   
   The Convert to Federated Repository dialog opens.
   
   
   
3. Click Convert.
4. Run the full sync using the Federated Repository Full Sync REST API.
   The Local repository is moved to the Federated tab.

Converting a Build-Info Repository to a Federated Repository

Build-Info repositories can be set as a federated member in a Federation.

From Artifactory 7.35, you can convert an existing Build-Info repository to a Federated repository using the Artifactory REST API.

Step 1: Convert the build-info repository to a federated repository
Step 2: Add members to the build-info federation

POST /api/federation/migrate/{buildInfoRepoName}

Sample Input

POST /api/federation/migrate/artifactory-build-info
Migration finished successfully. 

Step 2: Add member to the build-info Federation
Once you have converted the build-info repository into a federated build-info repository, proceed to add members to the federation using the Update Repository Configuration REST API

{
    "key": "projectKey-build-info",
    "rclass" : "federated",
    "members": [
        { "url": "http://<target-JPD>/artifactory/projectKey-build-info", "enabled":"true"},
        { "url": "http://<target-JPD>/artifactory/projectKey-build-info", "enabled":"true"}
  ]
}

For more information, see Repository Configuration JSON.

Working with Federated Artifacts

Once you have created the Federation, the Federated repositories are automatically displayed in your Artifact browser.

Any of the CRUD actions that you perform are automatically applied on the member Federated repositories. Users can perform actions according to their permission sets. The logic is applied according to the last action performed on the Artifact by any of the users in the Federation.

Geo Synchronized Topology Use case: Setting a Federated Base URL 

Setting an Initial Federated Repository URL

Geo Synchronized Topology requires that all Platform Deployments (Artifactory instances) be configured with the same Custom Base URL. Federated repositories must have their unique Custom Base URL in order to distinguish between the Federated members in the different Platform Deployments.
To work with Federated repositories within a Geo synchronized topology, add the federatedRepoUrlBase parameter to the Global Configuration Descriptor file, which is the the global Artifactory configuration file used to provide a default set of configuration parameters.
The following XML Tag should be added under the <systemProperties> section in the Global Configuration Descriptor.

<urlBase></urlBase> <!-- NOTE: Leave the url Base setting blank -->
<federatedRepoUrlBase>https://ARTIFACTORY_SERVER_HOSTNAME/artifactory</federatedRepoUrlBase>

For information on how to modify the Global Configuration Descriptor, see the Configuration Files Overview.

For information about changing the custom base URL or Federated base URL, see Changing the Base URL in Federated Repositories.

Migrating Federated Repositories to a Geo Synchronized Topology

From Artifactory version 7.21.13, you can migrate your existing Federated repositories to be configured with a dedicated Federated Repository URL instead of the Artifactory base URL.

1. Disconnect all the Federated repositories in either one of the sites, using one of the following methods:
• One at a time in the Federated Repository page in the UI
• Directly through the Config Descriptor file.
2. Wait for a while and then validate that all the Fed repositories are disconnected on the member site.

3. Navigate to $JFROG_HOME/artifactory/var/etc/artifactory/artifactory.config.xml and add the following XML tag.
   

   <federatedRepoUrlBase>https://<address>/artifactory</federatedRepoUrlBase>

4. Reconnect all the Federated repositories.
5. Validate that the configuration was applied in the member site.
6. Monitor the logs for related errors.
7. [Test] Upload a new file for each Repo to site 1 and download it in site 2. 
8. [Optional] Perform full synchronization if during the time that the Fed Repos were disconnected changes occurred on one of the sites (For example upload, override, change in properties, or delete actions).

Increasing the Predefined Socket Timeout for Larger Repositories

From Artifactory version 7.38.17 (Self-Hosted), you can increase the socket timeout for when syncing large repositories between federated members within a Federation.

View Federation Sync Status

From Artifactory version 7.55.1, administrators can see whether there are significant synchronization delays between the Federated repositories on the local JPD and other Federation members on remote JPDs. 

To view the Federation synchronization status, go to the Administration tab and select Monitoring > Federation Status.

The table contains the following information about each JPD:

Refresh the Federated Sync Status

In the Federated Sync Status window, click Refresh to update the information in the table.

View the List of Longest Delays in a JPD

In the Federated Sync Status table, click the row of a JPD to view a popup window containing a list of repositories (up to 10) with the longest delays.

The table contains the following information:

Monitor Federated Repositories

From Artifactory release 7.49.3, you can monitor the status of Federated repositories using a set of dedicated REST APIs. Use these APIs to get the status of the Federation for a specific repository, to get a list of Federation mirror lag times, and to get a list of unsynchronized mirrors.

Get Federated Repository Status

Returns the synchronization status of the Federation for a specific repository, including:

• The number of tasks in progress
• The number of failed tasks
• The number of various types of pending events (create, update, delete, node property, error)
• Server lag time (elapsed time since last event not handled)
• The number of fully (binary and metadata) and artificially (metadata only) replicated artifacts

Get Federation Mirror Lag Time

Returns the elapsed time since the last event that was not handled on each Federation mirror for all repositories. This API can be used to gather lag statistics at regular intervals. When a repository with significant lag is detected, the Get Federated Repository Status API can be used to get additional details.

Get Unavailable Mirrors

Returns a list of unsynchronized Federated mirrors from all repositories. Mirrors are unsynchronized in the following circumstances:

• When the mirrors are first added to the Federation.
• During repository migration from local to Federated.
• When a certain error threshold is crossed.

For detailed information about each API, see Artifactory REST API.

Troubleshooting Federated Member Out-of-Sync Notifications

If any of the Federated members in your Federation are out of synch, the system triggers an out-of-sync notification. This can occur due to network failures or changes to your federated members.
You can manually push the updated configuration to the target Federated member, using one of these methods:

• Run the Synchronize Federated Member's Configuration REST API.
  

  POST http://localhost: port/artifactory/api/federation/configSync/<repositoryKey>

  

• Directly in the JFrog Platform UI:
  1. In the JFrog Platform, navigate to the Administration module | Repositories | Federated.
  2. Select the relevant Federated member, and from the Actions list (located at the end of the list) select Push Configuration.