This is a fairly simple use case that shows how security entities in a group of Artifactory clusters are fully synchronized.
Each of the Access services is configured to synchronize changes to all the others, and each Access service has been configured to trust all the others by being provided with their respective root certificates, so they are all in one all-encompassing circle-of-trust.
A change made to the Access service in any of the clusters will be synchronized over to all other Access services clusters.
Mesh with DR
This example shows two sets of Artifactory clusters, (Full mesh access 1 and 2) federated with a Central Access server in a full mesh topology.
In addition, the clusters and the central Access service also synchronize security entities over to an Access service serving a DR cluster.
In this configuration, should either (or both) of the Full mesh Access clusters become inoperative, the DR cluster can be activated and it will already be correctly configured with all security entities.
Once the downed cluster(s) are back in operation, you can invoke a "full broadcast" (the same one used to revive a stale service) from the DR service to bring it up-to-date with all the current security entities. At this point, you can deactivate the DR instance and you are back with your original federation configuration.
Setting Up Access Federation
Before setting up
When setting up and using Access Federation, you need to keep the following three things in mind.
Server clocks must be synchronized
If the same security entity (e.g. a user) is simultaneously updated on two (or more) federated instances of Access, the update that is registered last will overwrite the other update(s). Therefore, to ensure consistent, predictable and trackable behavior of the system as a whole, the server clocks of all the machines running a federated group of Access services must be synchronized.
Only complete entities are synchronized
When a security entity is updated, the whole entity is synchronized to the other Access services in the federated group, not just the update. For example, if a user's password is updated, the whole "user" entity is synchronized, not just its password.
To federate a group of Access services take the following steps:
- Set up authentication
- Establish a circle of trust
- Configure synchronization
- Synchronize permission entities
Setting up Authentication
You need to set up authentication by configuring the credentials file in each of the federated services to allow Access to communicate with other Access services, and also, to allow JFrog Mission Control to communicate with Access when configuring Access federation through its UI. To set up authentication, make sure the credentials file is properly configured as described under Preparing the Credentials File.
Establishing a Circle of Trust
An Access service will only receive updates to entities from a trusted source. To establish a "source" service as trusted, you need to copy the source service's root certificate found under
$ARTIFACTORY_HOME/access/etc/keys/root.crt into the receiving service's
Rename the source service's certificate
Since an Access service may receive updates from several source services, you should rename each source service's certificate with a meaningful name. For example, if the Access service at site "us-east" should be trusted by the Access service at a site "us-west", then
$ARTIFACTORY_HOME/access/etc/keys/root.crt from us-east, should be copied to
$ARTIFACTORY_HOME/access/etc/keys/trusted/us-east.crt on us-west.
In this step, you configure which security entities an Access service should synchronize to its target services. At the same time, you may optionally set a number of parameters that govern the synchronization process. The configuration parameters are specified in the YAML configuration file that is uploaded to the different Access services.
Access Federation Parameters
The following sections in the YAML configuration file govern access federation:
- "inbound": The inbound parameters map service ids from a source Access service, to service ids to which the receiving Access service is attached. To determine the service ID of an Artifactory service, use the Get Service ID REST API endpoint.
- "outbound": The outbound parameters define parameters that govern synchronization of security entities outwardly from the source service to a target service.
The format of the federation YAML file parameters is shown below with default values:
Specifies which entities should be synchronized to the target servers configured. Possible values are:
To maintain consistency in the hierarchical relationship of entities between the different Access services in a circle of trust, the following rules apply to synchronization of security entities:
LDAP, SAML, Crowd etc.
When using 3rd party authentication providers such as LDAP, SAML and Crowd, only the Users defined in these providers can be federated, NOT configurations.
Note also that for federation to work, these users must be created AFTER logging in to Artifactory.
Specific users that should be excluded from synchronization
Optional [Default: 30000]
The number of milliseconds the service should wait before consecutive attempt to synchronize the configured entity types to target servers is triggered
Optional [Default: 500]
The maximum number of entities that can be updated before an attempt to synchronize the configured entity types to target servers is triggered
Optional [Default: 168 (one week)]
The length of time a receiving server can fail to respond or respond with an error before it is considered to be "stale" and unable to be synchronized with updates. Once a server is deemed to be "stale", it can only be brought back into operation using the REST API endpoint described in Updating a Stale Server.
Optional [Default: 60000]
If an Access service is updated, and another update is subsequently synchronized over to it, the Access service will only accept the subsequent change if its timestamp is at least
For more details, please refer to Handling Synchronization Conflicts.
Optional [Default: 3000]
The length of time after synchronizing an update that an Access service will wait for acknowledgement from the receiving Access service before retrying to synchronize the update. If no acknowledgement is received within this period of time, the sending server will retry synchronizing the update to the target server the number of times specified in the
Optional [Default: 3]
The number of times an Access service will attempt to synchronize an update to a distant service until. If an acknowledgement is not received from the target server after this number of retries the synchronization is deemed to have failed.
|The logical name and URL of target Access services to which this Access service can synchronize entities|
When a service such as Mission Control, Xray or Distribution selects an Artifactory instance as its authentication provider, that service's service ID is automatically registered with the corresponding Access service.
This field maps the service IDs of services being synchronized over to this Access service. The mapping is automatically generated by Access and need not be entered manually except for a few limitations described below.
For a synchronization target, the "from" and "to" fields define the mapping of service IDs.
The from: field should specify the service type with a wildcard character. For example, for Artifactory, specify
The to: field should specify the specific service to which the source service (in the from: field) is being mapped.
Watch your whitespace
The YAML file format is sensitive to whitespace, so you need to ensure that all required whitespace characters are in place. If the YAML format is invalid, permissions will not be synchronized successfully and you will not receive error messages.
Handling Synchronization Conflicts
Depending on the synchronization topology you have configured, an Access server may receive multiple updates from different sources at short time intervals Successive updates may even conflict. For example, in a full mesh topology, an Access server may have a user's password updated to one value sync'ed over from one source, but that same user's password may be updated to a different value sync'ed over from another source. To resolve such conflicts, an Access server will finalize upon the update that has the latest timestamp. However, since synchronization of server clocks may not always be completely accurate, there could still be an inconsistency when an Access server receives conflicting updates. To overcome the possible inaccuracy in synchronization of server clocks, an Access server uses a time buffer specified in the
maximum-future-time-diff-millis parameter specified above. Access will not update an entity if the update arrives sooner than
maximum-future-time-diff-millis milliseconds from the previous update.
Consider the following example using the default value of maximum-future-time-diff-millis = 60000.
On Access Service 1, the password for User1 is changed to "abc" at time 09:02:00:000.
However, at approximately the same time, User1 is synchronized over to Access Service 1 from Access Service 2 which has set the password to be "def".
If the change synchronized over from Access Service 2 has a timestamp that is between 09:02:00:000 and 09:02:59:999, then User1's password will be "abc" because the password synchronized from Access Service 2 is ignored.
If the change synchronized over from Access Service 2 has a timestamp that is 09:03:00:000 or later, then User1's password will be "def" because the password synchronized from Access Service 2 is deemed to be later than the one set directly on Access Server 1 so it will be accepted.
Once your access services are configured with their corresponding YAML files, you are ready to synchronize data from the source to the target Access services.
To synchronize data you need to manually apply the following REST API endpoint on the source service:
Description: Synchronize data from a source to a target Access service
Notes: Requires Artifactory Enterprise Plus
Security: Requires an admin user.
Usage: PUT /api/v1/system/federation/[targetServer]/full_broadcast
The access-admin user
REST API calls on an Access service need to be authenticated using the Admin user of the Access service on which you are making the call.
Don't confuse this with any Admin user defined in Artifactory. Those can be used to make REST API calls on Artifactory. This call is made on the Access service to revive it after a long period of inactivity, not on Artifactory.
The following is an example of how you might configure a 1-way replication of permissions by synchronizing permissions settings from a source Access service in your circle of trust to a target. Here are the steps in this process:
- Obtaining the source and target service IDs
- Inbound configuration on the target
- Outbound configuration on the source
- Invoke synchronization
Obtain Service IDs
The first step is to obtain the service IDs of the Artifactory services involved in the permission synchronization relationship. Note that we need the Artifactory service IDs, although, in fact, the permissions are managed and synchronized by the corresponding Access services.
For each of the source and target Artifactory services run the the Get Service ID REST API endpoint. In this example, we are assuming you are applying this call directly on the machine that the Artifactory service is running (i.e. on localhost):
Plan and map out your synchronization
We strongly recommend having the service ID of each Artifactory cluster involved ready for use, and creating a diagram that illustrates the permission synchronization topology you are implementing while assigning a logical name for service ID. In this example, we will assign the following logical names to the source and target service IDs:
Load the following YAML file on the target Access service to create the service ID mapping as described in Uploading the YAML File. Note that since the target service is receiving permissions (and not synchronizing anything out), there is no need to include the
Don't forget to restart the Access service after placing the YAML file in the designated location.
Outbound configuration goes on the source Access instance
Note that the Outbound configuration should be applied to the source Access instance from which you are synchronizing security entities.
In this example, we want to synchronize all security entities using default values except for the following settings that we want to modify:
timeout-millis = 4000
Load the following YAML file on the source Access service as described in Uploading the YAML File to specify the above parameters and set access-2 as the synchronization target for access-1 (this example assumes the access-2 service IP address is 126.96.36.199):
Don't forget to restart the Access service after placing the YAML file in the designated location.
Once your federation is setup, you are ready to synchronize data by applying the Federation REST API endpoint described in Synchronizing Data on access-1:
Reviving a Stale Service
A "stale" service is one that has been registered as a synchronization target, however it has not responded to any attempt to synchronize data for a period of time greater than that defined in the consider-stale-hours parameter with which the source Access service was configured. Once a target service is deemed to be stale, the source service will not may any further attempts to synchronize data to it. To "revive" a stale service and resume synchronizing data you need to manually apply the "Federation" REST API endpoint described in Synchronizing Data on the source service:
Removing a Registered Synchronization Target
To remove a synchronization target that has been registered with an Access service, you need to remove it from the YAML configuration file and reload the file as described in Uploading the YAML File.
Don't forget to restart the Access service once the file is in the designated location.