Can I use Amazon’s EFS with Artifactory HA?

It is possible to use Amazon's EFS with Artifactory HA and have it act as the NFS. There is, however, a limitation on the usage of this resource that Amazon calls a BurstCreditBalance. This Burst Credit Balance limits the total download/upload of this resource and once that balance is exceeded, the resource becomes slow to the point that it is not usable with Artifactory. The limit is not a fixed number, but rather depends on a storage/activity ratio, and can be monitored as per Amazon's instructions.

In order to reduce the activity on this resource, and to avoid exceeding the Burst Credit Balance, it is important to configure Artifactory so that it uses a local cache. This local cache will store artifacts on each individual node and serve them directly to the client instead of having to pull them from EFS. This means that each node may have a duplicate cache (since any node can serve any request) but this greatly reduces the reliance on the EFS. For example, if a file were downloaded 1000 times without a local cache, the EFS would have the download activity of 1000 * filesize but with local cache enabled, it may end up being just 2 * filesize. This would be even less if S3 were being used as the binary store.

It is important to note that when an Artifactory HA node is being run on Amazon's AWS, that the directory that is being used to store this cache is not ephemeral. This would typically mean using an EBS that is resistant to the ephemeral nature of AWS instances. 

Adding a local cache

When adding a local cache, the relevant properties should be set in $CLUSTER_HOME/ha-etc/storage.properties or in binarystore.xml if usingAdvanced Storage options. After the changes are made, the nodes must be restarted before they use the new values.

For example (using storage.properties):

binary.provider.cache.dir=/var/opt/jfrog/artifactory/data/cache

binary.provider.cache.maxSize=30GB

Using S3 with EFS


If S3 is being used as the binary store in conjunction with a local cache, the EFS will then only be used for configuration and for all uploads, but not for downloads. This will greatly reduce the activity on the EFS but keep in mind that downloads would still suffer if the Burst Credit Balance is exceeded, though it is less likely to occur.


Symptoms of exceeding the Burst Credit Balance


Symptoms of exceeding the Burst Credit Balance include ping delays (api/system/ping) and very slow downloads/uploads. This will happen for downloads even when using S3 as the binary store with local-cache since the overall system suffers. This may show up as timeout errors or broken connections in your Artifactory log.