How to configure Amazon S3 as Filestore for Artifactory (a sample walk-through)?

SAMPLE WALK THROUGH: How to configure Amazon AWS S3 as Filestore for Artifactory

(Last updated based on Artifactory version: 4.2.2)

==STEP1: Create or use an existing IAM user==

1.. Create IAM USER:

AWS Console => Identity & Access Management => Users => Create New Users 

==STEP2: Configure the IAM user==

1.. Give the IAM user a proper permission to read/write to the bucket

AWS Console => Identity & Access Management => Users => Select a user => Permissions => Attach Policy (e.g. AmazonS3FullAccess ) or create your own policy

For example: AmazonS3FullAccess, 

{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": "s3:*",            "Resource": "*"        }    ]}

 

See http://docs.aws.amazon.com/AmazonS3/latest/dev/access-control-overview.html for more details. Below is a section from this link to help you understand available permissions for s3:

For more information about IAM, go to AWS Identity and Access Management (IAM) product detail page.
The following is an example of a user policy. You cannot grant anonymous permissions in an IAM user policy, because the policy is attached to a user. The example policy allows the associated user that it’s attached to, to perform six different Amazon S3 actions on a bucket and the objects in it. You can attach this policy to a specific IAM user, group, or role. 

{    "Statement": [        {            "Effect":"Allow",            "Action": [                 "s3:PutObject",                "s3:GetObject",                "s3:DeleteObject",                "s3:ListAllMyBuckets",                "s3:GetBucketLocation",                "s3:ListBucket"            ],            "Resource":"arn:aws:s3:::examplebucket/*"        }    ]}

 

2.. Select the user and Create Access Key

AWS Console => Identity & Access Management => Users => Select a user => Security Credentials => Create Access Key => Note down or download Access Key ID and Secret Access Key

==STEP3: Configure or select a bucket in AWS==

1.. Create or select an existing bucket

AWS Console => S3 => Select your bucket

2.. Create a new folder that will be used by Artifactory to store binaries on (e.g. filestore)

AWS Console => S3 => Select your bucket => Create Folder

3.. Create Permission for IAM user

AWS Console => S3 => Select your bucket => Properties => Add Grantee “Any Authenticated AWS User” or anything Grantee that contains IAM users => Give List, Update/Delete (prior to Artifactory 4.3.0, ‘View Permissions’ is also required) => Save

For example,

          

==STEP4: Configure Artifactory==

Use this document to configure Artifactory: https://www.jfrog.com/confluence/display/RTF/S3+Object+Storage

For example, for the screenshot above, storage.properties file should look like :

type=derbyurl=jdbc:derby:{db.home};create=truedriver=org.apache.derby.jdbc.EmbeddedDriver## Determines where the actual artifacts binaries are stored. Available options:## filesystem - binaries are stored in the filesystem (recommended, default)## fullDb     - binaries are stored as blobs in the db, filesystem is used for caching## cachedFS   - binaries are stored in the filesystem, but a front cache (with faster access) is added## IMPORTANT NOTE: This property should not be change after the initial setup. To change binaries storage you have to export and import# S3 Configuration - IRELANDbinary.provider.type=S3binary.provider.s3.identity=<IAM User's Access Key ID>binary.provider.s3.credential=<IAM User's Secret Access Key>binary.provider.s3.endpoint=https://s3-eu-west-1.amazonaws.combinary.provider.s3.bucket.name=joshua-s3-artifactory-irelandbinary.provider.s3.bucket.path=filestore# S3 Configuration - US WEST -COMMENTED OUT!!!!!!!!#binary.provider.type=S3#binary.provider.s3.identity=<IAM User's Access Key ID>#binary.provider.s3.credential=<IAM User's Secret Access Key>#binary.provider.s3.endpoint=https://s3-us-west-1.amazonaws.com#binary.provider.s3.bucket.name=joshua-s3-artifactory-USW#binary.provider.s3.bucket.path=filestore## Determines the maximum filesystem cache size in bytes when using binary provider type fullDb or cachedFS. Default is 5GB## Supported units are TB (terabytes), GB (gigabytes), MB (megabytes) and KB (kilobytes)#binary.provider.cache.maxSize=5GB

  

==TROUBLESHOOTING==

If your Artifactory does not start because of a wrong configuration of S3, then you may enable following loggers at $ARTIFACTORY_HOME/etc/logback.xml to debug the issue. The logs get updated by itself and do not require restarting Artifactory.

    <logger name="org.artifactory.addon.filestore.s3">        <level value="DEBUG"/>    </logger>    <logger name="org.artifactory.addon.filestore.eventual">        <level value="DEBUG"/>    </logger>    <logger name="org.artifactory.addon.filestore.eventual.task">        <level value="DEBUG"/>    </logger>    <logger name="org.artifactory.storage.binstore.service.providers.RetryBinaryProvider">        <level value="DEBUG"/>    </logger>