Fully Reproducible Builds with CircleCI and Artifactory
Most of you are probably familiar with the following scenario. You build an NPM package using CircleCI using dependencies that are fetched from the public NPM registry. Suddenly, you discover that your package contains a major bug! You need to quickly rollback to the previous version, but discover that one of the dependencies from this previous version got unpublished from npmjs by the package owner, and now your build is no longer reproducible. What do you do?
This is exactly what happened with NPM gate. An open-source contributor unpublished an open-source module which broke thousands of builds which used his modules as dependencies.
CircleCI is a cloud-based continuous integration and delivery platform. It’s free for open source repositories, and supports most languages. It enables developers to quickly test their changes by building their software without having to manage a CI server. However, one of the main challenges when using this solution, is storing build artifacts for future distribution.
This blog post describes how you can continue to use CircleCI to create fully reproducible builds without worrying about losing dependencies, by using a private JFrog Artifactory repository manager to store your build’s artifacts and build information.
CircleCI fetches NPM dependencies and publishes created NPM package from/to the public NPM registry.
When incorporating JFrog Artifactory into your build process, CircleCI fetches NPM dependencies from Artifactory and caches them. It then publishes created NPM package to Artifactory, along with build information. The published packages along with their build information is then available to all developers.
Configuring CircleCI to push build artifacts to JFrog Artifactory
Let’s take a closer look at how this actually works. The example below describes how to easily configure CircleCI to push Maven builds as artifacts to Artifactory using JFrog CLI.
Step 1: Clone the sample repository
Create a repository in GitHub/Bitbucket.
Clone this sample GitHub project into your git repository.
Step 2: Create the CircleCI configuration files
Here, we use JFrog CLI to upload created artifacts and build information to Artifactory.
machine: java: version: openjdk7 dependencies: override: - mvn --fail-never dependency:go-offline || true pre: # Install JFrog CLI - wget https://dl.bintray.com/jfrog/jfrog-cli-go/1.7.1/jfrog-cli-linux-amd64/jfrog - chmod +x jfrog compile: override: - mvn clean install # Configure JFrog CLI with parameters of your Artifactory instance - ./jfrog rt config --url $ARTIFACTORY_URL --user $ARTIFACTORY_USER --apikey $ARTIFACTORY_PASSWORD # Upload artifacts to Artifactory using JFrog CLI - ./jfrog rt u "multi*/*.jar" circleci-generic-local --build-name=circleci-generic-artifactory --build-number=$CIRCLE_BUILD_NUM --flat=false - ./jfrog rt u "multi*/*.war" circleci-generic-local --build-name=circleci-generic-artifactory --build-number=$CIRCLE_BUILD_NUM --flat=false # Collect environment variables and attach them to a build. - ./jfrog rt bce circleci-generic-artifactory $CIRCLE_BUILD_NUM # Publish build info to Artifactory. - ./jfrog rt bp circleci-generic-artifactory $CIRCLE_BUILD_NUM
Step 3: Configure Artifactory credentials
Enable your project build in CircleCI by adding the ARTIFACTORY_URL, ARTIFACTORY_USERNAME and ARTIFACTORY_PASSWORD environment variables to your build settings.
For example:
ARTIFACTORY_URL=https://artifactory.mycompany.com/artifactory
ARTIFACTORY_USERNAME=admin
ARTIFACTORY_PASSWORD=password
Step 4: See your published artifacts in JFrog Artifactory
Once you run a build, you should be able to see the published artifacts available in Artifactory.
Step 5: See your published build information
You can view details of the build in Artifactory’s Build Browser. Click any build to drill down and see all the build info that JFrog Artifactory captures.
That’s it! You’re done.
Reproducible builds for different package types
Artifactory supports most package formats compatible with CircleCI which means you can get detailed build information for a variety of package types such as:
The Generic Artifacts solution will work for any different package format.
Using CircleCI for continuous integration is great and offers all the benefits of a cloud service. Adding Artifactory completes the picture giving you a place to store your builds, and with the exhaustive build information that Artifactory generates, you can now have fully reproducible builds with CircleCI.
Visit the JFrog Artifactory User Guide to learn more about Build Integration.
Get started with JFrog Artifactory and start saving your build artifacts today.
