Have a question? Want to report an issue? Contact JFrog support

Skip to end of metadata
Go to start of metadata

Introduction

The Pipeline Jenkins Plugin simplifies building a continuous delivery pipeline with Jenkins by creating a script that defines the steps of your build. For those not familiar with Jenkins Pipeline, please refer to the Pipeline Tutorial or the Getting Started With Pipeline documentation.

The Jenkins Artifactory Plugin has been extended to support Artifactory operations as part of the Pipeline script DSL. You have the added option of downloading dependencies, uploading artifacts, and publishing build-info to Artifactory from a Pipeline script.

Using the Artifactory DSL

Creating an Artifactory Server Instance

To upload or download files to and from your Artifactory server, you need to create an Artifactory server instance in your Pipeline script.

If your Artifactory server is already defined in Jenkins, you only need its server ID which can be obtained  under Manage | Configure System.

Then, to create your Artifactory server instance, add the following line to your script:

def server = Artifactory.server 'my-server-id'

If your Artifactory is not defined in Jenkins you can still create it as follows:

def server = Artifactory.newServer url: 'artifactory-url', username: 'username', password: 'password'

You can also user Jenkins Credential ID instead of username and password:

def server = Artifactory.newServer url: 'artifactory-url', credentialsId: 'ccrreeddeennttiiaall'

You can modify the server object using the following methods:

server bypassProxy = true
// If you're using username and password:
server username = 'new-user-name'
server password = 'new-password'
// If you're using Credentials ID:
server credentialsId = 'ccrreeddeennttiiaall'

Use variables

We recommend using variables rather than plain text to specify the Artifactory server details.

Page contents


Uploading and Downloading Files

To upload or download files you first need to create a spec which is a JSON file that specifies which files should be uploaded or downloaded and the target path.

For example:

def downloadSpec = """{
 "files": [
  {
      "pattern": "bazinga-repo/*.zip",
      "target": "bazinga/"
    }
 ]
}"""

The above spec specifies that all ZIP files in the bazinga-repo Artifactory repository should be downloaded into the bazinga directory on your Jenkins agent file system.

"files" is an array

Since the "files" element is an array, you can specify several patterns and corresponding targets in a single download spec.

To download the files, add the following line to your script:

server.download(downloadSpec)

Uploading files is very similar. The following example uploads all ZIP files that include froggy in their names into the froggy-files folder in the bazinga-repo Artifactory repository.

def uploadSpec = """{
  "files": [
    {
      "pattern": "bazinga/*froggy*.zip",
      "target": "bazinga-repo/froggy-files/"
    }
 ]
}"""
server.upload(uploadSpec)

You can read about using File Specs for downloading and uploading files here.

Publishing Build-Info to Artifactory

Both the download and upload methods return a build-info object which can be published to Artifactory as shown in the following examples:

def buildInfo1 = server.download(downloadSpec)
def buildInfo2 = server.upload(uploadSpec)
buildInfo1.append(buildInfo2)
server.publishBuildInfo(buildInfo1)
def buildInfo = Artifactory.newBuildInfo()
server.download(artifactoryDownloadDsl, buildInfo)
server.upload(artifactoryUploadDsl, buildInfo)
server.publishBuildInfo(buildInfo)

Capturing Environment Variables

To set the Build-Info object to automatically capture environment variables while downloading and uploading files, add the following to your script:

def buildInfo = Artifactory.newBuildInfo()
buildInfo.env.capture = true

By default, environment variables named "password", "secret", or "key" are excluded and will not be published to Artifactory.

You can add more include/exclude patterns as follows:

def buildInfo = Artifactory.newBuildInfo()
buildInfo.env.filter.addInclude("*a*")
buildInfo.env.filter.addExclude("DONT_COLLECT*")

Here's how you reset to the include/exclude patterns default values:

buildInfo.env.filter.reset()

You can also completely clear the include/exclude patterns:

buildInfo.env.filter.clear()

To collect environment variables at any point in the script, use:

buildInfo.env.collect()

You can get the value of an environment variable collected as follows:

value = buildInfo.env.vars['env-var-name']

Triggering Build Retention

To trigger build retention when publishing build-info to Artifactory, use the following method:

buildInfo.retention maxBuilds: 10
buildInfo.retention maxDays: 7
// deleteBuildArtifacts is false by default.

buildInfo.retention maxBuilds: 10, maxDays: 7, doNotDiscardBuilds: ["3", "4"], deleteBuildArtifacts: true

Promoting Builds in Artifactory

To promote a build between repositories in Artifactory, define the promotion parameters in a promotionConfig object and promote that. For example:

    def promotionConfig = [
        // Mandatory parameters
        'buildName'          : buildInfo.name,
        'buildNumber'        : buildInfo.number,
        'targetRepo'         : 'libs-release-local',

        // Optional parameters
        'comment'            : 'this is the promotion comment',
        'sourceRepo'         : 'libs-snapshot-local',
        'status'             : 'Released',
        'includeDependencies': true,
        'copy'               : true,
        // 'failFast' is true by default.
        // Set it to false, if you don't want the promotion to abort upon receiving the first error.
        'failFast'           : true
    ]

    // Promote build
    server.promote promotionConfig

Allowing Interactive Promotion for Published Builds

The 'Promoting Builds in Artifactory' section in this article describes how your Pipeline script can promote builds in Artifactory. In some cases however, you'd like the build promotion to be performed after the build finished. You can configure your Pipeline job to expose some or all the builds it publishes to Artifactory, so that they can be later promoted interactively using a GUI. Here's how the Interactive Promotions looks like:

 

When the build finishes, the promotion window will be accessible by clicking on the promotion icon, next to the build run.

Here's how you do this.

First you need to create a 'promotionConfig' instance, the same way it is shown in the 'Promoting Builds in Artifactory' section.

Next, you can use it, to expose a build for interactive promotion as follows:

Artifactory.addInteractivePromotion server: server, promotionConfig: promotionConfig, displayName: "Promote me please"

You can add as many builds as you like, by using the method multiple times. All the builds added will be displayed in the promotion window. 

The 'addInteractivePromotion' methods expects the following arguments:

  1. "server" is the Artifactory on which the build promotions is done. You can create the server instance as described in the beginning of this article. 
  2. "promotionConfig" includes the promotion details. The "Promoting Builds in Artifactory" section describes how to create a promotionConfig instance.
  3.  "displayName" is an optional argument. If you add it, the promotion window will display it instead of the build name and number.

Maven Builds with Artifactory

Maven builds can resolve dependencies, deploy artifacts and publish build-info to Artifactory. To run Maven builds with Artifactory from your Pipeline script, you first need to create an Artifactory server instance, as described at the beginning of this article.
Here's an example:

    def server = Artifactory.server('my-server-id')

The next step is to create an Artifactory Maven Build instance:

 

    def rtMaven = Artifactory.newMavenBuild()

Now let's define where the Maven build should download its dependencies from. Let's say you want the release dependencies to be resolved from the 'libs-release' repository and the snapshot dependencies from the 'libs-snapshot' repository. Both repositories are located on the Artifactory server instance you defined above. Here's how you define this, using the Artifactory Maven Build instance we created:

    rtMaven.resolver server: server, releaseRepo: 'libs-release', snapshotRepo: 'libs-snapshot'

Now let's define where our build artifacts should be deployed to. Once again, we define the Artifactory server and repositories on the 'rtMaven' instance:

    rtMaven.deployer server: server, releaseRepo: 'libs-release-local', snapshotRepo: 'libs-snapshot-local'

By default, all the build artifacts are deployed to Artifactory. In case you want to deploy only some artifacts, you can filter them based on their names, using the 'addInclude' method. In the following example, we are deploying only artifacts with names that start with 'frog'

    rtMaven.deployer.artifactDeploymentPatterns.addInclude("frog*")

You can also exclude artifacts from being deployed. In the following example, we are deploying all artifacts, except for those that are zip files:

    rtMaven.deployer.artifactDeploymentPatterns.addExclude("*.zip")

And to make things more interesting, you can combine both methods. For example, to deploy all artifacts with names that start with 'frog', but are not zip files, do the following:

    rtMaven.deployer.artifactDeploymentPatterns.addInclude("frog*").addExclude("*.zip")

If you'd like to add custom properties to the deployed artifacts, you can do that as follows:

    rtMaven.deployer.addProperty("status", "in-qa").addProperty("compatibility", "1", "2", "3")

In some cases, you want to disable artifacts deployment to Artifactory or make the deployment conditional. Here's how you do it:

    rtMaven.deployer.deployArtifacts = false

To select a Maven installation for our build, we should define a Maven Tool through Jenkins Manage, and then, set the tool name as follows:

    rtMaven.tool = 'maven tool name'

Here's how you define Maven options for your build:

    rtMaven.opts = '-Xms1024m -Xmx4096m'

In case you'd like Maven to use a different JDK than your build agent's default, no problem.
Simply set the JAVA_HOME environment variable to the desired JDK path (the path to the directory above the bin directory, which includes the java executable).
Here's you do it:

    env.JAVA_HOME = 'path to JDK'


OK, we're ready to run our build. Here's how we define the pom file path (relative to the workspace) and the Maven goals. The deployment to Artifactory is performed during the 'install' phase:

    def buildInfo = rtMaven.run pom: 'maven-example/pom.xml', goals: 'clean install'

The above method runs the Maven build. At the beginning of the build, dependencies are resolved from Artifactory and at the end of it, artifacts are deployed to Artifactory.

By default, Maven uses the local Maven repository inside the .m2 directory under the user home. In case you'd like Maven to create the local repository in your job's workspace, add the -Dmaven.repo.local=.m2 system property to the goals value as shown here:

def buildInfo = rtMaven.run pom: 'maven-example/pom.xml', goals: 'clean install -Dmaven.repo.local=.m2'

What about the build information?
The build information has not yet been published to Artifactory, but it is stored locally in the 'buildInfo' instance returned by the 'run' method. You can now publish it to Artifactory as follows:

    server.publishBuildInfo buildInfo

You can also merge multiple buildInfo instances into one buildInfo instance and publish it to Artifactory as one build, as described in the 'Publishing Build-Info to Artifactory' section in this article.

Gradle Builds with Artifactory

Gradle builds can resolve dependencies, deploy artifacts and publish build-info to Artifactory. To run Gradle builds with Artifactory from your Pipeline script, you first need to create an Artifactory server instance, as described at the beginning of this article.
Here's an example:

    def server = Artifactory.server 'my-server-id'

The next step is to create an Artifactory Gradle Build instance:

    def rtGradle = Artifactory.newGradleBuild()

Now let's define where the Gradle build should download its dependencies from. Let's say you want the dependencies to be resolved from the 'libs-release' repository, located on the Artifactory server instance you defined above. Here's how you define this, using the Artifactory Gradle Build instance we created:

    rtGradle.resolver server: server, repo: 'libs-release'

Now let's define where our build artifacts should be deployed to. Once again, we define the Artifactory server and repositories on the 'rtGradle' instance:

    rtGradle.deployer server: server, repo: 'libs-release-local'

By default, all the build artifacts are deployed to Artifactory. In case you want to deploy only some artifacts, you can filter them based on their names, using the 'addInclude' method. In the following example, we are deploying only artifacts with names that start with 'frog'

    rtGradle.deployer.artifactDeploymentPatterns.addInclude("frog*")

You can also exclude artifacts from being deployed. In the following example, we are deploying all artifacts, except for those that are zip files:

    rtGradle.deployer.artifactDeploymentPatterns.addExclude("*.zip")

And to make things more interesting, you can combine both methods. For example, to deploy all artifacts with names that start with 'frog', but are not zip files, do the following:

    rtGradle.deployer.artifactDeploymentPatterns.addInclude("frog*").addExclude("*.zip")

If you'd like to add custom properties to the deployed artifacts, you can do that as follows:

    rtGradle.deployer.addProperty("status", "in-qa").addProperty("compatibility", "1", "2", "3")

In some cases, you want to disable artifacts deployment to Artifactory or make the deployment conditional. Here's how you do it:

    rtGradle.deployer.deployArtifacts = false

In case the "com.jfrog.artifactory" Gradle Plugin is already applied in your Gradle script, we need to let Jenkins know it shouldn't apply it. Here's how we do it:

    rtGradle.usesPlugin = true

In case you'd like to use the Gradle Wrapper for this build, add this:

    rtGradle.useWrapper = true

If you don't want to use the Gradle Wrapper, and set a Gradle installation instead, you should define a Gradle Tool through Jenkins Manage, and then, set the tool name as follows:

    rtGradle.tool = 'gradle tool name'

In case you'd like Maven to use a different JDK than your build agent's default, no problem.
Simply set the JAVA_HOME environment variable to the desired JDK path (the path to the directory above the bin directory, which includes the java executable).
Here's you do it:

    env.JAVA_HOME = 'path to JDK'

OK, looks like we're ready to run our Gradle build. Here's how we define the build.gradle file path (relative to the workspace) and the Gradle tasks. The deployment to Artifactory is performed as part of the 'artifactoryPublish' task:

    def buildInfo = rtGradle.run rootDir: "projectDir/", buildFile: 'build.gradle', tasks: 'clean artifactoryPublish'

The above method runs the Gradle build. At the beginning of the build, dependencies are resolved from Artifactory and at the end of it, artifacts are deployed to Artifactory.
What about the build information?
The build information has not yet been published to Artifactory, but it is stored locally in the 'buildInfo' instance returned by the 'run' method. You can now publish it to Artifactory as follows:

    server.publishBuildInfo buildInfo

You can also merge multiple buildInfo instances into one buildInfo instance and publish it to Artifactory as one build, as described in the 'Publishing Build-Info to Artifactory' section in this article.

That's it! We're all set.

The rtGradle instance supports additional configuration APIs. You can use these APIs as follows:

    def rtGradle = Artifactory.newGradleBuild()
    // Deploy Maven descriptors to Artifactory:
    rtGradle.deployer.deployMavenDescriptors = true
    // Deploy Ivy descriptors (pom.xml files) to Artifactory:
    rtGradle.deployer.deployIvyDescriptors = true

    // The following properties are used for Ivy publication configuration.
    // The values below are the defaults.

    // Set the deployed Ivy descriptor pattern:
    rtGradle.deployer.ivyPattern = '[organisation]/[module]/ivy-[revision].xml'
    // Set the deployed Ivy artifacts pattern:
    rtGradle.deployer.artifactPattern = '[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]'
    // Set mavenCompatible to true, if you wish to replace dots with slashes in the Ivy layout path, to match the Maven layout:
    rtGradle.deployer.mavenCompatible = true

Maven Release Management with Artifactory

With the Artifactory Pipeline DSL you can easily manage and run a release build for your Maven project by following the instructions below:

First, clone the code from your source control:

git url: 'https://github.com/eyalbe4/project-examples.git'

If the pom file has a snapshot version, Maven will create snapshot artifacts, because the pom files include a snapshot version (for example, 1.0.0-SNAPSHOT).
Since you want your build to create release artifacts, you need to change the version in the pom file to 1.0.0.
To do that, create a mavenDescriptor instance, and set the version to 1.0.0:

def descriptor = Artifactory.mavenDescriptor()
descriptor.version = '1.0.0'

If the project's pom file is not located at the root of the cloned project, but inside a sub-directory, add it to the mavenDescriptor instance:

descriptor.pomFile = 'maven-example/pom.xml'

In most cases, you want to verify that your release build does not include snapshot dependencies. The are two ways to do that.

The first way, is to configure the descriptor to fail the build if snapshot dependencies are found in the pom files. In this case, the job will fail before the new version is set to the pom files.
Here's how you configure this:

descriptor.failOnSnapshot = true

The second way to verify this is by using the hasSnapshots method, which returns a boolean true value if snapshot dependencies are found:

 def snapshots = descriptor.hasSnapshots()
 if (snapshots) {
     ....
 }

That's it. Using the mavenDescriptor as it is now will change the version inside the root pom file. In addition, if the project includes sub-modules with pom files, which include a version, it will change them as well.
Sometimes however, some sub-modules should use different release versions. For example, suppose there's one module whose version should change to 1.0.1, instead of 1.0.0. The other modules should still have their versions changed to 1.0.0. Here's how to do that:

descriptor.setVersion "the.group.id:the.artifact.id", "1.0.1"

The above setVersion method receives two arguments: the module name and its new release version. The module name is composed of the group ID and the artifact ID with a colon between them.
Now you can transform the pom files to include the new versions:

descriptor.transform()

The transform method changed the versions on the local pom files.
You can now build the code and deploy the release Maven artifacts to Artifactory as described in the "Maven Builds with Artifactory" section in this article. 

The next step is to commit the changes made to the pom files to the source control, and also tag the new release version in the source control repository. If you're using git, you can use the git client installed on your build agent and run a few shell commands from inside the Pipeline script to do that.
The last thing you'll probably want to do is to change the pom files version to the next development version and commit the changes. You can do that again by using a mavenDescriptor instance.

Conan Builds with Artifactory

Conan is a C/C++ Package Manager. The Artifactory Pipeline DSL includes APIs that make it easy for you to run Conan builds, using the Conan Client installed on your build agents. Here's what you need to do before you create your first Conan build job with Jenkins:

1, Install the latest Conan Client on your Jenkins build agent. Please refer to the Conan documentation for installation instructions.

2. Add the Conan Client executable to the PATH environment variable on your build agent, to make sure Jenkins is able to use the client.

3. Create a Conan repository in Artifactory as described in the Conan Repositories Artifactory documentation.

OK. Let's start coding your first Conan Pipeline script.

We'll start by creating an Artifactory server instance, as described at the beginning of this article.
Here's an example:

def server = Artifactory.server 'my-server-id'

Now let's create a Conan Client instance

def conanClient = Artifactory.newConanClient()

When creating the Conan client, you can also specify the Conan user home directory as shown below:

def conanClient = Artifactory.newConanClient userHome: "conan/my-conan-user-home"

We can now configure our new conanClient instance by adding an Artifactory repository to it. In our example, we're adding the 'conan-local' repository, located in the Artifactory server, referenced by the server instance we obtained:

String remoteName = conanClient.remote.add server: server, repo: "conan-local"

As you can see in the above example, the 'conanClient.remote.add' method returns a string variable - 'remoteName'.

What is this 'remoteName' variable? What is it for?

Well, a 'Conan remote' is a repository, which can be used to download dependencies from and upload artifacts to. When we added the 'conan-local' Artifactory repository to our Conan Client, we actually added a Conan remote. The 'remoteName' variable contains the name of the new Conan remote we added.

OK. We're ready to start running Conan commands. You'll need to be familiar with the Conan commands syntax, exposed by the Conan Client to run the commands. You can read about the commands syntax in the Conan documentation.

Let's run the first command:

def buildInfo1 = conanClient.run command: "install --build missing"

The 'conanClient.run' method returns a buildInfo instance, that we can later publish to Artifactory. If you already have a buildInfo instance, and you'd like the 'conanClient.run' method to aggregate the build information to it, you can also send the buildInfo instance to the run command as and an argument as shown below:

 

conanClient.run command: "install --build missing", buildInfo: buildInfo

The next thing we want to do is to use the Conan remote we created. For example, let's upload our artifacts to the Conan remote. Notice how we use the 'remoteName' variable we got earlier, when building the Conan command:

String command = "upload * --all -r ${remoteName} --confirm"
conanClient.run command: command, buildInfo: buildInfo

We can now publish the the buildInfo to Artifactory, as described in the 'Publishing Build-Info to Artifactory' section in this article. For example:

server.publishBuildInfo buildInfo

Docker Builds with Artifactory

The Jenkins Artifactory Plugin supports a Pipeline DSL that enables you to collect and publish build-info to Artifactory for your Docker builds. To collect the build-info, the plugin uses an internal HTTP proxy server, which captures the traffic between the Docker Daemon and your Artifactory reverse proxy. To setup your Jenkins build agents to collect build-info for your Docker builds, please refer to the setup instructions

    // Create an Artifactory server instance, as described above in this article:
    def server = Artifactory.server 'my-server-id'

    // Create an Artifactory Docker instance. The instance stores the Artifactory credentials and the Docker daemon host address:
    def rtDocker = Artifactory.docker username: 'artifactory-username', password: 'artifactory-password', host: "tcp://<daemon IP>:<daemon port>"

    // If the docker daemon host is not specified, "/var/run/dokcer.sock" is used as a default value:
    def rtDocker = Artifactory.docker username: 'artifactory-username', password: 'artifactory-password'

    // You can also use the Jenkins credentials plugin instead of username and password:
    def rtDocker = Artifactory.docker credentialsId: 'ccrreeddeennttiiaall'

    // Push a docker image to Artifactory (here we're pushing hello-world:latest). The push method also expects
    // Artifactory repository name:
    def buildInfo = rtDocker.push('<artifactory-docker-registry-url>/hello-world:latest', '<target-artifactory-repository>')

    // Publish the build-info to Artifactory:
    server.publishBuildInfo buildInfo

Scanning Builds with JFrog Xray

From version 2.9.0, Jenkins Artifactory Plugin is integrated with JFrog Xray through JFrog Artifactory allowing you to have build artifacts and dependencies scanned for vulnerabilities and other issues. If issues or vulnerabilities are found, you may choose to fail a build job or perform other actions according to the Pipeline script you write. This integration requires JFrog Artifactory v4.16 and above and JFrog Xray v1.6 and above. 

You may scan any build that has been published to Artifactory. It does not matter when the build was published, as long as it was published before triggering the scan by JFrog Xray.

The following instructions show you how to configure your Pipeline script to have a build scanned.

First, create a scanConfig instance with the build name and build number you wish to scan:

  def scanConfig = [
      'buildName'     : 'my-build-name',
      'buildNumber'   : '17'
    ]

If you're scanning a build which has already been published to Artifactory in the same job, you can use the build name and build number stored on the buildInfo instance you used to publish the build. For example:

  server.publishBuildInfo buildInfo
  def scanConfig = [
      'buildName'      : buildInfo.name,
      'buildNumber'    : buildInfo.number
    ]

Before you trigger the scan, there's one more thing you need to be aware of. By default, if the Xray scan finds vulnerabilities or issues in the build that trigger an alert, the build job will fail. If you don't want the build job to fail, you can add the 'failBuild' property to the scanConfig instance and set it to 'false' as shown here:

  def scanConfig = [
      'buildName'      : buildInfo.name,
      'buildNumber'    : buildInfo.number,
      'failBuild'      : false
    ]

OK, we're ready to initiate the scan. The scan should be initiated on the same Artifactory server instance, to which the build was published:

 def scanResult = server.xrayScan scanConfig

That's it. The build will now be scanned. If the scan is not configured to fail the build job, you can use the scanResult instance returned from the xrayScan method to see some details about the scan.
For example, to print the result to the log, you could use the following code snippet:

 echo xrayResults as String

For more details on the integration with JFrog Xray and JFrog Artifactory to scan builds for issues and vulnerabilities, please refer to CI/CD Integration in the JFrog Xray documentation.

Distributing Build Artifacts

From version 2.11.0, you can easily distribute your build artifacts to the world or to your community using Pipeline. Your artifacts are distributed to JFrog Bintray, using a Distribution Repository in Artifactory. You can read more about Distribution Repositories and the steps for setting them up here.

In order to distribute a build, it should be first published to Artifactory, as described in the "Publishing Build-Info to Artifactory" section in this article.

Once you have your Distribution Repository set up and your build is published to Artifactory, define a distributionConfig instance in your Pipeline script as shown here:

def distributionConfig = [
	// Mandatory parameters
	'buildName'             : buildInfo.name,
	'buildNumber'           : buildInfo.number,
	'targetRepo'            : 'dist-repo',
       
	// Optional parameters
	'publish'               : true, // Default: true. If true, artifacts are published when deployed to Bintray.
    'overrideExistingFiles' : false, // Default: false. If true, Artifactory overwrites builds already existing in the target path in Bintray. 
    'gpgPassphrase'         : 'passphrase', // If specified, Artifactory will GPG sign the build deployed to Bintray and apply the specified passphrase.
    'async'                 : false, // Default: false. If true, the build will be distributed asynchronously. Errors and warnings may be viewed in the Artifactory log.
    "sourceRepos"           : ["yum-local"], // An array of local repositories from which build artifacts should be collected.
    'dryRun'                : false, // Default: false. If true, distribution is only simulated. No files are actually moved.
]

OK, we're ready to distribute the build. The distribution should be donw on the same Artifactory server instance, to which the build was published:

server.distribute distributionConfig

File Spec Schema

You can read the File Spec schema here.

Examples

The Jenkins Pipeline Examples can help get you started using the Artifactory DSL in your Pipeline scripts.

 

  • No labels