Overview

Pipelines provides facilities to manage the execution of a Jenkins job from your pipeline, passing information into and receiving output from the job.

For organizations that have a significant legacy investment in Jenkins, it may not be practical or cost-effective to convert many existing CI/CD worfklows to Pipelines. These facilities provide you with a way to interoperate between your existing Jenkins workflows and Pipelines.

For example:

  • a Jenkins build job may create a base Docker image and publish it to Artifactory, then send information from that job to the subsequent steps in your Pipeline to build variant Docker images for testing and release
  • a Pipelines pipeline may publish a build to Artifactory, then send the build info as a parameter to a Jenkins job to deliver it for staging
  • a Jenkins build job may push a Docker image to Artifactory, trigger Pipelines to build supplemental images, then Pipelines may trigger Jenkins to complete the build of variants

This facility presumes the use of Artifactory repositories to store and manage all builds and metadata.


This facility requires that Jenkins Artifactory Plugin be installed with your Jenkins installation.

Running a Jenkins Job From Pipelines

A pipeline in Pipelines can transfer control to a Jenkins job and then is returned to execution after the Jenkins job completes. 

In summary, this is the sequence of events:

  1. In Pipelines, a pipeline executes a Jenkins step to invoke a Jenkins job through the Jenkins Server Integration
  2. When the Jenkins job completes, it returns control to Pipelines and optionally updates Pipelines resources by calling a function in the Jenkins Artifactory Plugin
  3. In Pipelines, the pipeline resumes execution with any updated resources

Jenkins Server Integration

To enable Pipelines to pass control from and return to Jenkins, an administrator user must first add a Jenkins Server Integration The Jenkins Server integration specifies the Jenkins URL, as well as the username/token credentials for a valid Jenkins user. In addition, the Jenkins Server Integration must generate a bearer token to authenticate messages sent from Jenkins to Pipelines through the Jenkins Artifactory Plugin.

The Jenkins Artifactory plugin must be configured with the Callback URL, and the generated bearer token must be added to the plugin's credentials. See the Jenkins Server Integration for details.

Pipelines DSL

If the Jenkins job will update Pipelines resources, you must declare those resources in your Pipelines DSL. For example, a PropertyBag resource can be used to signal Jenkins job results:

resources:
  - name: app_test_results
    type: PropertyBag
    configuration:
      passing: 0
      failing: 0


To initiate execution of a Jenkins job from within your pipeline, use the Jenkins native step. You must specify the Jenkins Server integration in your integrations block. If the Jenkins job updates Pipelines resources, you must specify them in the outputResources block.

- name: test_app
  type: Jenkins
  configuration:
    jenkinsJobName: basic-api
    inputResources:
      - name: app_docker
    buildParameters:
      imageName: ${res_app_docker_imageName}
      imageTag: ${res_app_docker_imageTag}
    integrations:
      - name: myJenkins
    outputResources:
      - name: app_test_results

When loaded into Pipelines, the pipeline diagram shows the Jenkins step with a Docker image as an input resource, and the PropertyBag updated by the Jenkins job as an output resource.


Sending Build Parameters to Jenkins

The Jenkins step can pass parameters to the Jenkins job through the buildParameters property. These values can be fixed or you can use Pipelines environment variables to send dynamic values.

In this example, we specify an Image resource called "app_docker" in the inputResources  of our Jenkins Step. Our buildParameters property references the resource's environment variables.

buildParameters:
  imageName: ${res_app_docker_imageName}
  imageTag: ${res_app_docker_imageTag}
  environment: "test"

Run state environment variables are not available to buildParameters.

Jenkins Build Job

The Jenkins job must be prepared to recognize incoming buildParameters from Pipelines. When our Jenkins job executes, our example build parameters are received from Pipelines and available for use.

steps {
  echo "environment: ${params.environment}"
  echo "Running tests on image ${params.imageName}:${params.imageTag}"
}

At the end of your Jenkins build job, call the plugin's function jfPipelines() to signal its completion and return control back to Pipelines.

jfPipelines()

Your build job may optionally pass information back to Pipelines by updating the properties of Pipelines resources. To do so, declare each resource by name with the new property values in you call to jfPipelines(). 

For example, the following call references a Pipelines PropertyBag resource we have called app_test_results and provides new values for two of its properties:

jfPipelines(
  outputResources: """[
    {
      "name": "app_test_results",
      "content": {
        "passing": 1234,
        "failing": 0
      }
    }
  ]"""
)

As noted above, these Pipelines resources must be declared as outputResources of the invoking Jenkins step.

Triggering Pipelines from Jenkins Jobs (Deprecated)

The method described in this section has been deprecated as of release 1.6.0. It is available only for use with prior releases of Pipelines.

A pipeline in Pipelines can be triggered from a Jenkins build job through the build Jenkins publishes to Artifactory. 

In summary, this is how Jenkins triggers Pipelines:

  1. A Jenkins job successfully completes a build and publishes it to Artifactory.
  2. The Jenkins job messages Pipelines through a REST API to create a new version of the BuildInfo resource.
  3. In Pipelines, the change of the BuildInfo resource triggers execution of the pipeline.

The functioning of this sequence requires that the BuildInfo resource must also specify a Jenkins integration. 

Jenkins Integration

To enable Jenkins to trigger Pipelines, an administrator user must first add a Jenkins Integration The Jenkins integration specifies the Jenkins URL, as well as the username/password credentials for a valid user in JFrog Platform. These user credentials will be used to authenticate messages sent from Jenkins to Pipelines.


Jenkins Build Job

Once the Jenkins integration has been added to Pipelines, the UI view for the integration provides the callback URL and curl command lines. These provide you with the usage format of the command that Jenkins will need in order to message Pipelines. This can be viewed by all users, not just administrators.

In your Jenkins build job, add the curl command to POST using basic authorization to the Pipelines REST API  on completion of the build. You will need to add the -d option to specify the affected BuildInfo.

  post {
        success {
            script {
                sh "curl -XPOST -H \"Authorization: Basic am9obi1kb2U6cGFzc3dvcmQ=\" \"http://pipelines.mycompany.info:8082/pipelines/api/v1/projectIntegrations/627/hook\" -d '{\"buildName\":\"$JOB_NAME\",\"buildNumber\":\"$BUILD_NUMBER\",\"buildInfoResourceName\":\"jenkinsBuildInfo\"}' -H \"Content-Type: application/json\""            
            }    
        }
    }

The buildInfoResourceName parameter must specify the friendly name of the BuildInfo resource in the Pipelines DSL that represents the Jenkins-created build. In the example above, it names the jenkinsBuildInfo  resource declared in the example pipeline in the section that follows.

Pipelines DSL

The BuildInfo resource referenced by Jenkins must be declared in the Pipelines DSL. It must also be connected to the Jenkins integration by specifying it in the resource's externalCI property.

resources:
  - name: jenkinsBuildInfo
    type: BuildInfo
    configuration:
      sourceArtifactory: myArtifactory		# Replace with your Artifactory integration name
      buildName: sample-build				# Replace with your build name
      buildNumber: 1
      externalCI: MyJenkins					# Replace with your Jenkins integration name


When the BuildInfo resource is specified in the step's inputResources, then the update of that build information by the Jenkins build job will trigger the step.

pipelines:
  - name: pipelines_trigger
    steps:
      - name: start_from_jenkins
        type: Bash
        configuration:
          inputResources:
            - name: jenkinsBuildInfo
        execution:
          onExecute:
            - echo 'Jenkins job triggered Pipelines'


When a pipeline run has been triggered by Jenkins in this manner, the BuildInfo resource will be populated with additional information about the Jenkins agent. These properties can be viewed in the run history:




Triggering Jenkins Jobs from Pipelines (Deprecated)

The method described in this section has been deprecated as of release 1.6.0. It is avaialable only for use with prior releases of Pipelines.

A pipeline in Pipelines can trigger execution of a Jenkins build job through an outgoing webhook.

In summary, this is how Pipelines triggers Jenkins:

  1. Pipelines sends an outgoing webhook message to Jenkins.
  2. Jenkins receives the webhook and begins execution.

The functioning of this sequence requires that Jenkins job be configured to receive and act on a webhook. 

Preparing Jenkins

In your Jenkins job that you want Pipelines to trigger, add the . You should configure the webhook plugin with

  • a token that will restrict triggering only to requests that reference that token
  • if needed, any additional parameters you intend to send from Pipelines

Preparing Pipelines

An administrator user must create an Outgoing Webhook Integration that specifies the Jenkins URL and user credentials.

You should specify the URL to address the Generic Webhook Plugin on your Jenkins installation, with basic  authorization,. You must provide a valid Jenkins username an password.

Pipelines DSL

An OutgoingWebhook resource must be declared in your Pipelines DSL, and specify the Outgoing Webhook integration. It must  also declare the token parameter that the Jenkins plugin is expecting.

resources:
  - name: myJenkinsTrigger
    type: OutgoingWebhook
    configuration:
      webhookName: MyJenkinsWebhook 		# relpace with your Outgoing Webhook integration name
      parameters: token=PIPELINES1          # replace with your token value


When you specify the OutgoingWebHook resource in the step's outputResources property, the step will send the webhook to Jenkins. This will trigger execution of the Jenkins job.

pipelines:
  - name: jenkins_trigger
    steps:
      - name: step_to_jenkins
        type: Bash
        configuration:
          outputResources:
            - name: myJenkinsTrigger         # the outgoing webhook to trigger Jenkins
        execution:
          onExecute:
            - echo 'Triggered Jenkins job'