Cloud customer?
Start for Free >
Upgrade in MyJFrog >
What's New in Cloud >

Search





Overview

This pipeline demonstrates the definition of a pipeline that builds a single Docker Image and pushes it to Artifactory, and then publishes BuildInfo. An example Pipelines DSL is used to show how to use integrations, resources, and steps to construct a simple, automated workflow.

This pipeline demonstrates the following:

  • Creating a GitHub and Artifactory Integration.
  • Adding a Pipeline Source.
  • Creating a GitRepo trigger, which triggers a step when the contents of the source control repository changes.
  • Using an Image resource to add a reference to a Docker image to your pipeline.
  • Using DockerBuild and DockerPush native steps to build and push a docker image.
  • Using the PublishBuildInfo native step to publish a BuildInfo to Artifactory, and the PromoteBuild native step to promote a BuildInfo.
  • Using inputResources and inputSteps to set up dependencies between steps and resources.
  • Using outputResources.

Page Contents

A successful run of the pipeline in this quickstart looks like this:


Tutorial Video



Before you Begin

Before trying this quickstart, ensure that you have:


Running This Example

Perform the steps below to build and push your Docker image:

  1. Fork the Repository

    The Pipelines DSL for this example is available in the jfrog-pipelines-docker-sample repository in the JFrog GitHub account. 

    The DSL file is a yaml file that contains the pipeline definitions. The example uses a single YAML file, pipelines.yml. This file contains the declarations for all resources and workflow steps. For a full breakup of all the resources, pipelines and steps used in the yml file, see the pipelines.yml section below.

    Fork this repository to your account or organization. This is important since you need admin access to repositories that are used as Pipeline Sources or GitRepo resources, in order to add webhooks to these repositories and listen for change events.

  2. Sign in to Artifactory

    Sign in to JFrog Platform with your Artifactory credentials.

  3. Add Integrations

    a. Go to Administration | Pipelines | Integrations to add two integrations:

    • GitHub Integration: This integration is used to add the Pipeline source, as well as the GitRepo resource defined in pipelines.yml
    • Artifactory Integration: This integration is used to authenticate with Artifactory to push the Docker image to the docker-local repository in Artifactory.

    b. Write down the names of both GitHub and Artifactory integrations as these are required for the next step. Ensure that the names are unique and easy to remember.

  4. Update pipelines.yml file

    The pipelines configuration is available in the pipelines.yml file. Edit this file in your fork of the Git repo and replace the following:

    TagDescriptionExample
    gitProviderProvide the name of the Github integration you added in the previous step. gitProvider: my_github
    pathProvide the path to your fork of this repository.path: myuser/repo-name
    registryProvide the the name of the Artifactory integration you added in the previous step.registry: art
    imageNameProvide your Docker image path and name.

    docker.artprod.mycompany.com/docker_local/dbp
    Where, docker.artprod.mycompany.com/docker_local is the image path and dbp is the image name.

    sourceArtifactoryProvide your Artifactory integration.sourceArtifactory: art

    All pipeline definitions are global across JFrog Pipelines within a Project. The names of your pipelines and resources need to be unique within the Project in JFrog Pipelines.

  5. Add Pipeline Sources

    The Pipeline Source represents the Git repo where our Pipelines definition files are stored. A pipeline source connects to the repository through an integration, which we added in the previous step.

    Follow instructions to add a Pipeline Source and point it to the pipelines.yml in your fork of the repo. This automatically adds your configuration to the platform and a pipeline is created based on your YAML. The pipelines.yml file is parsed and resources, steps, and pipelines are added as configured.

    After your pipeline source syncs successfully, navigate to Pipelines | My Pipelines in the left navbar to view the newly added pipeline. In this example, pipeline_dbp is the name of our pipeline.

    Click the name of the pipeline. This renders a real-time, interactive, diagram of the pipeline and the results of its most current run.

  6. Execute the Pipeline

    You can trigger the pipeline by committing a change to your Git repository, or by manually triggering it through the UI. The steps in the pipeline execute in sequence. Multiple steps can execute in parallel if the node pool has multiple build nodes available.

    Once the pipeline has completed, a new run is listed. 





pipelines.yml

The pipelines.yml file is made up of resources, pipelines and steps, as shown below:

Resources

This example uses the following types of resources:

GitRepo

GitRepo resource is used to connect JFrog Pipelines to a source control repository. Adding it creates a webhook to the repo so that future commits will automatically create a new version with the webhook payload.

resources
  - name: dbp_repo
    type: GitRepo
    configuration:
      gitProvider: my_github                       # <-- replace with your integration
      path: jfrog/jfrog-pipelines-docker-sample    # <-- replace with your repository name
      branches:
        include: master

Tag

Description

Required/Optional
name

dbp_repo is the name of the GitRepo resource pointing to the repository containing the Dockerfile and other source code required to build the image. 

This name is used to refer to the resource in steps, and must be unique across all repositories in your JFrog Pipelines environment.

Required

gitProvider 

The name of the GitHub integration.Required
path The path of the repository from the integration root. Required
branches
  • include -- (optional) Regular expression to include branches from the repo
  • exclude -- (optional) Regular expression to exclude branches from the repo 

The include: master tag indicates that the GitRepo resource is listening to the master branch.

Optional


Image

An Image resource is used to add a reference to a Docker image to your pipeline. An Image resource can be used as both input and output. Steps like DockerBuild will generate it as an output and steps like DockerPublish will use it as input to publish to a Docker registry. In our example, it is used as an outputResource for the DockerPush step.

resources
  - name: dbp_image
    type: Image
    configuration:
      registry: art                                # <-- replace with your artifactory integration
      sourceRepository: docker_local               # <-- required if registry is Artifactory
      imageName: docker.artprod.mycompany.com/docker_local/dbp  # < -- replace with your image path and name
      imageTag: latest
      autoPull: true

Tag

Description

Required/Optional

namedbp_image is the name that identifies the resource.Required
registry art is the name of a Docker Registry Integration.Required
sourceRepository docker_local is the name of the repository in Artifactory to which the images will be published.Required if registry is in JFrog Artifactory
imageName The file path of the Docker image, of the form imageRepo/imageName.Required
imageTagThe version tag of the initial version of the Docker image.Required
autoPull 

When true, the image is automatically pulled to the machine running the step.

Defaults to false.

Optional


BuildInfo

BuildInfo is automatically created when the DockerBuild step is used to generate packages. BuildInfo is then published to the configured Artifactory repo (sourceArtifactory: art) through the PublishBuildInfo step, by providing that resource in the DockerPush step.

resources
  - name: dbp_build_info
    type: BuildInfo
    configuration:
      sourceArtifactory: art
      buildName: dbp_build
      buildNumber: 1    

  - name: dbp_promoted_build_info
    type: BuildInfo
    configuration:
      sourceArtifactory: art
      buildName: dbp_build
      buildNumber: 1

Tag

Description

Required/Optional
name

dbp_build_info and dbp_promoted_build_info are the name of the BuildInfo resources, which is the metadata associated with the build in Artifactory.

This name is used to refer to the resource in steps, and must be unique across all repositories in your JFrog Pipelines environment.

Required

sourceArtifactory

art is the name of the Artifactory Integration.

Required
buildName

dbp_build is the name of the build to associate with the BuildInfo.

Optional
buildNumberThe number of the build to associate with the BuildInfo.Optional

Pipelines

pipeline_dbp, the pipeline definition for building and pushing the Docker image.

Steps

The pipeline_dbp pipeline contains the following native steps:

DockerBuild

The DockerBuild native step performs a build to produce a Docker image from a Dockerfile in a Git source repository.

This step builds a Docker image from a GitRepo source repository resource.  You must provide in the step configuration the name and directory of the Dockerfile that contains the command to be processed by a docker build  command, as well as the name and tag of the resulting image.  The image is built on the build node, and information about that image is stored in the run state.

steps
      - name: docker_build
        type: DockerBuild
        configuration:
          affinityGroup: dbp_group
          dockerFileLocation: .
          dockerFileName: Dockerfile
          dockerImageName: docker.artprod.mycompany.com/docker_local/dbp  # replace with your image path and name
          dockerImageTag: ${run_number}
          inputResources:
            - name: dbp_repo
          integrations:
            - name: art                            # <-- replace with your artifactory integration

Tag

Description of usage

Required/Optional

name

docker_build is the name that identifies the step.

This is the name that is used when the step is assigned as an input to the next step, docker_push.

Required
affinityGroup

Must specify an affinity group string that is the same as specified in a subsequent DockerPush step.

Both DockerBuild and DockerPush steps are assigned to the dbp_group affinityGroup so that they share the same state. If they are not, the output of the DockerBuild step won't be available to the DockerPush step. 

Optional
dockerFileLocationDirectory containing the Dockerfile or the file that has Docker build configurationRequired
dockerFileNameName of the DockerfileRequired
dockerImageName

The name of the Docker image to create. This can be set using environment variables or triggering a run using parameters.

Required
dockerImageTag

The tag for the Docker image to create. This can be set using environment variables or triggering a run using parameters.

In our example, the tag for the image is set to the pipeline's run number, which is the number of the run currently executing.

Required
inputResources 

Must specify:

  • a GitRepo resource (that contains the DockerFile)

This step accepts `dbp_repo` as an inputResource to build the image based on the provided Dockerfile.

Optionally, may also specify:

  • an Image resource of a base Image to include in the built Image
  • a FileSpec resource that specifies what files to include in the built Image. These files are automatically copied to dockerFileLocation.
Required/Optional
integrationsMust specify an Artifactory Integration.Required


DockerPush

The DockerPush native step pushes the specified Docker Image to Artifactory.

steps
      - name: docker_push
        type: DockerPush
        configuration:
          affinityGroup: dbp_group
          targetRepository: docker-local
          integrations:
            - name: art                            # <-- replace with your artifactory integration
          inputSteps:
            - name: docker_build
          outputResources:
            - name: dbp_image

Tag

Description of usage

Required/Optional

namedocker_push is the name that identifies the step.Required
affinityGroupMust specify an affinity group string that is the same as specified in the prior DockerBuild step.Optional
targetRepositoryThe name of the Docker repository in Artifactory.Required
integrationsMust specify an Artifactory Integration.Required
inputSteps 

Must specify the named DockerBuild step in the same affinity group.

The output (result) of the previous docker_build step is used as an inputStep for this step.

Required
outputResources 

May specify an Image resource. If one is specified, the imageTag  property of that resource will be updated with the dockerImageTag of the preceding DockerBuild step.

This step outputs `dbp_image` as an outputResource, which can be used to trigger downstream dependent steps or pipelines if needed.

Optional

May be required


PublishBuildInfo

The PublishBuildInfo step publishes BuildInfo attached to the image to Artifactory. BuildInfo provides a manifest for the build and includes metadata about the modules, dependencies and other environment variables.

BuildInfo can also be published by any of the language-specific publish steps, when its autoPublishBuildInfo tag is set to true.

steps
      - name: publish_dbp_build
        type: PublishBuildInfo
        configuration:
          inputSteps:
            - name: docker_push
          outputResources:
            - name: dbp_build_info

Tag

Description of usage

Required/Optional

namepublish_dbp_build is the name that identifies the step.Required
inputSteps 

Must specify the name of the DockerPush step. The output (result) of the previous step is used as an inputStep for this step.

The step must not have set autoPublishBuildInfo to true.

Required
outputResources 

Must specify a BuildInfo resource to publish.

Required


PromoteBuild

The PromoteBuild native step promotes thdbp_build_info BuildInfo and moves or copies the related artifacts from one Artifactory repository to another. 

This step requires either a PublishBuildInfo step as previous step or a BuildInfo resource as an input to successfully execute.

steps
      - name: promote_dbp_build
        type: PromoteBuild
        configuration:
          targetRepository: demo-pipelines
          integrations:
            - name: art
          inputResources:
            - name: dbp_build_info
          outputResources:
            - name: dbp_promoted_build_info

Tag

Description of usage

Required/Optional

namepromote_dbp_build is the name that identifies the step.Required
targetRepositorydemo-pipelines is the name of the repository in Artifactory to promote the build to.Required
integrationsMust specify an Artifactory Integration.Required
inputResources Must specify a named BuildInfo resource whose buildName and buildNumber properties identify the build to promote.Required
outputResources Must specify a named BuildInfo resource to map to the promoted build. The BuildInfo will be updated with the buildName and buildNumber of the input BuildInfo resource, and its targetRepo with the value of targetRepository .Required
  • No labels
Copyright © 2021 JFrog Ltd.