Pipeline Example: Docker Build and Push

A successful run of the pipeline in this quickstart looks like this:

Tutorial Video

Before you Begin

Before trying this quickstart, ensure that you have:

• A GitHub account. This is required for forking the example repository.
• A JFrog Platform account, or self-hosted JFrog Pipelines.
• A user account in Artifactory with deploy permissions to at least one binary repository.
• Create a local Docker repository to set up Artifactory as a Docker Registry.
• At least one node pool. This is the set of nodes that all pipeline steps will execute in. For more information, see Managing Pipelines Node Pools.

Running This Example

Perform the steps below to build and push your Docker image:

1. Fork the Repository

   The Pipelines DSL for this example is available in the jfrog-pipelines-docker-sample repository in the JFrog GitHub account. 

   The DSL file is a yaml file that contains the pipeline definitions. The example uses a single YAML file, pipelines.yml. This file contains the declarations for all resources and workflow steps. For a full breakup of all the resources, pipelines and steps used in the yml file, see the pipelines.yml section below.

   Fork this repository to your account or organization. This is important since you need admin access to repositories that are used as Pipeline Sources or GitRepo resources, in order to add webhooks to these repositories and listen for change events.
   
   

2. Sign in to Artifactory

   Sign in to JFrog Platform with your Artifactory credentials.
   
   

3. Add Integrations

   a. Go to Administration | Pipelines | Integrations to add two integrations:

   • GitHub Integration: This integration is used to add the Pipeline source, as well as the GitRepo resource defined in pipelines.yml. 
   • Artifactory Integration: This integration is used to authenticate with Artifactory to push the Docker image to the docker-local repository in Artifactory.

   b. Write down the names of both GitHub and Artifactory integrations as these are required for the next step. Ensure that the names are unique and easy to remember.
   
   

4. Update pipelines.yml file

   The pipelines configuration is available in the pipelines.yml file. Edit this file in your fork of the Git repo and replace the following:
   

   Tag	Description	Example
   gitProvider	Provide the name of the Github integration you added in the previous step.	gitProvider: my_github
   path	Provide the path to your fork of this repository.	path: myuser/repo-name
   registry	Provide the the name of the Artifactory integration you added in the previous step.	registry: art
   imageName	Provide your Docker image path and name.	docker.artprod.mycompany.com/docker_local/dbp Where, docker.artprod.mycompany.com/docker_local is the image path and dbp is the image name.
   sourceArtifactory	Provide your Artifactory integration.	sourceArtifactory: art

   All pipeline definitions are global across JFrog Pipelines within a Project. The names of your pipelines and resources need to be unique within the Project in JFrog Pipelines.

5. Add Pipeline Sources

   The Pipeline Source represents the Git repo where our Pipelines definition files are stored. A pipeline source connects to the repository through an integration, which we added in the previous step.

   Follow instructions to add a Pipeline Source and point it to the pipelines.yml in your fork of the repo. This automatically adds your configuration to the platform and a pipeline is created based on your YAML. The pipelines.yml file is parsed and resources, steps, and pipelines are added as configured.

   After your pipeline source syncs successfully, navigate to Pipelines | My Pipelines in the left navbar to view the newly added pipeline. In this example, pipeline_dbp is the name of our pipeline.
   

   

   Click the name of the pipeline. This renders a real-time, interactive, diagram of the pipeline and the results of its most current run.
   
   

6. Execute the Pipeline

   You can trigger the pipeline by committing a change to your Git repository, or by manually triggering it through the UI. The steps in the pipeline execute in sequence. Multiple steps can execute in parallel if the node pool has multiple build nodes available.

   

   Once the pipeline has completed, a new run is listed. 

   

   
   

   

pipelines.yml

The pipelines.yml file is made up of resources, pipelines and steps, as shown below:

Resources

This example uses the following types of resources:

• GitRepo
• Image
• BuildInfo

GitRepo

A GitRepo resource is used to connect JFrog Pipelines to a source control repository. Adding it creates a webhook to the repo so that future commits will automatically create a new version with the webhook payload.

  - name: dbp_repo
    type: GitRepo
    configuration:
      gitProvider: my_github                       # <-- replace with your integration
      path: jfrog/jfrog-pipelines-docker-sample    # <-- replace with your repository name
      branches:
        include: master

Image

An Image resource is used to add a reference to a Docker image to your pipeline. An Image resource can be used as both input and output. Steps like DockerBuild will generate it as an output and steps like DockerPublish will use it as input to publish to a Docker registry. In our example, it is used as an outputResource for the DockerPush step.

  - name: dbp_image
    type: Image
    configuration:
      registry: art                                # <-- replace with your artifactory integration
      sourceRepository: docker_local               # <-- required if registry is Artifactory
      imageName: docker.artprod.mycompany.com/docker_local/dbp  # < -- replace with your image path and name
      imageTag: latest
      autoPull: true

BuildInfo

BuildInfo is automatically created when the DockerBuild step is used to generate packages. BuildInfo is then published to the configured Artifactory repo (sourceArtifactory: art) through the PublishBuildInfo step, by providing that resource in the DockerPush step.

  - name: dbp_build_info
    type: BuildInfo
    configuration:
      sourceArtifactory: art
      buildName: dbp_build
      buildNumber: 1    

  - name: dbp_promoted_build_info
    type: BuildInfo
    configuration:
      sourceArtifactory: art
      buildName: dbp_build
      buildNumber: 1

Pipelines

pipeline_dbp, the pipeline definition for building and pushing the Docker image.

Steps

The pipeline_dbp pipeline contains the following native steps:

• DockerBuild
• DockerPush
• PublishBuildInfo
• PromoteBuild

DockerBuild

The DockerBuild native step performs a build to produce a Docker image from a Dockerfile in a Git source repository.

This step builds a Docker image from a GitRepo source repository resource.  You must provide in the step configuration the name and directory of the Dockerfile that contains the command to be processed by a docker build  command, as well as the name and tag of the resulting image.  The image is built on the build node, and information about that image is stored in the run state.

      - name: docker_build
        type: DockerBuild
        configuration:
          affinityGroup: dbp_group
          dockerFileLocation: .
          dockerFileName: Dockerfile
          dockerImageName: docker.artprod.mycompany.com/docker_local/dbp  # replace with your image path and name
          dockerImageTag: ${run_number}
          inputResources:
            - name: dbp_repo
          integrations:
            - name: art                            # <-- replace with your artifactory integration

DockerPush

The DockerPush native step pushes the specified Docker Image to Artifactory.

      - name: docker_push
        type: DockerPush
        configuration:
          affinityGroup: dbp_group
          targetRepository: docker-local
          integrations:
            - name: art                            # <-- replace with your artifactory integration
          inputSteps:
            - name: docker_build
          outputResources:
            - name: dbp_image

PublishBuildInfo

The PublishBuildInfo step publishes BuildInfo attached to the image to Artifactory. BuildInfo provides a manifest for the build and includes metadata about the modules, dependencies and other environment variables.

BuildInfo can also be published by any of the language-specific publish steps, when its autoPublishBuildInfo tag is set to true.

      - name: publish_dbp_build
        type: PublishBuildInfo
        configuration:
          inputSteps:
            - name: docker_push
          outputResources:
            - name: dbp_build_info

PromoteBuild

The PromoteBuild native step promotes the dbp_build_info BuildInfo and moves or copies the related artifacts from one Artifactory repository to another. 

This step requires either a PublishBuildInfo step as previous step or a BuildInfo resource as an input to successfully execute.

      - name: promote_dbp_build
        type: PromoteBuild
        configuration:
          targetRepository: demo-pipelines
          integrations:
            - name: art
          inputResources:
            - name: dbp_build_info
          outputResources:
            - name: dbp_promoted_build_info