A successful run of the pipeline in this quickstart looks like this:
Tutorial Video
Before you Begin
Before trying this quickstart, ensure that you have:
- A GitHub account. This is required for forking the example repository.
- A JFrog Platform account, or self-hosted JFrog Pipelines.
- A user account in Artifactory with deploy permissions to at least one binary repository.
- Set up Artifactory as a Docker registry. For more information, see Getting Started with Artifactory as a Docker Registry and Docker Registry.
- At least one node pool. This is the set of nodes that all pipeline steps will execute in. For more information, see Managing Pipelines Node Pools.
Running This Example
Perform the steps below to build and push your Docker image:
Fork the Repository
The Pipelines DSL for this example is available in the jfrog-pipelines-docker-sample repository in the JFrog GitHub account.
The DSL file is a yaml file that contains the pipeline definitions. The example uses a single YAML file,
pipelines.yml
. This file contains the declarations for all resources and workflow steps. For a full breakup of all the resources, pipelines and steps used in the yml file, see the pipelines.yml section below.Fork this repository to your account or organization. This is important since you need admin access to repositories that are used as Pipeline Sources or GitRepo resources, in order to add webhooks to these repositories and listen for change events.
Sign in to Artifactory
Sign in to JFrog Platform with your Artifactory credentials.Add Integrations
a. Go to Administration | Pipelines | Integrations to add two integrations:
- GitHub Integration: This integration is used to add the Pipeline source, as well as the GitRepo resource defined in pipelines.yml.
- Artifactory Integration: This integration is used to authenticate with Artifactory to push the Docker image to the docker-local repository in Artifactory.
b. Write down the names of both GitHub and Artifactory integrations as these are required for the next step. Ensure that the names are unique and easy to remember.
Update pipelines.yml file
The pipelines configuration is available in the pipelines.yml file. Edit this file in your fork of the Git repo and replace the following:
Tag Description Example gitProvider
Provide the name of the Github integration you added in the previous step. gitProvider: my_github
path
Provide the path to your fork of this repository. path: myuser/repo-name
registry
Provide the the name of the Artifactory integration you added in the previous step. registry: art
imageName
Provide your Docker image path and name. docker.artprod.mycompany.com
/docker_local/dbp
Where, docker.artprod.mycompany.com/docker_local
is the image path anddbp
is the image name.sourceArtifactory
Provide your Artifactory integration. sourceArtifactory: art
All pipeline definitions are global across JFrog Pipelines within a Project. The names of your pipelines and resources need to be unique within the Project in JFrog Pipelines.
Add Pipeline Sources
The Pipeline Source represents the Git repo where our Pipelines definition files are stored. A pipeline source connects to the repository through an integration, which we added in the previous step.
Follow instructions to add a Pipeline Source and point it to the
pipelines.yml
in your fork of the repo. This automatically adds your configuration to the platform and a pipeline is created based on your YAML. Thepipelines
.yml
file is parsed and resources, steps, and pipelines are added as configured.After your pipeline source syncs successfully, navigate to Pipelines | My Pipelines in the left navbar to view the newly added pipeline. In this example,
pipeline_dbp
is the name of our pipeline.Click the name of the pipeline. This renders a real-time, interactive, diagram of the pipeline and the results of its most current run.
Execute the Pipeline
You can trigger the pipeline by committing a change to your Git repository, or by manually triggering it through the UI. The steps in the pipeline execute in sequence. Multiple steps can execute in parallel if the node pool has multiple build nodes available.
Once the pipeline has completed, a new run is listed.
pipelines.yml
The pipelines.yml
file is made up of resources, pipelines and steps, as shown below:
Resources
This example uses the following types of resources:
GitRepo
A GitRepo resource is used to connect JFrog Pipelines to a source control repository. Adding it creates a webhook to the repo so that future commits will automatically create a new version with the webhook payload.
- name: dbp_repo type: GitRepo configuration: gitProvider: my_github # <-- replace with your integration path: jfrog/jfrog-pipelines-docker-sample # <-- replace with your repository name branches: include: master
Tag | Description | Required/Optional |
---|---|---|
name |
This name is used to refer to the resource in steps, and must be unique across all repositories in your JFrog Pipelines environment. | Required |
| The name of the GitHub integration. | Required |
path | The path of the repository from the integration root. | Required |
branches |
The | Optional |
Image
An Image resource is used to add a reference to a Docker image to your pipeline. An Image resource can be used as both input and output. Steps like DockerBuild will generate it as an output and steps like DockerPublish will use it as input to publish to a Docker registry. In our example, it is used as an outputResource for the DockerPush step.
- name: dbp_image type: Image configuration: registry: art # <-- replace with your artifactory integration sourceRepository: docker_local # <-- required if registry is Artifactory imageName: docker.artprod.mycompany.com/docker_local/dbp # < -- replace with your image path and name imageTag: latest autoPull: true
Tag | Description | Required/Optional |
---|---|---|
name | dbp_image is the name that identifies the resource. | Required |
registry | art is the name of a Docker Registry Integration. | Required |
sourceRepository | docker_local is the name of the repository in Artifactory to which the images will be published. | Required if registry is in JFrog Artifactory |
imageName | The file path of the Docker image, of the form imageRepo/imageName. | Required |
imageTag | The version tag of the initial version of the Docker image. | Required |
autoPull | When true, the image is automatically pulled to the machine running the step. Defaults to false. | Optional |
BuildInfo
BuildInfo is automatically created when the DockerBuild step is used to generate packages. BuildInfo is then published to the configured Artifactory repo (sourceArtifactory: art
) through the PublishBuildInfo step, by providing that resource in the DockerPush step.
- name: dbp_build_info type: BuildInfo configuration: sourceArtifactory: art buildName: dbp_build buildNumber: 1 - name: dbp_promoted_build_info type: BuildInfo configuration: sourceArtifactory: art buildName: dbp_build buildNumber: 1
Tag | Description | Required/Optional |
---|---|---|
name |
This name is used to refer to the resource in steps, and must be unique across all repositories in your JFrog Pipelines environment. | Required |
|
| Required |
buildName |
| Optional |
buildNumber | The number of the build to associate with the BuildInfo. | Optional |
Pipelines
pipeline_dbp
, the pipeline definition for building and pushing the Docker image.
Steps
The pipeline_dbp
pipeline contains the following native steps:
DockerBuild
The DockerBuild native step performs a build to produce a Docker image from a Dockerfile in a Git source repository.
This step builds a Docker image from a GitRepo source repository resource. You must provide in the step configuration the name and directory of the Dockerfile that contains the command to be processed by a docker build
command, as well as the name and tag of the resulting image. The image is built on the build node, and information about that image is stored in the run state.
- name: docker_build type: DockerBuild configuration: affinityGroup: dbp_group dockerFileLocation: . dockerFileName: Dockerfile dockerImageName: docker.artprod.mycompany.com/docker_local/dbp # replace with your image path and name dockerImageTag: ${run_number} inputResources: - name: dbp_repo integrations: - name: art # <-- replace with your artifactory integration
Tag | Description of usage | Required/Optional |
---|---|---|
name |
This is the name that is used when the step is assigned as an input to the next step, | Required |
affinityGroup | Must specify an affinity group string that is the same as specified in a subsequent DockerPush step. Both DockerBuild and DockerPush steps are assigned to the dbp_group affinityGroup so that they share the same state. If they are not, the output of the DockerBuild step won't be available to the DockerPush step. | Optional |
dockerFileLocation | Directory containing the Dockerfile or the file that has Docker build configuration | Required |
dockerFileName | Name of the Dockerfile | Required |
dockerImageName | The name of the Docker image to create. This can be set using environment variables or triggering a run using parameters. | Required |
dockerImageTag | The tag for the Docker image to create. This can be set using environment variables or triggering a run using parameters. In our example, the tag for the image is set to the pipeline's run number, which is the number of the run currently executing. | Required |
inputResources | Must specify:
This step accepts `dbp_repo` as an inputResource to build the image based on the provided Dockerfile. Optionally, may also specify: | Required/Optional |
integrations | Must specify an Artifactory Integration. | Required |
DockerPush
The DockerPush native step pushes the specified Docker Image to Artifactory.
- name: docker_push type: DockerPush configuration: affinityGroup: dbp_group targetRepository: docker-local integrations: - name: art # <-- replace with your artifactory integration inputSteps: - name: docker_build outputResources: - name: dbp_image
Tag | Description of usage | Required/Optional |
---|---|---|
name | docker_push is the name that identifies the step. | Required |
affinityGroup | Must specify an affinity group string that is the same as specified in the prior DockerBuild step. | Optional |
targetRepository | The name of the Docker repository in Artifactory. | Required |
integrations | Must specify an Artifactory Integration. | Required |
inputSteps | Must specify the named DockerBuild step in the same affinity group. The output (result) of the previous | Required |
outputResources | May specify an Image resource. If one is specified, the This step outputs `dbp_image` as an outputResource, which can be used to trigger downstream dependent steps or pipelines if needed. | Optional May be required |
PublishBuildInfo
The PublishBuildInfo step publishes BuildInfo attached to the image to Artifactory. BuildInfo provides a manifest for the build and includes metadata about the modules, dependencies and other environment variables.
BuildInfo can also be published by any of the language-specific publish steps, when its autoPublishBuildInfo
tag is set to true.
- name: publish_dbp_build type: PublishBuildInfo configuration: inputSteps: - name: docker_push outputResources: - name: dbp_build_info
Tag | Description of usage | Required/Optional |
---|---|---|
name | publish_dbp_build is the name that identifies the step. | Required |
inputSteps | Must specify the name of the DockerPush step. The output (result) of the previous step is used as an inputStep for this step. The step must not have set | Required |
outputResources | Must specify a BuildInfo resource to publish. | Required |
PromoteBuild
The PromoteBuild native step promotes the dbp_build_info BuildInfo and moves or copies the related artifacts from one Artifactory repository to another.
This step requires either a PublishBuildInfo step as previous step or a BuildInfo resource as an input to successfully execute.
- name: promote_dbp_build type: PromoteBuild configuration: targetRepository: demo-pipelines integrations: - name: art inputResources: - name: dbp_build_info outputResources: - name: dbp_promoted_build_info
Tag | Description of usage | Required/Optional |
---|---|---|
name | promote_dbp_build is the name that identifies the step. | Required |
targetRepository | demo-pipelines is the name of the repository in Artifactory to promote the build to. | Required |
integrations | Must specify an Artifactory Integration. | Required |
inputResources | Must specify a named BuildInfo resource whose buildName and buildNumber properties identify the build to promote. | Required |
outputResources | Must specify a named BuildInfo resource to map to the promoted build. The BuildInfo will be updated with the buildName and buildNumber of the input BuildInfo resource, and its targetRepo with the value of targetRepository . | Required |