Pipelines Step-By-Step

Overview

After you have a Pipelines installation working within the JFrog Platform, you can get started by connecting Pipelines to execution nodes, source repositories and other services. After those essentials are in place, you can start coding (using YAML) and running your pipelines.

This section introduces you to the essential concepts and provides information about the primary building blocks for creating and running a pipeline.

This page is intended to provide a brief overview of all the steps required for executing a pipeline. For detailed information about each step, use the links provided in the Dive Deeper sections.
If you want to be more hands-on, try our Quickstarts.

Administrators and Developers

Pipelines can have two types of users, based on the permissions assigned to them:

Administrator: Pipelines does not grant access to all facilities to all users, restricting most configuration facilities to administrator users only. An administrator can add node pools, integrations, and pipeline sources.
Developer (non-administrator users): A developer can create YAML files, view and use the node pools, integrations and pipeline sources assigned to them, and run their pipelines. To connect Git repositories and other services to Pipelines, a developer will need assistance from an administrator.

Page Contents

Creating and Running a Pipeline

Before you start, an administrator user must perform the required procedures that connect Pipelines to the machines and services that enable a pipeline to run.
The section below is meant to provide you an overview of the steps involved in creating and running a pipeline. However, successful run of a pipeline requires a few prerequisites. For more information, see Pipelines Quickstart.

The main steps for creating and running a pipeline are as follows:

Create a Node Pool
Add Integrations
Create Pipeline DSL
Add a Pipeline Source
Run Pipeline

Step 1: Create a Node Pool

Who can perform this step?	Administrators only. Other users can go to Application \| Pipelines \| Node Pools to view the node pools that an administrator has assigned to them.
Description	To run a pipeline, you must provide Pipelines with machines for steps to execute on. In Pipelines, these machines are called nodes, and they are organized into node pools. Pipelines must be configured with at least one node pool that contains at least one node. One node pool is set as the default node pool and available to all users. You have a variety of choices in how node pools can be configured. Your nodes can be static (a VM in at a fixed IP address) or dynamic (on-demand in a cloud service).
Step(s)	To add a node pool and nodes, from the Administration tab, go to Pipelines \| Node Pools, and click Add Node Pool.
Dive Deeper	For information about adding a static or dynamic node pool, see Managing Pipelines Node Pools.

Step 2: Add Integrations

Who can perform this step?	Administrators only. Other users can go to Application \| Pipelines \| Integrations to view the integrations that an administrator has assigned to them.
Description	For Pipelines to connect to other services, such as GitHub, Artifactory, or Kubernetes, integrations must be added for those services. The integration must be provided with the URL endpoint for those services and credentials for a user account on that service, along with any other relevant parameters.
Step(s)	To add the integration, from the Administration tab, go to Pipelines \| Integrations, and click Add an Integration. Here, we add a GitHub Integration – but you can add an integration for the VCS system you prefer to use, whether that's GitHub Enterprise, GitLab, Bitbucket, or Bitbucket Server. For the full list of all the integrations you can add, see Pipelines Integrations. After your integration is successfully added, it is listed among the available integrations.
Dive Deeper	For more information, see Managing Pipelines Integrations.

Step 3: Create the Pipeline DSL

Who can perform this step?	Administrators and Developers.
Description	Pipelines are defined using Pipelines DSL, stored in one or more YAML files of key-value pairs, known as a pipeline config. A Pipelines DSL looks like this:
Step(s)	Create and then commit the .yml file to the path repository in your source VCS account. Then add (or have an administrator user add) your source VCS repository as a pipeline source.
Dive Deeper	For more information, see Defining a Pipeline.

Step 4: Add a Pipeline Source

Who can perform this step?	Administrators only. Other users can go to Application \| Pipelines \| Pipelines Sources view the pipeline sources that an administrator has assigned to them.
Description	For Pipelines to read and sync the Pipelines DSL from the source VCS repository, you must tell it where to find it by adding a pipeline source. This is best performed only after the Pipelines DSL file is checked into the source repo, so that Pipelines can sync the file immediately.
Step(s)	To add a pipeline source, from the Administration tab, go to Pipelines \| Pipelines Sources, and click Add Pipeline Source. When adding the pipeline source, ensure to specify the full path of the source repository in the VCS where your Pipelines DSL will be (or is currently) stored. Once the pipeline source is successfully added, Pipelines will sync the file to load the DSL file and create the declared resources and pipelines.
Dive Deeper	For more information, see Managing Pipeline Sources.

Step 5: Run the Pipeline

Who can perform this step?	Administrators and Developers.
Description	Trigger either a manual and automatic run of the pipeline.
Step(s)	The following are the steps for running a pipeline using the new UI. If you want the perform the same steps using the old UI, see the expandable section below. To browse pipelines loaded from configured pipelines sources, in the Application tab, go to Pipelines \| My Pipelines. After your Git repo has been added as a pipeline source, you can see your pipeline in My Pipelines. Click the name of the pipeline to see a real time, interactive, diagram of the pipeline and the results of its most current run. A pipeline can be defined to trigger execution when a new commit is made to the Git repo. You can also execute the pipeline by manually click the Run button or triggering the first step. Once the pipeline has completed, a new run will be listed. Click the row of the run to view the run log for what just executed. Steps using old UI The following are the steps for running a pipeline using the old UI. If you want the perform the same steps using the new UI, see the section above. To browse pipelines loaded from configured pipelines sources, in the Application tab, go to Pipelines \| My Pipelines. After your Git repo has been added as a pipeline source, you can see your pipeline in My Pipelines. Click the name of the pipeline to see a real time, interactive, diagram of the pipeline and the results of its most current run. A pipeline can be defined to trigger execution when a new commit is made to the Git repo. You can also execute the pipeline by manually triggering the first step. Once the pipeline has completed, a new run will be listed. Click the row of the run to view the run log for what just executed.
Dive Deeper	For more information on triggering pipelines, see Running a Pipeline. For information on examining a pipeline's execution, see Pipeline Run Logs.

Content

Space Tools