Managing Pipeline Sources

Viewing Pipeline Sources

To view the list of pipeline sources already added to Pipelines, go to Application | Pipelines | Pipeline Sources.

The page displays the list of pipelines sources that are available to you to use according to permission targets defined in the JFrog Platform. Your user account must be granted permissions for a pipeline source for it to be shown.

Each row of the pipeline sources list includes the following:

If the pipeline source is a multibranch source, the row presents aggregate information for all branches, and can be expanded/collapsed to show the sync status of each branch.

Sync Pipeline Source

When any of the pipeline config files have changed, you will need to sync the pipeline source to reload:

1. Hover over the row in the pipeline source list at the far right until the Sync icon appears.
2. Click the Sync icon.

If the Status indicates that the sync failed, click Sync Logs to view the logs of all synchronized config files.

Administering Pipeline Sources

Managing a pipeline source can be performed only by users with administrator privilege.

Requirements

To successfully add a pipeline source, ensure the following

1. The credentials used in the source control integration should have admin access to the source control repository specified. Admin privileges are used to add a webhook that allows Pipelines to be notified of changes in the repository.
2. The repository path is valid. To know the exact path to specify inspect the git clone URL of that repository in your source control system and copy the value. Here are some examples 
   1. GitHub / GitHub Enterprise: https://github.com/myuser/basic-pipeline.git - use myuser/basic-pipeline
   2. Bitbucket Server (Private repository): https://git.mycompany.com/scm/project-id/repo-name.git - use project-id/repo-name
   3. Bitbucket Server (Individual repository): https://git.mycompany.com/scm/~myuser/myfirstrepo.git - use ~myuser/myfirstrepo
   4. Bitbucket: https://username@bitbucket.org/teamspace/test-repo.git - use teamspace/test-repo
   5. GitLab: https://gitlab.com/user1/repo1.git - use user1/repo1 
3. Branch name is valid and the credential used in the Git integration for the pipeline source has access to the branch. 

Adding a Pipeline Source (Single Branch)

When you have added a repository in your source control system that holds your Pipelines config files, you can add it as a pipeline source. At the time of creating the pipeline source, only the source repository and branch have to exist, the Pipeline config files need not exist. 

If you haven't already done so, add an integration for the source control system where your pipeline file repository is (or will be) located. The integration can be one of these source control system providers:

• GitHub Integration
• GitHub Enterprise Integration
• GitLab Integration
• Bitbucket Integration
• Bitbucket Server Integration

To add a source control repository as a Pipeline Source:

1. Go to Application | Pipelines | Pipeline Sources.
   
   
2. In the resulting Pipeline Sources display, click Add New Pipeline Sources.
   
   
   
3. Complete the resulting Add New Pipeline Source form: 
   
   1. Click the Integration field and select your source control integration from the dropdown list. Only integrations that are compatible for use as a Pipeline Source will be included in the dropdown list.
   2. In the Repository Full Name field, enter the path of the repository where your pipeline files are stored.
   3. For a single branch pipeline source, leave the Add MultiBranch box unchecked.
   4. Specify the Branch of the repository (for example, master)
   5. Enter a regular expression for the the Pipeline Config File Filter. Any filenames that match the expression will be loaded. For example, if you plan to declare your resources in a file named pipelines.resources.yml and your pipelines in pipelines.workflows.yml, then you might enter pipelines.*.yml.
      
      
4. Click Create to complete adding the Pipeline Source.

If JFrog Pipelines successfully synced with the repository, the Status field will display  "Success". Otherwise, it will display "Failed".

All pipeline config files committed to the repository will be automatically loaded into JFrog Pipelines, and all defined pipelines from that source are listed in the pipeline list.

Adding a Multibranch Pipeline Source

A multibranch pipeline automatically creates and destroys pipeline config files in branches of a project from the main branch. This is useful for development teams that create new branches for features in-progress and merge back to the main branch and delete the branch when feature development is complete. 

A prerequisite for a multibranch pipeline is that its pipeline source must be configured as multibranch.  The pipeline config file must also be written as a multibranch pipeline, but it will not work if its pipeline source is not also configured as multibranch.

When configuring a pipeline source as multibranch, you can specify Exclude and Include Branch Patterns to limit which branches participate in automatic pipeline creation/deletion.

To add a source control repository as a multibranch Pipeline Source:

1. Go to Application | Pipelines | Pipeline Sources, then click Add New Pipeline Sources.
   
   
2. Complete the resulting Add New Pipeline Source form: 
   
   1. Select the Integration.
   2. In the Repository Full Name field, enter the path of the main project repository where your pipeline files are stored.
   3. Select the Add MultiBranch checkbox.
   4. Specify the Exclude Branch Pattern as a regular expression for the matching branch names to exclude.
   5. Specify the Include Branch Pattern as a regular expression for the matching branch names to include.
   6. Enter a regular expression for the Pipeline Config File 
      
      
3. Click Create to complete adding the Pipeline Source.