Cloud customer?
Start for Free >
Upgrade in MyJFrog >
What's New in Cloud >

Search





Overview

Pipeline Source  represents a source control repository (such as GitHub or BitBucket) where Pipelines definition files can be found. A pipeline source connects to the repository through an integration.Once a pipeline source is added, Pipelines will automatically load all config files from the repository that matches the specified filename filter. 

Administrator users can create, view, edit and sync pipeline sources. Non-administrator users can only view and sync pipeline sources.
Page Contents


Administering Pipeline Sources

Managing a pipeline source can be performed only by users with  administrator privilege.

Requirements

To successfully add a pipeline source, ensure the following:

Adding a Pipeline Source

To add a source control repository as a Pipeline Source:

  1. Go to Administration | Pipelines | Pipeline Sources.
  2. In the resulting Pipeline Sources display, click Add Pipeline Source.



  3. Click one of the following:
    • From YAML: Add the pipeline source from a YAML configuration file, which contains the declarations of all resources and workflow steps.
    • From Template: Add the pipeline source from a pre-defined template by providing a values file. For more information, see Pipelines Templates.

  4. From YAML
    1. In the resulting Add YAML Pipeline Source page, click one of the following:
      • Single Branch
      • Multi Branch

    2. Complete the  Single/Multi Branch form: 
      • SCM Provider Integration: Click the SCM Provider Integration field and select your source control integration from the dropdown list. Only integrations that are compatible for use as a Pipeline Source will be included in the dropdown list.
      • Repository Full Name: Based on the information you have provided for the selected integration, such as API token, all the relevant repositories are listed in the Repository Full Name  field. Select the path of the repository where your pipeline files are stored. If the name of the repository is not auto-fetched, enter the full name of the repository where your pipeline files are stored.
      • Branch (Single Branch only): Based on the SCM provider and Repository Full Name you have provided, all the available branches are automatically fetched (for example,  main ). Select the required branch. If the name of the branch is not auto-fetched, enter the name of the branch.
      • Exclude Branch (Multi Branch only): Specify the Exclude Branch Pattern as a regular expression for the matching branch names to exclude.
      • Include (Multi Branch only): Specify the Include Branch Pattern  as a regular expression for the matching branch names to include.
      • Pipeline Config File Filter: Enter a regular expression for the Pipeline Config File Filter. Any filenames that match the expression will be loaded.
        For example, if you plan to declare your resources in a file named pipelines.resources.yml and your pipelines in pipelines.workflows.yml, then you might enter pipelines.*.yml.

  5. From Template
    1. Complete the resulting  Template Properties  form: 



      • Click the Select Template Namespace field and select the namespace.
      • Click the Select Template Names field and select the template. 
      • Click the Select Template Version field and select the version for the template.

    2. Click Next.

    3. In the resulting Specify values file page, click one of the following:
      • Single Branch
      • Multi Branch

    4. Complete the Single/Multi Branch form: 
      • SCM Provider Integration: Click the SCM Provider Integration field and select your source control integration from the dropdown list. Only integrations that are compatible for use as a Pipeline Source will be included in the dropdown list.
      • Repository Full Name: Based on the information you have provided for the selected integration, such as API token, all the relevant repositories are listed in the Repository Full Name  field. Select the path of the repository where your pipeline files are stored. If the name of the repository is not auto-fetched, enter the full name of the repository where your pipeline files are stored.
      • Branch (Single Branch only): Based on the SCM provider and Repository Full Name you have provided, all the available branches are automatically fetched (for example,  main ). Select the required branch. If the name of the branch is not auto-fetched, enter the name of the branch.
      • Exclude Branch (Multi Branch only): Specify the Exclude Branch Pattern as a regular expression for the matching branch names to exclude.
      • Include (Multi Branch only): Specify the Include Branch Pattern  as a regular expression for the matching branch names to include.
      • Values File Path: Enter the path to the .values.yml file, which contains the values for the template.

  6. Click Create Source  to complete adding the Pipeline Source.

If JFrog Pipelines successfully syncs with the repository, the  Status field displays  "Success". Otherwise, it displays "Failed".

All pipeline config files committed to the repository will be automatically loaded into JFrog Pipelines, and all defined pipelines from that source are listed in the pipeline list.


Viewing Pipeline Sources

To view the list of pipeline sources already added to Pipelines, go to Administration | Pipelines | Pipeline Sources or Application | Pipelines | Pipeline Sources.

The page displays the list of pipelines sources that are available to you according to the permission targets defined in the JFrog Platform. Your user account must be  granted  permissions   for a pipeline source for it to be shown.

Each row of the pipeline sources list includes the following:

Repository
The source repository path of the pipeline source
Branch
The source repository branch
Latest Status
The success/failure status of the last sync
Pipeline Config File Filter
The filter string for the pipeline config files
Last Sync
The time and date of the last sync
Context
The commit SHA that triggered the last sync
Logs
Click the Logs link to view the log from the last sync. Use this to diagnose a failure to sync a pipeline source.

If the pipeline source is a multi branch source, the row presents aggregate information for all branches, and can be expanded/collapsed to show the sync status of each branch.

Sync Pipeline Source

When any of the pipeline config files have changed, you will need to sync the pipeline source to reload:

  1. Click the  Actions  button located at the far right.
  2. Click the Sync  icon.

If the Status  indicates that the sync failed, click Sync Logs  to view the logs of all synchronized config files.

Fetching Branches

When one or more branches are created for a Multibranch pipeline source, the webhooks may not be processed correctly, resulting in those branches not being added to the pipeline source.

Pipelines can  automatically fetch and sync missing branches in a multibranch pipeline source based on the selected SCM integration and the full name of the repository.

To do this:

  1. Click the Actions button for the multibranch pipeline, and then click Fetch Branches.


    The Fetch Branches window appears.

  2. In the  Branch  field, you can either:
    • Enter the name of the branch you want to fetch and sync.
      or
    • Click the  Fetch Branches  button to try to automatically fetch all the branches based on the configured SCM integration and repository.
      If one or more branches are found, they are listed in the drop-down menu.
  3. Click to select the relevant branches.


  4. Click  Confirm to sync the pipeline source.
    The newly found branches are now listed on the Pipeline Sources page.

  • No labels
Copyright © 2021 JFrog Ltd.