System Templates

Add System Template (1.31.0 and higher)

This section provides information about the steps required for creating and publishing system templates.

Step 1: Create System Template Files

1. In your SCM repository, create a new directory. For example: templates.

2. In the directory, create three new files:

   • templatesDefinition.yml: This should contain the pipeline definition and should be a valid yml file. For more information, see the example templatesDefinition.yml file.

   • values.yml.example: This should be a valid yml file and it is used to specify the details required for the templateDefinition.yml file. For more information, see the example values.yml file.

   • readme.md (optional): This is optional and the information in the readme.md file will be available as Documentation for the template.

Step 2: Use System Template to Create a Pipeline Source

After creating the relevant files for your template, use the pre-loaded PublishTemplate template to publish and use your system templates:

1. In the Pipelines UI:
   • Add an integration for one of the following Git providers:
     • Bitbucket Integration
     • Bitbucket Server Integration
     • GitHub Enterprise Integration
     • GitHub Integration
     • GitLab Integration
   • Add an Artifactory Integration

2. In your SCM repository, create a new directory named .jfrog-pipelines.

3. In the directory, create two new files:

   • pipelines.yml

   • values.yml

4. Add the following in the pipelines.yml file:

   valuesFilePath: ./values.yml
   Include:
   	template: jfrog/PublishTemplate/1.0.0

   Where:

   • jfrog is the predefined namespace to be used for the PublishTemplate template. This is mandatory and cannot be changed.
   • PublishTemplate is the predefined name of the pre-loaded template that is used for publishing other templates. This is mandatory and cannot be changed.
   • 1.0.0 is the version of the PublishTemplate template. This is mandatory and cannot be changed.

5. Update the values.yml with the following values:

   templateRepository:
     gitProvider: <git_integration_name>
     path: <git_repository_path>
     branch: <template_branch>
     templateFolder: <template_files_folder>
     
   publishTemplate:
     namespace: <template_namespace>
     name: myPipelineTemplate <template_name>
     version: 1.0.0 <template_version>
     pipelineName: <optional; pipeline_alternate_name>
     artifactoryIntegration: <artifactory_integration_name>

   Example

   templateRepository:
     gitProvider: myGithub
     path: john/templates
     branch: main
     templateFolder: templates/my_name_space/myPipelineTemplate
    
   publishTemplate:
     namespace: newTemplates
     name: myPipelineTemplate
     version: 1.0.0
     pipelineName: PublishTemplate
     artifactoryIntegration: myArtifactory

   Where:

   • templateRepository provides information about where the template is stored.
   • gitProvider is the name of the Git integration you created in the first step.
   • path is the Git repository path.
   • branch is the name of the branch where the template is available.
   • templateFolder is the relative path from the root where the template files are located.
   • publishTemplate provides requisite metadata for publishing the template.
   • namespace is the namespace used for storing the template in Artifactory.
   • name is the name used for storing the template in Artifactory.
   • version is the version of the template. 
   • pipelineName is the d efault name of the pipeline that is used for publishing the template to Artifactory. While PublishTemplate is the default name, this can be used to provide an alternative name for the pipeline. It is recommended that you change the default name as there could be a pipeline name conflict if you are using the same template multiple times.
   • artifactoryIntegration is the name of the Artifactory integration you created in the first step.
6. In the JFrog Platform, go to Administration → Pipelines → Pipeline Sources and add the PublishTemplate pipeline as a pipeline source.
   After your Pipeline Source syncs successfully, you can view the newly added pipeline by navigating to My Pipelines on the left navbar and clicking Pipelines → My Pipelines.
7. Go to Application → Pipelines → My Pipelines → All Pipelines and run the pipeline.

Structure of Template Source Repository (1.30.0 and lower)

The system template files should be stored in the template source repository in a specific order for them to be uploaded to Pipelines:

• There must be a directory called templates in the root directory of the repository.
• In the templates directory, there must be a subdirectory path of the form: namespace/<name>/<files>.

Where:

• namespace is the name the template belongs to.
• name is the name of the template.

• files are the template-related files.
  

  The namespace and subdirectories can only contain letters, underscores, and numbers.

  
  

• The following files can be added as part of the templates:

  File	Description	Mandatory/Optional
  templateDefinition.yml	This should contain the pipeline definition and should be a valid yml file.	Mandatory
  values.yml.example	Use this file as a sample for creating a values.yml file. This should be a valid yml file and it is used to specify the details required for templateDefinition.yml file.	Optional
  readme.md	This should contain the documentation for the template.	Optional

  Using {{gitBranch}}

  • Both templateDefintion.yml and values.yml support the in-built keyword {{gitBranch}}. The value of {{gitBranch}} is substituted with the branch against which the pipeline source was created. For more information, see Creating Multibranch Pipelines.
  • When used in a conditional or comparison logic,  {{gitBranch}} placeholders must be wrapped within double quotes.
    Example:  {{ if eq "{{gitBranch}}" "master" }}

templateDefinition.yml: This is a sample templateDefinition.yml. This can be edited to create your own version of the file.

pipelines:
  - name: basic
    steps:
      - name: basic1
        type: Bash
        configuration:
          runtime:
            type: image
        execution:
          onExecute:
            - printenv {{ .Values.foo.bar }}
      - name: basic2
        type: Bash
        configuration:
          runtime:
            type: image
          inputSteps:
            - name: basic1
        execution:
          onExecute:
            - printenv {{ .Values.foo.baz }}
      - name: basic3
        type: Bash
        configuration:
          runtime:
            type: image
          inputSteps:
            - name: basic2
        execution:
          onExecute:
            - printenv {{ .Values.foo.zoo }}

values.yml.example: Use this file as a sample to create the values.yml file.

artifactoryIntegration: myArtifactoryIntegration

GitRepo:
  name: myGitRepo
  gitProvider: myGitIntegration
  path: myorg/myrepo
  branches:
    include: master


foo:
  name: bar

Image:
  name: myDockerImage
  sourceRepository: mySourceRepo      


Pipeline:
  name: myDockerPipeline

DockerBuild:
  name: myDockerBuild   
  dockerFileName: Dockerfile
  dockerImageName: <image name>

DockerPush:
  targetRepository: docker-local 

Bash:
  name: myBashStep 

Template Functions List

The pipeline templateDefinition.yml supports these helm chart style functions:

• String Functions: trim, wrap, randAlpha, plural, and others.

  • String List Functions: splitList, sortAlpha, and others.

  Default Value for String Functions

  If you are using a string function (such as trunc or abbrev) that expects a value, then ensure that the template gives it a default value.
  Example: .Values.app | default "" | trunc 6

• Integer Math Functions: add, max, mul, and others.
  • Integer Slice Functions: until, untilStep
• Float Math Functions: addf, maxf, mulf, and others.
• Date Functions: now, date, and others.
• Defaults Functions: default, empty, coalesce, fromJson, toJson, toPrettyJson, toRawJson, ternary
• Encoding Functions: b64enc, b64dec, and others.
• Lists and List Functions: list, first, uniq, and others.
• Dictionaries and Dict Functions: get, set, dict, hasKey, pluck, dig, deepCopy, and others.
• Type Conversion Functions: atoi, int64, toString, and others.
• Path and Filepath Functions: base, dir, ext, clean, isAbs, osBase, osDir, osExt, osClean, osIsAbs
• Flow Control Functions: fail
• Advanced Functions:
  • UUID Functions: uuidv4
  • Version Comparison Functions: semver, semverCompare
  • Reflection: typeOf, kindIs, typeIsLike, and others.
  • Cryptographic and Security Functions: derivePassword, sha256sum, genPrivateKey, and others.
  • Network: getHostByName

For more information about these functions, see  Sprig Function Documentation and  Template Functions.

Adding System Templates (1.30.0 and lower)

Perform the following steps to add system templates:

1. Ensure that templateDefinition.yml and values.yml files are available in the appropriate repository.
2. Add an admin project integration:
   System templates are uploaded to pipelines through an SCM repository. To add the repository, add a SCM admin integration from Administration | Pipelines | Integrations. For more information, see Administering Integrations.
3. Add a template source:
   Go to Pipelines | Extensions & Templates and click the Sources tab.  Click Add Source and Templates to add the template source repository.
   
   

4. Select the admin integration, add the repository name, and branch and click Create Source to create the template source.
   

   The admin integration should have admin access to the template source repository.

   

Pipelines adds all the templates from the template source and performs a sync, and the newly added or updated templates are available in Pipelines as the latest version. If there is an error during the sync, it fails.

Managing Templates

The templates from the newly uploaded template source is now available for creating pipelines. For information about managing, retiring, and git-tagging and releasing templates, see Managing Pipelines Templates.

Using System Templates to Create a Pipeline Source

Perform the following steps to use a system template to create a pipeline source:

1. Go to Administration | Pipelines | Pipeline Sources, and click Add Pipeline Source, and From Template. 
   
   

2. Complete the resulting Template Properties form: 
   
   
   
   
   • Click the Select Template Namespace field and select the namespace where the required system template is defined.
   • Click the Select Template Name field and select the relevant system template. 

   • Click the Select Template Version field and select the relevant version for the template.
     
     

3. Click Next and complete the resulting Specify values file form to add the value.yml file. 
   
   In the Values File Path field, provide the path to the values.yml file, which contains the values for the system template. By default, the field is set to take the file from the root directory.
   
   

   For more information about the Specify values file form, see the section Adding a Pipeline Source in Administering Pipeline Sources.

   
   
   
   

4. Click Create Source to complete adding the Pipeline Source.

The combination of pipeline template name, namespace, version and values.yml is parsed to create the pipeline definition. 

After your Pipeline Source syncs successfully, you can view the newly added pipeline by navigating to the My Pipelines on the left navbar and clicking on Pipelines → My Pipelines.