Cloud customer?
 Upgrade in MyJFrog >

Search





Overview

The Matrix step commences multiple parallel build processes across multiple containers, with different settings for each.


Common use cases for Matrix steps are:

  • Splitting a large test suite into smaller units, executing them in parallel to reduce total execution time.
  • Testing against multiple values of environment variables or multiple runtime images.
  • Testing against multiple base operating system versions.

The matrix configuration can specify:

  • Multiple sets of environment variable definitions
  • Multiple runtime images
  • Multiple node pools.

The Matrix step executes the specified shell scripts multiple times in parallel steplets, in each specified runtime for each set of environment variables on each specified platform. For example, if a Matrix step specifies 3 sets of environment variables and 2 runtime images, it will run a total of 6 steplets (3 times in runtime 1, 3 times in runtime 2). If the Matrix step also specifies 2 node pools, it will run 12 steplets.

A Matrix step can be optionally preceded by a PreMatrix step to prepare the build node environment for the steplets, and a PostMatrix step to aggregate information produced by the steplets. 

For more information, see Using the Matrix Step.

Usage


Matrix
pipelines:
  - name: <string>
    steps:
      - name: <string>
        type: Matrix
        stepMode: Bash | Powershell
        configuration: 
          #inherits from Bash 
          multiNode: <boolean>        # optional, only needed if steplets 
                                      # need to execute on separate nodes

        execution:
          onStart:
            - echo "Preparing for work..."
          onExecute:                  		# required     
            - echo "Executing steplet $step_name"
            - echo "env1 = $env1"
            - echo "env2 = $env2"
          onSuccess:
            - echo "Job well done!"
          onFailure:
            - echo "uh oh, something went wrong"
          onComplete: #always
            - echo "Cleaning up some stuff"

        stepletMultipliers:
          nodePools:                  # optional, only needed if multiNode
            - <nodePoolName>          # is true AND user wants to
            - <nodePoolName>          # execute against multiple platforms 
 
          environmentVariables:       # optional, only needed if user
            - env1: one               # wants to execute step against
              env2: two               # multiple values of env
            - env1: abc                
              env2: xyz

          runtimes:                   # optional, only needed if user
            - image:                  # wants to execute step against 
                auto:                 # multiple images
                  language:
                  version:
                custom:
                  <same config as Bash>
            - image:               
                auto:
                  language:
                  version:
                custom:
                  <same config as Bash>

          fastFail: <true/false>     # default false. If specified, matrix
                                     # step fails when any steplet fails, 
                                     # unless specified in allowFailures

          allowFailures:             # optional, array of combinations
            - nodePool:              # that are allowed to fail  
              environmentVariables:
                env1: abc                              
                env2: xyz
              runtime:               

          exclude:                   # optional, array of combinations
            - nodePool:              # that are not executed  
              environmentVariables:
                env1: one
                env2: two
              runtime:  
Page Contents

Tags

name 

An alphanumeric string (underscores are permitted) that identifies the step.

type 

Must be Matrix for this step type.

stepmode

Specifies the runtime OS mode. May be either Bash or PowerShell. If not specified, defaults to Bash.

configuration

Specifies all configuration selections for the step's execution environment. This step inherits the Bash/PowerShell step configuration tags, including these pertinent  tags:

Tag

Description of usage

Required/Optional
inputStep 

May specify a PreMatrix preamble step.

Optional


In addition, these tags can be defined to support the step's native operation:

Tag

Description of usage

Required/Optional

multiNode

When true, steplets will be required to run on separate nodes.

Default is false.

Optional


stepletMultipliers

In addition, these tags can be defined to support the step's native operation:

Tag

Description of usage

Required/Optional
nodePools

A collection of node pool names. Recognized only when multiNode is set to true. When specified, the step will be executed in each of the defined runtimes for each set of environmentVariables on each node pool.

If not defined, the step will be executed in the default node pool, or the node pool specified in the configuration section.

Optional
environmentVariables

A collection of sets of environment variable definitions. Each set of definitions will be used in an execution of a steplet in each of the defined runtimes.

If not defined, then a single steplet will execute for each of the defined runtimes on each of the specified nodePools.

Optional
runtimes

A collection of runtime defintions. The step will be executed in each defined runtime, on each of the specified nodePools, for each set of environmentVariables.

If not defined, then each steplet will be executed in the default runtime, or the runtime specified in the configuration section.

Optional
fastFail

When set to true the step fails when any steplet fails, unless permitted to fail in allowFailures.

Default is false.

Optional
allowFailures

A collection that specifies the combinations of nodepoolsenvironmentVariables, and runtimes that are permitted to fail without terminating the entire Matrix step. This only has effect when fastFail is set to true.

Optional
excludeA collection that specifies the combinations of nodepoolsenvironmentVariables, and runtimes that will not be executed.Optional


execution

Declares collections of shell command sequences to perform for pre- and post-execution phases:
TagDescription of usageRequired/Optional
onStartCommands to execute in advance of onExecuteOptional
onExecuteMain commands to execute for the stepOptional
onSuccessCommands to execute on successful completionOptional
onFailureCommands to execute on failed completionOptional
onCompleteCommands to execute on any completionOptional

Example

This is an example of how to use the Matrix step to perform a build activity. 

Matrix Step Example
      - name: matrix_example
        type: Matrix
        stepMode: Bash
        configuration:
          inputSteps: 
            - name: matrix_fan_out_example

        stepletMultipliers:
          environmentVariables:             # Sets of environment variables for steplets
            - animal:    dog				# - Set 1
              mineral:   copper
              vegetable: carrot
            - animal:    goat				# - Set 2
              mineral:   iron
              vegetable: broccoli
            - animal:    lizard             # - Set 3
              mineral:   lead

          runtimes:                         # Runtimes for steplets
            - type: image
              image:
                auto:
                  language: java
                  versions:
                    - 13.0
            - type: image
              image:
                auto:
                  language: node
                  versions:
                    - 8.17.0
        execution:
          onExecute:
            - echo "I am executing matrix steplet ${steplet_id}"
            - echo "Envs animal= $animal , mineral = $mineral , vegetable = $vegetable"
            - echo "Runtime image name: $step_image_name | Runtime image tag: $step_image_tag"


When run, the above example will execute in a total of 6 steplets: in 2 runtimes for each of 3 sets of environment variables:

RuntimeSet 1Set 2Set 3
java 13.0dog, copper, carrrotgoat, iron, broccoli

lizard, lead, <null>

node 8.17.0dog, copper, carrrotgoat, iron, broccoli

lizard, lead, <null>



  • No labels
Copyright © 2020 JFrog Ltd.