Embedded Pipelines

JFrog Pipelines Documentation

Products
JFrog Pipelines
Content Type
User Guide
ft:sourceType
Paligo

The TriggerPipeline step enables one step in a parent pipeline to "embed" another pipeline and wait for the resulting run to complete before returning the control back to the parent pipeline's step. The parent pipeline then continues its run with access to some additional information on the embedded run.

For instructions on creating and running an embedded Pipeline, see Creating and Running a Pipeline.

For example, in the image below, Step 2 in the parent pipeline X triggers the embedded pipeline Y. When pipeline Y is successful, it returns control to Step 2 in pipeline X, which triggers Step 3 and Step 4 in the pipeline.

Note

If the embedded run ends with a bad status (failure, timed out, cancelled), the TriggerPipeline step will end with a failure status.

embeddedPipelines1b.gif

Tip

  • Node Usage: The step triggering the embedded pipeline uses a dedicated node for processing, polling, and waiting for status. Keep this in mind when designing your embedded pipeline. Depending on how often you use the step, you could find that your nodepool runs out of capacity to execute the embedded pipeline leaving you in a stuck state. If you plan to make heavy use of this step type, we recommend considering assigning your TriggerPipeline steps to a separate nodepool to restrict the overall node capacity that can be used by this step type. By using a separate node pool, you can ensure that the pipeline does not go into a deadlock.

  • Wait time: TriggerPipeline steps are subject to timeout just like any other step. Keep this in mind when choosing an embedded pipeline. If your embedded pipeline takes longer than the system timeout limit, then it is likely that your TriggerPipeline step will timeout while waiting for the child to complete. One measure we have taken to avoid this is to reset the timeout timer of the TriggerPipeline step when the embedded run moves from a "waiting" status to a "processing" status.

  • Multiple TriggerPipeline steps: There is no limit to how many TriggerPipeline steps you can include in your pipelines, but keep in mind that it can be easy to find yourself waiting on multiple tiers of embedded runs. Before using this step type, consider if your scenario is better suited to simply connecting two pipelines via resource.