To enable retention policy at the system-level, in the
system.yaml file, set the retentionPolicy’s
enabled flag to
true. This policy applies to all your pipelines.
|By default, Retention Policy is disabled at system-level, which effectively means infinite retention.|
When enabled, system-level Retention Policy uses two properties to control which pipelines run data is to be retained:
retentionPolicy: enabled: true maxAgeDays: 90 minRuns: 10
In this example, system-level Retention Policy is enabled, and pipelines run data for the last 90 days is set to be retained and data older than 90 days is set to be deleted. The minimum runs property (
minRuns) ensures that for each one of your pipelines, the latest 10 pipeline runs data is retained, regardless of the policy’s max allowed age (
maxAgeDays). This means, if some (or all) of your 10 latest pipelines runs are older than 90 days, Pipelines will still keep their run data. If you have less than 10 pipeline runs for certain pipelines, Pipelines will keep all their pipelines run data.
Once Retention Policy is enabled at system-level, it can also be configured at at pipeline-level. For information about setting the values for pipeline-level Retention Policy, see Defining a Pipeline.
Just like Retention Policy at system-level, you can control the values of the number of days and minimum number of runs to keep at pipeline-level. These settings override the system-level Retention Policy, but pipeline-level policy values can only be equal to or lesser than the system-level configuration.
For example, if the system-level Retention Policy configuration is:
And the pipeline-level Retention Policy configuration is:
Then Pipelines will discard Pipeline run data for runs that are older than 30 days. Pipelines, however, will also keep at least 5 pipeline runs data, even if they are older than 30 days (or even if they are older than system-level 90 days value).
|If you set the values of pipeline-level Retention Policy to greater than your system-level values, the Pipeline Source Sync will fail.|
pipelines: - name: example_pipeline_retention_policy configuration: retentionPolicy: maxAgeDays: 30 minRuns: 5 steps: - name: step_1 type: Bash execution: onExecute: - echo "Hello world"