Setting Retention Policy at System-level
To enable retention policy at the system-level, in the
system.yaml file, set the retentionPolicy’s
enabled flag to
true. This policy applies to all your pipelines.
When enabled, system-level Retention Policy uses two properties to control which pipelines run data is to be retained:
- Max Age Days: Specifies the number of days, after which the pipelines run data will be deleted. This can be set in granularity of days. Default value is 0 (infinite retention).
- Minimum Runs: Specifies the minimum number of pipeline runs to keep, regardless of their age. Default value is 10.
In this example, system-level Retention Policy is enabled, and pipelines run data for the last 90 days is set to be retained and data older than 90 days is set to be deleted. The minimum runs property (
minRuns) ensures that for each one of your pipelines, the latest 10 pipeline runs data is retained, regardless of the policy’s max allowed age (
maxAgeDays). This means, if some (or all) of your 10 latest pipelines runs are older than 90 days, Pipelines will still keep their run data. If you have less than 10 pipeline runs for certain pipelines, Pipelines will keep all their pipelines run data.
Setting Retention Policy at Pipeline-level
Once Retention Policy is enabled at system-level, it can also be configured at at pipeline-level. For information about setting the values for pipeline-level Retention Policy, see Defining a Pipeline.
Just like Retention Policy at system-level, you can control the values of the number of days and minimum number of runs to keep at pipeline-level. These settings override the system-level Retention Policy, but pipeline-level policy values can only be equal to or lesser than the system-level configuration.
For example, if the system-level Retention Policy configuration is:
And the pipeline-level Retention Policy configuration is:
Then Pipelines will discard Pipeline run data for runs that are older than 30 days. Pipelines, however, will also keep at least 5 pipeline runs data, even if they are older than 30 days (or even if they are older than system-level 90 days value).