This section provides information about using the UI on the Live Artifactory instance to perform the following tasks:
While you can access and use the Cold Artifactory instance UI, its functionality is limited to the following: |
Artifactory Cold Storage enables admins to create retention policies that determine which artifacts need to be removed and archived from a Live Artifactory instance. When the policy is executed, artifacts are moved from the Live Artifactory instance to the Cold Artifactory instance, and the binaries are stored in the storage solution connected to the Cold Artifactory.
During the archiving process, new repositories are created dynamically in the Cold instance for each repository.
Only Artifactory admins can create and run retention policies. |
To create a new retention policy:
Configure the following details:
Field | Description | ||
---|---|---|---|
Policy Name | Provide a unique and meaningful name for the policy. Enabled: The policy is disabled by default. When enabled, the policy will be executed on the scheduled date and time. | ||
Schedule Execution | Use a cron expression to set the schedule for the policy to execute. Example: | ||
Max Execution Duration | Enable and select the maximum duration that the policy has to execute. While this is useful for adhering to a strict archiving schedule, it can cause the archive policy to stop before completion. | ||
Active Policy | Check the checkbox to make this an active policy. Uncheck the checkbox if you do not wish to run the policy. | ||
Location | |||
Include Repository | Click the dropdown to select all the repositories to be included in the policy. | ||
Include/Exclude Path Pattern | Click Add to add all the path patterns that are to be included/excluded in the policy. | ||
Include/Exclude Artifact Name Pattern | Click Add to add all the artifact name patterns that are to be included/excluded in the policy. | ||
Properties | |||
Include/Exclude Name and Value | Click Add to add all the artifact properties that are to be included/excluded in the policy. | ||
Age | |||
Deployed before | Select artifacts based on when they were last deployed. For example, select artifacts that were deployed a year back. | ||
Last downloaded before | Select artifacts based on when they were last downloaded. For example, select artifacts that were downloaded a year back. | ||
Purge Artifacts from Archive | Specify if the archived artifacts need to be permanently deleted:
| ||
Description | If required, add a description for the policy. | ||
AQL | This enables you to use Artifactory Query Language (AQL) queries to select the artifacts that need to be archived.
To use AQL:
|
After the archive policy is defined and created:
It executes on the scheduled date and time.
The archiving policy can also be executed manually using the Trigger button.
Do not run more than 10 policies at the same time as it cause performance issues. |
Using Cold Artifact Storage, single file package types, such as Debian can be easily archived using a simple age-based criteria. In contrast, a Docker image contains multiple files, such as the manifest.json, and the individual image layers. Because of this, you may run into a situation where a Docker tag gets archived partially, due to asymmetrical download dates on individual files that belong to the same package. To archive these type of packages, use props/location-based policies on top of time-based policies. To ensure that your packages do not disappear while archiving, use the following 3-step approach:
|
The Retention Policies page lists the name of all the retention policies, their last execution status, last run date and time, and the schedule for their next run.
Hover over a policy and:
Status Icons
The Last Execution column displays the current status of each retention policy execution, which can be one of the following:
Icon | Last Execution Status | Meaning |
---|---|---|
Pending | Pending execution - policy has never been executed. | |
Triggered | User manually triggered the execution of the policy. | |
Running | Execution of the policy is in progress and there are no errors. | |
Running | Execution of the policy is in progress, but there are some errors. For information about these errors, see the logs. | |
Complete | Execution of the policy completed successfully, without any errors. | |
Complete | Execution of the policy completed, but there are some errors. For information about these errors, see the logs. | |
Stopping | User manually stopped the execution of the policy and the execution is being stopped. | |
Stopped | User manually stopped the execution of the policy. |
Search enables you to perform a dedicated search for artifacts in the Cold instance that you wish to restore to the Live instance. The search results indicate the origin of the resolved artifact (repository) and the availability, which depends on the type of storage used in the Cold instance.
After performing a search for the archived artifacts, admins can provide an explicit list of artifacts to be restored and trigger the restoration from the Cold instance to the Live instance. Restoration is a non-destructive process and none of the restored artifacts are deleted from the Cold instance.
To search and restore archived artifacts:
Advanced search: Click the Filter icon and filter your search based on the following fields:
Field | Description |
---|---|
Original Repository | Search for artifacts based on the original repository name. Wildcard characters are supported. |
Archiving Time | Search for all the artifacts archived within the specified time period. |
Artifact Name | Search for artifacts based on the artifact name. Wildcard characters are supported. |
Artifact Checksum | Search for artifacts based on MD5, SHA1 or SHA256 checksum value. Wildcard characters are not supported in Checksum search, so the term entered in the search field must be a valid MD5, SHA1 or SHA256 value. |
Properties | Search for artifacts or folders based on Properties assigned to them, whether they are standard properties assigned by Artifactory or custom properties. |
Based on your preference, select the following:
Field | Description | |
---|---|---|
Original Location | Restore to the original artifact location. | |
Fallback Location | If original location is selected, it is mandatory to provide a fallback location. If the original location does not exist anymore, the fallback location is used as the location for restoring artifacts.
| |
New Location | Restore to a single central destination on the Live instance.
|
|
After initiating an archive or restore process, you can monitor its progress and view the status of archive and restore operations based on policy name and timeframe. The last execution will always be displayed on top.
To monitor the status of an archiving or restoration process:
In the Administration module, select Artifactory | Cold Storage Monitoring .
The Cold Storage Monitoring page appears.
The Cold Storage Monitoring page provides information about the following:
Field | Description |
---|---|
Binaries Size | The total number and total size of the archived binaries. |
Artifacts Size | The total number and total size of the archived artifacts. |
Optimization | The ratio between the total size and the size of the binaries. |
Click Archiving/Restoring tab to view the following details:
Field | Description | |||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Filter (Archiving only) | Filter the list based on policy name. | |||||||||||||||||||||||||||
Hide Blank Executions (Archiving only) | Click this check box to hide all the executions that resulted in zero archival. | |||||||||||||||||||||||||||
Policy Name (Archiving only) | Name of the retention policy. | |||||||||||||||||||||||||||
Execution Id | Shows the unique execution Id generated for each execution of a policy. This Id is useful when reviewing the archive/restore logs. | |||||||||||||||||||||||||||
Date Range | The time period for which you would like to see the archive/restore status. | |||||||||||||||||||||||||||
Start Time | The date and time when the archive/restore operation started running. | |||||||||||||||||||||||||||
Status | The current status of the archive/restore operation, which can be one of the following:
|
This section provides information about using the Cold Artifact Storage APIs to perform the following tasks.
Artifactory Cold Storage enables admins to create retention policies that determine which artifacts need to be removed and archived from a Live Artifactory instance. When the policy is executed, artifacts are moved from the Live Artifactory instance to the Cold Artifactory instance, and the binaries are stored in the storage solution connected to the Cold Artifactory.
To perform archiving operations, the Live instance uses standard Artifactory REST API to deploy artifacts to the Cold instance. During the archiving process, new repositories are created dynamically in the Cold instance for each repository.
Only Artifactory admins can create and run retention policies. |
To set up a policy, use the Create an Archive Policy API. This will create a new archive policy in Artifactory using the policy configuration you set up in the API.
After the archive policy is defined and created:
It executes on the scheduled date and time.
The archiving policy can also be executed manually using the Run an Archive Policy API.
Do not run more than 10 policies at the same time as it cause performance issues. |
After creating an archiving policy, you can use the following APIs to view, edit, and delete the policy:
Search enables you to perform a dedicated search for artifacts in the Cold instance that you wish to restore to the Live instance. The search results indicate the origin of the resolved artifact (repo) and the availability, which depends on the type of storage used in the Cold instance.
After performing a search for the archived artifacts, admins can select the artifacts to be restored and trigger the restoration from the Cold instance to the Live instance. Restoration is a non-destructive process and none of the restored artifacts are deleted from the Cold instance.
You may run AQL queries only on the Item domain. Other primary domains, such as Build, Entry, Promotion, and Release are not supported. |
|
After initiating an archive or restore process, you can monitor its progress and view the status of archive and restore operations based on policy name and timeframe. The last execution will always be displayed on top.
In addition to using the Cold Artifact Storage UI and APIs to monitor the archive and restore processes, you can also review the Artifactory service logs to learn more about the status of archive and restore operations.
Archive Logs: To find and review the archive logs, in the Live Artifactory service logs , search for:
artifact_retention_archive_
prefix, which is appended to the migration and archive operation Ids, as shown in the image below
|
Restore Logs : To find and review the restore logs, in the Cold Artifactory service logs , search for:
artifact_retention_restore_
prefix, which is appended to the migration and restore operation Ids, as shown in the image below
|
These are some of the limitations that you may encounter when using Cold Artifact Storage, and any possible workarounds.
Limitation | In the event of an unexpected shutdown, archive policy execution might be interrupted. Due to this, the policy's corresponding operation record in the database will not be marked as done. |
Workaround | In this situation, items that were not archived will be picked up for archive during the next policy run. Stop policy execution before restarting the server. |
Limitation | If the restore path that the system generates during the restore operation exceeds this limit, the restore operation will fail. |
Workaround | No workaround. The maximum path length supported is 1024 characters. The maximum repository name length supported is 64 characters. |
Limitation | Restore by folder path is not supported. |
Workaround | Select all files that are to be restored. |
Limitation | When creating a retention policy, cannot save basic search criteria as AQL, even though preview is able to identify content. |
Workaround | Create and save the retention policy using the Create an Archive Policy API. |
Limitation | Archive policy cron expression validation not working. |
Workaround | Though the cron expression always appears as invalid in UI, you can still save it. So use an external cron expression validator to validate your cron expressions and then save the policy. |
Limitation | Cold storage retention policy does not persist Age settings. |
Workaround | This is a UI issue. Though the Age settings do not look like they persist, the real data is still correct and will work as expected when saved. |