Cold Artifact Storage Architecture
For information about configuring Cold Artifact Storage, see Setting Up Cold Artifact Storage.
The Cold Artifact Storage archiving solution is comprised of three main components:
A Live Artifactory Instance
Live Artifactory is a standard Artifactory instance from which artifacts will be archived.
A Cold Artifactory Instance
Cold Artifactory is a stripped-down, limited-functionality version of an Artifactory instance that is used for archiving purposes. The Cold instance is implemented as a dedicated Artifactory instance, connected to a binary storage provider (such as AWS Glacier).
One Cold Artifactory instance can be connected to only one Live Artifactory cluster. Every cluster of Artifactory that needs to archive requires a single Cold Artifact Storage instance with a dedicated license.
An Archive: A Binary Storage Provider
When setting up the Cold instance, you will need to connect it to a binary storage provider. You can use one of the following storage options:
- A cloud provider, such as AWS Glacier
- A local storage provider
Cold Artifact Storage Features
Cold Artifact Storage includes a number of features that enable you to archive, search, restore and monitor the archiving process. In the following section, you will find a detailed overview of the Cold Artifact Storage features as well as information about the logs that are created for some of these functions.
Cold Artifact Storage enables admins to create archiving policies, which are used to decide which artifacts need to be archived. When the archiving policy is executed, artifacts are moved from the Live Artifactory instance to the Cold Artifactory instance, while the artifacts' binaries are stored in the storage connected to the Cold Artifactory.
The archiving policies can be configured using the following options:
Scheduled Execution: Scheduling enables you to configure the periodic execution of a policy that can be set to a daily, weekly, or monthly frequency. You can also define when (or after what period) the policy execution will be stopped, which enables you to perform archiving execution during weekends or maintenance windows.
If required, the archiving policy can also be executed manually.
- AQL-based Selection Criteria: You can define an AQL query that is executed to collect artifacts targeted for archival. The selection criteria can include:
- Folders and Repositories: Select artifacts located in specific folders within specific repositories
- Properties: Select artifacts based on specific properties, such as application versioning
- Age: Select artifacts based on usage and age by defining 'Last Downloaded before' and 'Deployed before'
After artifacts are successfully archived in the Cold instance, the archived artifacts and corresponding metadata are deleted from the original repository in the Live instance. If the archive process fails, the artifacts and corresponding metadata are not removed, and the incident is logged with the appropriate error messages.
As the archive process progresses, the system generates detailed logs in the Artifactory service logs in the Live Artifactory instance. If an email server has been configured, the system sends notifications to the admin after the archiving policy execution is completed.
Admins can create, view, edit, and delete archiving policies as the need arises using the UI or APIs. For more information, see Working with Cold Artifact Storage.
Only Artifactory admins can create and run archiving policies.
Cold Storage Archive Search
The search functionality enables admins to perform a dedicated AQL-based search from the Live Artifactory for artifacts that can be restored later.
You may run AQL queries only on the Item domain. Other primary domains, such as Build, Entry, Promotion, and Release are not supported.
Admins can use AQL to search within an archive based on the following criteria:
- Artifact name
- Artifact checksum
- Artifact properties
- Repository and path
- Archiving period
Archived Artifact Restoration
After performing a search for the archived artifacts, admins can trigger the restoration of artifacts from the Cold instance to the Live instance, by providing an explicit list of items. Restoration is a non-destructive process and none of the restored artifacts are deleted from the Cold instance.
When creating a request for restoration, the admin can choose one of the following destinations:
- Restore items to their original repository and provide a fallback repository (if the original repository was deprecated)
- Restore all the items to a dedicated repository
The system performs the restore operation in the background and updates the logs in the Artifactory service logs in the Cold Artifactory instance. After the restore operation is complete, the admin is notified through e-mail about the completion. This e-mail summarizes the operation.
- The admin must have an email server configured to receive email notifications.
- Depending on the number of artifacts being restored, the restore process can take a few minutes to a few hours.
- If the restore process encounters an error, the process fails and the corresponding error messages are logged accordingly.
Monitoring Cold Storage
Policy and Archived Artifacts Logs
Cold Artifact Storage adds dedicated logs in the Live Artifactory service logs for the events that are related to the artifacts being archived, including running archiving policies and the archive process itself. The following events are documented in the log:
- Create archiving policy
- Execute archiving policy execution
- Finish archiving policy execution, which includes the following information:
- A short summary about the execution, including information about when the process started and ended, and the success percentage
- The number of total processed artifacts, successfully archived artifacts, and artifacts that failed to be archived
- The status of each archived artifact
Restored Artifacts Logs
After restoring archived artifacts, Cold Artifact Storage adds dedicated logs in the Cold Artifactory service logs for the events related to the artifacts being restored. Dedicated logs are created for the following events:
- Start restore process: Each time the restore process starts, a log is created with the size, number of artifacts, and approximate retrieval process end time
- Finish restore process
- Status for each restored object
Monitor Archive and Restoration Operations
Because both the archive and restore operations take place in the background, Cold Artifact Storage provides a monitoring functionality that enables you to track the status and progress of these operations. The monitoring process provides the following information:
- Summary of the archived artifacts with status: Provides the status and summary of the number of archived artifacts and their size
- Can be filtered by 'from' and 'to' dates
- Can be searched using a policy key
- Summary of restored artifacts with status: Provides the status and summary of the number of restored artifacts and their size
- Can be filtered by 'from' and 'to' dates
Now that you understand the setup of the Cold Artifact Storage solution, you can begin configuring your system to enable Cold Artifact Storage.
After Cold Artifact Storage is configured, proceed to Working with Cold Artifact Storage to continue the setup.