Performance Enhancements
You can use the following options with the migration in the migrate-artifacts command to increase the speed of migration and manage the network bandwidth.
concurrent-files- Specifies the number of files that the migration tool handles at a time. The migration tool spawns threads equivalent to the value. If your network bandwidth supports a large number of connections and the system resources in the source machine can handle many threads, specify a high value to handle several files at the same time. The default value is 8.skip-statistics- Skip the migration of the artifact download statistics. If you have a large number of artifacts, the download statistics can be a significant amount of data. You can skip the statistics migration if you want to reduce the migration load. If you set the option as true, you will lose the historical download statistics for all the artifacts.overwrite- Overwrite existing files in the target environment when you run the migration tool again. When you enable this option, the speed of migration can improve in certain situations.
Ensure that you configure these parameters according to your system resources and network capacity.
When you run the migrate-artifacts command, the migration verifies whether you have enough space to handle the temp cache in the source machine with the formula, size of the largest artifact * number of threads. If you have a very large artifact and you configure a large number of threads, the check to see whether there is enough space can take a lot of time.
You can set the environment variable, JFROG_MIGRATION_TOOL_DISABLE_SIZE_CHECK, to false before you run the migrate-artifacts command to bypass the size check. You must ensure that you have enough space of your disk before you run the migration tool.
Performance Benchmarks
Scenario 1
The following table provides information on the source system that hosts Artifactory.
CPU | Memory | Storage Type | Number of Repositories | Artifact Count | Binary Size |
|---|---|---|---|---|---|
12 cores | 24 GB | File System | 1 | 500000 | 1.2 TB |
The migrate artifacts command was run with the options, overwrite and skip-statistics, set as false.
You can configure the number of threads by setting the value for the concurrent-files option in the migrate artifacts command.
The following table lists the time taken against the number of threads used by the migration.
Number of Threads | Time |
|---|---|
20 | 6 hours 43 minutes |
50 | 3 hours 40 minutes |
100 | 3 hours 40 minutes |
Scenario 2
The following table provides information on the source system that hosts Artifactory.
CPU | Memory | Storage Type | Number of Repositories | Artifact Count | Binary Size |
|---|---|---|---|---|---|
12 cores | 24 GB | File System | 1 | 500000 | 1.2 TB |
The migrate-artifacts command was run with the options, overwrite and skip-statistics, set as true.
You can configure the number of threads by setting the value for the concurrent-files option in the migrate artifacts command.
The following table lists the time taken against the number of threads used by the migration.
Number of Threads | Time |
|---|---|
20 | 5 hours 4 minutes |
50 | 3 hours 58 minutes |
100 | 3 hours 38 minutes |
