Cloud customer?
Start for Free >
Upgrade in MyJFrog >
What's New in Cloud >





Overview

The topic provides information on how you can improve the performance of the JFrog Cloud Migrator tool based on your network bandwidth and platform setup. The topic also provides some performance benchmarks.

Page Contents

Performance Enhancements

You can use the following options with the migration in the migrate-artifacts command to increase the speed of migration and manage the network bandwidth.

  • concurrent-files - Specifies the number of files that the migration tool handles at a time. The migration tool spawns threads equivalent to the value. If your network bandwidth supports a large number of connections and the system resources in the source machine can handle many threads, specify a high value to handle several files at the same time. The default value is 8.

  • skip-statistics - Skip the migration of the artifact download statistics. If you have a large number of artifacts, the download statistics can be a significant amount of data. You can skip the statistics migration if you want to reduce the migration load. If you set the option as true, you will lose the historical download statistics for all the artifacts.
  • overwrite - Overwrite existing files in the target environment when you run the migration tool again. When you enable this option, the speed of migration can improve in certain situations.

Ensure that you configure these parameters according to your system resources and network capacity.

When you run the migrate-artifacts command, the migration verifies whether you have enough space to handle the temp cache in the source machine with the formula, size of the largest artifact * number of threads. If you have a very large artifact and you configure a large number of threads, the check to see whether there is enough space can take a lot of time.

You can set the environment variable, JFROG_MIGRATION_TOOL_DISABLE_SIZE_CHECK, to false before you run the migrate-artifacts command to bypass the size check. You must ensure that you have enough space of your disk before you run the migration tool.

Performance Benchmarks

Scenario 1

The following table provides information on the source system that hosts Artifactory.

CPU

Memory

Storage Type

Number of Repositories

Artifact Count

Binary Size

12 cores

24 GB

File System

1

500000

1.2 TB

The migrate artifacts command was run with the options, overwrite and skip-statistics, set as false.

You can configure the number of threads by setting the value for the concurrent-files option in the migrate artifacts command.

The following table lists the time taken against the number of threads used by the migration.

Number of Threads

Time

20

6 hours 43 minutes

50

3 hours 40 minutes

100

3 hours 40 minutes


Scenario 2

The following table provides information on the source system that hosts Artifactory.

CPU

Memory

Storage Type

Number of Repositories

Artifact Count

Binary Size

12 cores

24 GB

File System

1

500000

1.2 TB

The migrate-artifacts command was run with the options, overwrite and skip-statistics, set as true.

You can configure the number of threads by setting the value for the concurrent-files option in the migrate artifacts command.

The following table lists the time taken against the number of threads used by the migration.

Number of Threads

Time

20

5 hours 4 minutes

50

3 hours 58 minutes

100

3 hours 38 minutes

  • No labels
Copyright © 2022 JFrog Ltd.