How Caching Works
Caching is performed through utility functions that store and restore data to and from the Artifactory filestore. In this way, a step can benefit from the dependencies that were installed or loaded from a previously executed run.
Cache File Scope
A cache file that a step stores using the
add_cache_files function will be associated with that step, and will only be available to a
restore_cache_files function from that same step in a subsequent run.
You cannot pass a cache file to other steps in a pipeline. For example, if
adds the cache file
tries to restore
step_1_cache, nothing will be loaded.
Steps running in parallel can overwrite the cache. So, steps trying to update step or pipeline state in parallel will result in only the last version of the state saved for the step/pipeline being available to future steps.
In general, the Artifactory filestore provides the highest available performance for storing and restoring data.
However, the speed you experience will depend on which storage medium Artifactory has been configured to use. If Artifactory has been configured to use the file system on a local or mounted filestore, this is fast storage and caching will always accelerate step execution. If Artifactory has been configured to use remote storage such as S3 or Google Cloud Storage, then the slower roundtrip to and from the filestore may diminish the usefulness of caching:
- Files that take a long time to install always benefit from caching. So anything related to bundler, npm, composer, pip, etc are great candidates for caching.
- Files that take a long time to download but are installed quickly do not benefit from caching since it takes as much time to download from S3 as from the original source. Examples are compiled binaries, JDK packages, etc.
The following example caches the results of an
npm install for subsequent runs.
- The step's
onExecuteblock performs a
restore_cache_filesfunction to load the cached npm dependencies if they are available from a previous run. If none exist, no error will result, so the remainder of the step will execute without interruption.
- When the
npm installis run, it will recognize if the dependencies are already present from the cache, so the step will execute more rapidly. If there was no cache to load, then the npm dependencies will be installed.
- When the step is complete, it will always write the npm dependencies to the cache so they will be available to the step in the next run of the pipeline.