Skip to end of metadata
Go to start of metadata
Table of Contents

Background

Together with the growing number of choices for build-tools and frameworks there are also many opinions for how modules are stored within a repository.

Initially, Artifactory supported the Maven layout conventions for dealing with modules (and relying on Maven-specific metadata). However, your build tool should be able to "talk" to the repository "naturally", so if you are using Ivy or Gradle, there is no need to configure them to use the Maven conventions in order to "fit in".  Moreover, combining and chaining repositories that use different layouts should work out-of-the-box.

This is where the Repository Layouts Add-on comes into play!

The Freedom of Custom Layouts

With the Repository Layouts Add-on you are no-longer limited to Maven-centric module handling - Artifactory allows you to take full control over the layout used by each repository and use layout definitions to identify module artifacts and descriptors. By using repository layouts, Artifactory can offer these smart module management facilities for any build technology:

  • Automatic snapshot/integration versions cleanup
  • Deleting old versions
  • Conversions between remote and local layouts
  • Conversions between 2 local layouts when moving or copying
  • Resolution conversions from a virtual repository to its underlying repositories (where the virtual repository has its own layout defined)

Bundled Layouts

Out-of-the-box Artifactory comes with a number of default predefined layouts requiring no additional configuration:

  • Maven 2/3
  • Ivy (default layout)
  • Gradle (Wharf cache default layout)
  • Maven 1

Support for repository layouts in Artifactory OSS

Layout configuration for conversion and resolution is available only to Artifactory Power Pack users. Users of the OSS version can only Configure their Repositories to use the default repository layouts bundled with Artifactory.

The OSS version only supports the automatic snapshot/integration version cleanup and deleting old version features.

Modules and Path Pattens used by Repository Layouts

Module Fields

To support smart module management, Artifactory must construct module information for stored files. Artifactory construct this information based on path pattern information defined as part of Repository Layout configuration (detailed below).

A module is comprised of various sub-elements or fields, which are typically expressed in the path of a stored artifact.

The module-sub elements recognized by Artifactory are listed below. At a minimum, there are three  mandatory fields required for module identification:

  • Organization
  • Module
  • Base Revision.

 

Field

Description

Example

Mandatory

Organization

A sequence of literals that identifies the artifact's organization

"org.slf4j"

(tick)

Module

A sequence of literals that identifies the artifact's module

"slf4j-api"

(tick)

Base Revision

A sequence of literals that identifies the base revision part of the artifact version, excluding any integration information

"1.5.10", or in case of an integration revision "1.2-SNAPSHOT" the base revision is "1.2"

(tick)

Folder Integration Revision

A sequence of literals that identifies the integration revision part used in folder names in the artifact's path, excluding the base revision

in case of an integration revision "1.2-SNAPSHOT" the folder integration revision is "SNAPSHOT"

(error)

File Integration Revision

A sequence of literals that identifies the integration revision part in the artifact's file name, excluding the base revision

in case of an integration revision "1.2-20110202.144533-3" the file integration revision is "20110202.144533-3"

(error)

Classifier

A sequence of literals that identifies the artifact's classifier

"sources"

(error)

Extension

A sequence of literals that identifies the artifact's extension

"zip"

(error)

Type

A sequence of literals that identifies the artifact's type.
Typically used when the artifact's extension cannot be reused as the artifact's type

"java-source"

(error)

Using Module Fields to Define Path Patterns

A path pattern is used in the configuration of a Repository Layout.

The pattern is similar to that of the Ivy pattern and is used to define a convention for artifact resolution and publication paths.

Artifactory uses path patterns to construct module information for stored files. This module information is then used to facilitate all the features mentioned above (version cleanup, cross-repo path conversions, etc.).

Path Pattern Tokens

A path pattern is constructed of tokens (explained below), path separators ('/'), optional parentheses ('(' and ')') and literals ('.', '-', etc.). Tokens are modeled after module fields, as presented above.

Path patterns can be defined for every artifact in the repository or you can define a separate path patterns for descriptor-type artifacts (such as, a .pom or an ivy.xml file).

The following tokens are available:

Token

Description

[org]

Represents the Organization field where the levels are separated by dots ('.'), a-la Ivy. For example: "org.slf4j"

[orgPath]

Represents the Organization field where the levels are separated by path separators ('/'), a la Maven. For example: "org/slf4j"

[baseRev]

Represents the Base Revision field

[module]

Represents the Module field

[folderItegRev]

Represents the Folder Integration Revision field

[fileItegRev]

Represents the File Integration Revision field

[classifier]

Represents the Classifier field

[ext]

Represents the Extension field

[type]

Represents the Type field

[customTokenName<customTokenRegex>]

A custom token. Can be used to create a new type of token when the provided defaults are insufficient.

For example, [myIntegRev<ITEG-(?:[0-9]+)>] creates a new custom token named myIntegRev that matches the word ITEG followed by a dash and a minimum of one digit.

Multiple Custom Tokens

Artifactory supports any number of custom tokens, but when provided with multiple custom tokens of the same key, Artifactory only takes into account the regular expression of the first occurrence while substituting the rest with a repetition expression (even if each occurrence has a different regular expression value.

For example:

Translates to:

Optional parts

To specify tokens or a sequence of tokens and literals as optional in the path pattern, surround the sequence with the optional parentheses '(' and ')' lterals.

For example, the pattern "[module](-[classifier])" matches both "bobs-tools-sources" and "bobs-tools", and the pattern "[baseRev](-[fileItegRev])" matches both "1.2-SNAPSHOT" and "1.2".

Artifact Path Patterns

An artifact path pattern represents the typical structure that all module artifacts are expected to be stored in.

For example -

  • To represent a normal Maven artifact path: "org/eclipse/jetty/jetty-ajp/7.0.2.v20100331/jetty-ajp-7.0.2.v20100331.jar"

    Use the artifact path pattern:

  • To represent a default Ivy artifact path: "org.eclipse.jetty/jetty-ajp/7.0.2.v20100331/jars/jetty-ajp-7.0.2.v20100331.jar"

    Use the artifact path pattern:

Descriptor Path Patterns

A descriptor path pattern is used to recognize descriptor files (like .pom or ivy.xml files).

Using a specific descriptor path pattern is optional. When not used, Artifactory constructs module information for descriptor file using the artifact path pattern.

 Even though descriptor paths patterns are optional, usage of them is highly recommended in cases of distinctive descriptors, such as Ivy ivy_-1.0.xml and Maven bobs-tools-1.0.pom.

For example -

  • To represent a normal Maven descriptor path: "org/eclipse/jetty/jetty-ajp/7.0.2.v20100331/jetty-ajp-7.0.2.v20100331.pom"

    Use the descriptor path pattern:

  • To represent a default Gradle descriptor path: "org.eclipse.jetty/jetty-ajp/ivy-7.0.2.v20100331.xml"

    Use the descriptor path pattern:

Configuration

Repository layouts are configured on the global level of your Artifactory instance, so that any layout can be shared and reused across any number of repositories.

Layout Configuration

Layout configuration is available to administrator users from the Admin tab and then Configuration -> Repository Layouts.

Additional layouts can be created from scratch by clicking the "New" button or by duplicating an existing layout by clicking the "Copy" button on a selected layout.

Path Patterns

Define the artifact path pattern and the descriptor path pattern (optional), as explained above.

Use patterns in the directory part of the path

To achieve best path matching results, it is highly recommended that artifact and descriptor patterns also contain the mandatory tokens ([org] or [orgPath], [module] and [baseRev]) within the directory structure itself.
For example, Gradle's artifact path pattern:

File and Folder Integration Revision Regular Expressions

These fields should contain regular expressions that exactly match and describe the integration revision (excluding the base revision) formats as they are expected in the artifact's file name and path-structure folder name.

Avoid using capturing group syntax in regexp

Regular expressions entered in these fields are wrapped and treated as a single capturing group.

Refrain from introducing any capturing groups within the expressions. Failure to do so may result in unexpected behavior and compromise the accuracy of the matcher.

Folder integration revision regular expression examples:

  • Maven's folder integration revision is simply the constant -SNAPSHOTappended to the base revision ("1.2-SNAPSHOT"), so the regular expression is

  • Ivy's default folder integration revision is usually equal to the file integration revision and is normally a 14 digit timestamp ("5.1-20101202161531"), so the regular expression can be

File integration revision regular expression examples:

  • Maven's file integration revision can be either the -SNAPSHOTconstant ("1.2-SNAPSHOT") or a timestamp, where the date and time are separated by a dot ('.'), with an addition of a dash ('-') and a build-number ("2.3-20110108.100922-2"), so the regular expression should be able to fit them both

  • Ivy's default file integration revision is is normally a 14 digit timestamp ("5.1-20101202161531") and usually equal to the folder integration revision, so the regular expression may be the same as suggested in the file's example

Testing Layouts

Now that you have finished configuring your layout, you can test it out via the user interface on an artifact or a descriptor path, and see how Artifactory would build module information from the path, using the layout definitions.

Repository Configuration

Local Repository Configuration

Layouts are mandatory for local repositories, since they define the structure of the artifact storage.

NOTE! that repositories configured prior to the introduction of custom repository layouts are auto-configured with the default Maven 2 layout.

Remote Repository Configuration

Layouts are mandatory only for the remote repository cache configuration. However, you can also define a layout of the remote repository.

In this case, if the remote repository itself uses a different layout than the one chosen for the cache, all requests the to the remote target are translated from the path of the cache layout to the path of the remote layout.

For example, the remote repository http://download.java.net/maven/1 stores its artifacts according to the Maven 1 convention; you can configure the cache repository to use Maven 2's layout so that it handles Maven 2 requests and artifact storage, and configure the remote target to use Maven 1's layout so that outgoing requests are translated to Maven 1's convention.

NOTE! that caches of remote repositories configured prior to the introduction of the repository layouts are auto-configured with the default Maven 2 layout.

Virtual Repository Configuration

You can also configure a layout for a virtual repository.

When configured, all resolution requests can be made according to the virtual repository layout. When trying to resolves requests to the virtual repository Artifactory attempts to translate the request path to the layout of each nested repository, according to the module information constructed from the virtual request.

 Request path translation is not made and requests pass through to nested repositories with their original path in any of the following scenarios:

  • No module information can be constructed
  • The virtual module information cannot be mapped to a nested repository (missing fields on one of the side)
  • The virtual repository or the nested repository are not configured with a layout

  • No labels