Cloud customer?
Start for Free >
Upgrade in MyJFrog >
What's New in Cloud >





Overview

Private Distribution Network (PDN) is a lightweight, storage-savvy distribution solution that enables you to meet your growing distribution needs. To learn more, see Private Distribution Network. The Private Distribution Network (PDN) setup is based on the following stages:

  1. Install the PDN Server.

  2. Install a PDN Node(s).

    Deploying the JFrog PDN Nodes

    You will need to deploy your PDN Nodes in the network according to the deployment strategies within your organization.

    When the PDN Nodes connect to the PDN Server, they will be populated with the default PDN Node configurations that are stored in the JFrog Platform. These configurations are verified dynamically each time the PDN Nodes connect to the PDN Server. They can be modified by pushing an additional YAML file using the REST API, uploading to the UI, or by bootstrapping from the file system.

    For more information, see Advanced Private Distribution Network (PDN) Configurations.

  3. Set up your PDN by following the four steps below:
    1. Step 1: Establish trust between the JFrog Platform Deployment (JPD) and PDN Nodes.
    2. Step 2: Customize the PDN.
    3. Step 3: Configure the PDN Node as an HTTP/HTTPS proxy.

    4. Step 4: Configure Nginx to support load balancing between PDN Nodes

  4. Optional Advanced Setup: You can use the default settings that are set during the installation to use PDN; however, you can also fine-tune your setup using the optional advanced settings.

PDN License Requirements

When a JFrog Artifactory license expires, Artifactory continues to work (for example, customers can upload and download components). However, because PDN is an advanced functionality, it will be blocked once the instance license has expired.

If your Artifactory license expires, you will receive a failure response of code 503 (service unavailable) with a different error message relating the action you were trying to take:

  • The Topology view will be disabled - when you try to access this view, a message will appear explaining that the view is disabled

  • There is no Eager cache warmup

  • Client pulls/downloads will not be possible

Important

To continue using PDN, you will need to obtain a valid license. However, because PDN samples Artifactory for the license every 5 minutes, the longest that PDN will continue to be disabled after the license is provided is 5 minutes. 



Setting up the Private Distribution Network

Once you have completed the installation of the PDN Server and PDN Node. you are ready to set up your PDN.

The basic process of setting up the PDN Nodes comprises these main steps:

  1. Download the JFrog PDN Server and customize the basic PDN Node YAML file. 
  2. Configure the PDN Node as an HTTP/HTTPS Proxy.

Step 1:  Establish Trust between the JFrog Platform Deployment (JPD) and PDN Nodes

To create trust between the JFrog Platform Deployment (JPD) and a PDN Node, the PDN Node can use any of the following authentication methods:

  • Token-based API authentication.
  • Mutual TLS (mTLS) Authentication.

Authentication is specified in the PDN Node configuration YAML file.

Token-based API Authentication

There are two methods for authenticating using a token:

  • To use a token-based API key authenticationconfigure the joinKey attribute in your customized PDN Node YAML file.
  • To generate an API key (an Access Token)
    1. In the JFrog Platform, go to Admin | User Management | Settings | PDN Access Token., generate and copy the key. 
    2. To view the join key, click the Show join key icon.


Mutual TLS Authentication 

To use Mutual TLS Authentication configure pdnnode.PEMFilePath attribute to specify the path to the local .pem file that contains the certificate for SSL client authentication and the client certificate key. 

By default, PDN Node uses the list of trusted certificate authorities (CA) from the operating system where it is running.
If the certificate authority that signed your certificates is not in the host system’s trusted certificate authorities list, you need to add the path to the file that contains your CA’s certificate by defining the  pdnnode.certificatAuthorityPath attribute.

Sample File
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAs3HKxB+AwcqQxxfiKWI0lmGxR/c1gigGMoEuqx5dTWNpG4Ow
ypOITOndzvKFE5HJXzwYoqQJnjjOsqI+5mJ2S1ZsVyLEKfPM5O60kfN1nBTd/5Ex
FSiKfmjJchRTrRNe6kHvn5SnBnIWm3sj3/w7pkBnkhv2tNbpzUdAwxOwi886QPKw
Q7pGp8EM9DfEMqtz37Zv5dLNFDqIiQkMEAjJQBEuW8WnE9vqNd8o1unvPZeZUps/
nQCaqPvD7aonNKY4GLJPtaaAY4DZcF/ndj/s/HbWOTbQCUk5sGVzqUWJUw4IYkgc
IBVFlnWcPGQ1Xvofi1H4N1uAs/UgzBOk9dthzwIDAQABAoIBAH71Y6CfKVnzF7vF
Ch59fPdo5yUjopgT+U6CV6J6OUknrJXq9opdF3JThLuGmuDdpLvcqRFpPeTUDGoW
oKofv9JNsxlzX5bPB4hx2w/oUK29ldPbxf5SUKpfOSedehZaPi0/uWoOop3M2nHn
DdrLeuiFaMHXRxVreL/kSc27gI1pQOD++W2p05uXiE8V+Sn3fDq4x4R060aIOB2F
O2ptXjWFRJu0qsDulrUumdnxuXWFI+K3RZ2hoFVvNj31M9gICuIwU5L8bE+HjPbr
9MqVR1eg+8mhdXZaZtY/vGqb1U0ZGWBBH8cpm45NEueT8jKCpLiXIUmrE1p2V0mM
Vws1rAECgYEA4yznuBfcuYjC3tbs1fdRZgoL9XKBoNH9ilC5gGbTSTe7UQsHUXaN
DTv4jZ86YW6EF6l+nVQAwGfD27rvNycTCdKv92tHimwIw8GOLNHFCi17SpAlrrji
FSfHf5GUtd6spmbZHKbNrCHCxJ9bTXV29BjIGiBUq9Zkx4cuDselhdECgYEAyjZ+
uKz+73+SY0Gyam8GdAcx+X53mdJL11HxauPQ7Nj3utvSncE1pSBi5slgpoDRSlGC
LNvGcePZUnuiGulhKoHAU1NLACXTF0gIyVvPxFly3qNttkuLjZAAQfLPCaTi2UYF
K5is/rwG7qTFRjhe0siZ+oetGQfpeIp8dm0JNZ8CgYBbgVT85C+2VA4acHwJW4tg
DJ/wh+ULAEqeEhIGVGS600YSSNzzxUrfj854TTDY2gk+9rjXamS7G5DayxtcHdMf
W8whXwb8DLQjFa3NA9nYVaWZzSJAlJwMZW+IRP2fnEfy5u6BCfdCZH7tcDr4ol39
yJWhYHB6nusNK7gzn9teYQKBgHn1qjJqLIx4jesNibVF50d9u3d1/l6pBpMEWi4U
30KqjaeGw53gFSHVybDWTlO+J/zB8lhp6WNxf/aOmLPQhIklnsDi4L5jEdDd4mbH
2CTdqjl7qkUjhTTJYE5mStbPOv6+i4nC2Lv+3dN7T3xNrWtGEBCZWh5Ztke8htrK
Yp6TAoGBALxArpy2uUkfMnV53gVXz5+O/lq9p42zKiX++D3tb67TqGGIUUobvDpO
Jlp2ZnZlqfPR9jSvKSN4LhY/KmRlqA4EihvRMph5eUSVILllznIT1hs0vuvi6jJ6
K6oknsjjn+MQldHJ3Jz+6wZHxHmqZD9VB96oo0WIaaR2dTn7gZYf
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDmDCCAoCgAwIBAgIUN2Q/afFhsleutFOYPNhhJNkWsjAwDQYJKoZIhvcNAQEL
BQAwfjELMAkGA1UEBhMCSUwxDzANBgNVBAgMBklzcmFlbDERMA8GA1UEBwwIVGVs
LUF2aXYxDjAMBgNVBAoMBUpmcm9nMQwwCgYDVQQLDANQRE4xDjAMBgNVBAMMBUpm
cm9nMR0wGwYJKoZIhvcNAQkBFg5hZGlnQGpmcm9nLmNvbTAeFw0yMTA5MTUxMDQy
NTdaFw0yMzEyMTkxMDQyNTdaMEAxCzAJBgNVBAYTAklMMQ8wDQYDVQQIDAZpc3Jh
ZWwxEDAOBgNVBAcMB3RlbGF2aXYxDjAMBgNVBAoMBWpmb3JnMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs3HKxB+AwcqQxxfiKWI0lmGxR/c1gigGMoEu
qx5dTWNpG4OwypOITOndzvKFE5HJXzwYoqQJnjjOsqI+5mJ2S1ZsVyLEKfPM5O60
kfN1nBTd/5ExFSiKfmjJchRTrRNe6kHvn5SnBnIWm3sj3/w7pkBnkhv2tNbpzUdA
wxOwi886QPKwQ7pGp8EM9DfEMqtz37Zv5dLNFDqIiQkMEAjJQBEuW8WnE9vqNd8o
1unvPZeZUps/nQCaqPvD7aonNKY4GLJPtaaAY4DZcF/ndj/s/HbWOTbQCUk5sGVz
qUWJUw4IYkgcIBVFlnWcPGQ1Xvofi1H4N1uAs/UgzBOk9dthzwIDAQABo0wwSjAf
BgNVHSMEGDAWgBTAv1IjT+kx/YRCOZTXfcXsERdDmDAJBgNVHRMEAjAAMAsGA1Ud
DwQEAwIE8DAPBgNVHREECDAGhwSsHyocMA0GCSqGSIb3DQEBCwUAA4IBAQCpQvGE
IgptR7FHhLYILG+aIq0Kg2QhGdt3gCAgx6jnncnogQWyTb4t7tB2OCXuif+FG3lh
9xkKrb3AX0w+rKDiL3HDG+kqH/ll9NveFCCT1AgNaku+mx6CtcmWpn+JoW2nvpkF
JAIEaZF9BSpQcpveRIM9PQzKiyeeexQYQo4N9IUVNVjrKVh10OexdxnKnWBCeAFn
tec7icwmvoU4IPYrp7L0jmpUEvH0Wm6hjsYUa1LiwHcRIFCt6qV1mZWjm3E6oFkj
sewgkO3e6j7l6n7vIKM28ps6azmOTqtW/QMEmzjT+237atM46j11mnXsOidLoaNW
6DYeXsGLrxNPLuEL
-----END CERTIFICATE-----

Step 2:  Customize the PDN

Establishing Connectivity between the PDN Nodes

Upon startup, the PDN Nodes need to establish connectivity with the other PDN Nodes through a secure gRPC connection by applying an access root certificate. The PDN Node can retrieve this certificate using one of the following:

  • Using mTLS Authentication (see above)
  • A digital envelope through an insecure connection.
  • A certificate located on the PDN  local machine by setting the pdnnode.accessRootCertificate attribute.
  1. Choose your PDN Node installation.
  2. Customize a PDN file with the Static Node Attributes yaml file for the basic static attributes including addresses, ports, certificates, and directories. 

    Using Environment Variables

    You can also configure the PDN Application using the environment variables starting with the JF_PDNNODE prefix.

  3. Deploy the YAML with the Static attributes and run the JFrog PDN Application on all of the PDN Nodes.

    When the PDN Nodeinitially connect to the PDN Server, they will automatically be populated with the default PDN Nodes configurations that are centrally stored in the JFrog Platform. These configurations are updated dynamically every time the PDN Nodes connect to the PDN Server and can be modified using the REST API, directly in the UI, or through the bootstrap from the file system.
    To modify the dynamic node attributes, see Advanced Private Distribution Network (PDN) Configurations.

Step 3: Configure the PDN Node as an HTTP/HTTPS Proxy

To provide you with a seamless experience when working opposite Artifactory via the PDN Nodeyou need to set the PDN Node as an HTTP/HTTS Proxy. After you have set up the PDN Node as a proxy, all the requests sent from the Docker client will be routed through the PDN Node. For more information, see Getting Started with Artifactory.

  1. Set the location on your PDN to the following path.

    $JF_PRODUCT_HOME=[path on your machine]
  2. Copy your customized PDN Node system.yaml to the following path. 

    $JF_PRODUCT_HOME/var/etc/system.yaml
  3. Run the PDN Node with your preferred installation.

  4. Copy the certificates with a .crt extension located under $JF_PRODUCT_HOME/var/etc/security/certificates to the folder in which your operating system runs the Docker Daemon.
    For example, Alpine users can copy their files to the following location in which the Docker Daemon has been designated to run the files. 

    apk add ca-certificates
    cp ./certificates/jfrog_proxy.crt /usr/local/share/ca-certificates/jfrog_proxy.crt
    cp ./certificates/jfrog_access_root.crt /usr/local/share/ca-certificates/jfrog_access_root.crt
    update-ca-certificates
  5. Start the Docker Daemon with an HTTP proxy pointing to the PDN Node self HTTP address.
    As displayed in the following example, you can run the Docker Daemon directly.

    HTTPS_PROXY=selfHttpAddress dockerd

Step 4: Configure Nginx to Support Load Balancing Between PDN Nodes.

Load Balancing allows you to set up your PDN in a way that provides high availability with high performance, which is critical for service quality, with minimal configuration. 

PDN retrieves files by accessing a specific PDN Node in a specific PDN Group according to the group's IP address. Because each PDN Node in each group is dynamic, meaning a PDN Node can be shut down or started at any time, it is more efficient to work with a small subset of PDN Node IPs rather than managing them all. This can be achieved by configuring the Nginx configuration template with all the active PDN Nodes in a group, thus improving the high availability and reducing the management of PDN Nodes IPs. The PDN Nodes within the same group will balance the load of incoming requests between each other, while continuously synching the cache for optimal performance.

Configuration Steps

Follow the steps below to configure an Nginx node.

  1. Run the following REST API request.

    GET /api/v1/template

    The configuration retrieved will include the IP addresses of all the PDN Nodes in the group. The HTTP port will be different from the direct communication port, as this new port will expose the PDN Node with a load balancing functionality.

    upstream backend {
        server node1:8090;
        server node2:8090;
        server node3:8090;
    }
    server {
        location / {
            proxy_pass http://backend  
       }
    }
  2. Create a new configuration file under /etc/nginx/conf.d with the file ending *.conf.
  3. Use the output generated by the REST API request and set the Nginx server as a reverse proxy to the PDN Nodes. 
    Make sure to use the exact output content, and verify that there are no other configuration files that might affect this configuration. Note that the default .conf file that also configures the reverse proxy settings must be removed or modified. 

Advanced Setup

Working with Docker Strategies

To support the different Docker resolution strategies, you will need to apply the following PDN Node rewrite mechanism. For more information, see Using a Reverse Proxy. The following examples illustrate the supported strategies.

Supported Strategies

Repository Path Mapping

This is the default configuration supported by the PDN Node. The repository path method allows a single point of entry (URL) to access different repositories. This is done by embedding the name of the repository being accessed into the image path.

If you want to add support for resolving Docker images without specifying a repository path, follow the rewrite rules as displayed.

pdnnode:
     rewrite:
        - 
          from: https://registry-1.docker.io/v2/library/(.*)
          to: https://<myartifactory.com>/v2/<default Docker repository>/$1

Subdomain Mapping

To support subdomain mapping, configure the rewrite tool based on your configuration. For example, if you are using the Nginx configuration used in Artifactory, follow these rewrite rules.

pdnnode:
     rewrite:
        - 
          from: https://(.*).<myartifactory.com>/(.*)
          to: https://<myartifactory.com>/artifactory/api/docker/$1/$2

Port Mapping

To support port mapping, you will have to map the port to the required repository by following these rewrite rules.

pdnnode:
     rewrite:
        - 
          from: https://<myartifactory.com>:<port1>/(.*)
          to: https://<myartifactory.com>/artifactory/api/docker/<repokey1>/$1 //repokey relates to the port we want to map to this key 
        
        - 
          from: https://myartifactory.com:<port2>/(.*)
          to: https://myartifactory.com/artifactory/api/docker/<repokey2>/$1 //repokey relates to the port we want to map to this key

        - 
          from: https://myartifactory.com:<port3>/(.*)
          to: https://myartifactory.com/artifactory/api/docker/<repokey3>/$1 //repokey relates to the port we want to map to this key

Downloading Files from the PDN Node Using Docker Client

Validate that you have a Docker repository in Artifactory.

Running Docker commands in the Docker client through the PDN Node is the same as running Docker commands opposite Artifactory. Your permissions and credentials for downloading files are identical to your permissions in Artifactory and are retrieved when performing actions opposite the PDN Node.

Docker Login Request Examples

For the Docker client to connect to the PDN Node and download through this nod, you will need to run docker login to the PDN Node.
Note that HOST_IP refers to your machine's IP address.

  • To run a Docker login command using the repository path:

    $ docker login --username admin --password password1 <artifactory_URL>
    Example: Docker Login using the repository path
    $ docker login --username admin --password password1 myartifactory.com
  • To run a Docker login command using using the subdomain:

    $ docker login --username admin --password password1 <repokey>.<artifactory_URL>
    Example: Docker Login using the subdomain
    $ docker login --username admin --password password1 docker-local.myartifactory.com
  • To run a Docker login command using the port:

    $ docker login --username admin --password password1 <artifactory_URL>:<port>
    Example: Docker login using the port
    $ docker login --username admin --password password1 myartifactory.com:9922

Docker Pull Request Examples

Running a Docker Pull command using the repository path.

$ docker pull <artifactory_URL>/<repokey>/<image_name>:<version>
Example: Docker Pull using the repository path
$ docker pull myartifactory.com/docker-local/mysql:latest

Running a Docker Pull command using the subdomain.

$ docker pull <repokey>.<artifactory_URL>/<image_name>:<version>
Example: Docker Pull using the subdomain
$ docker pull docker-local.myartifactory.com/mysql:latest

Running a Docker Pull command using the port.

$ docker pull <artifactory_URL:port>/<image_name>:<version>
Example: Docker Pull using the port
docker pull myartifactory.com:9922/mysql:latest

Docker Push Request Example

Running a Docker Push command using the repository path.

$ docker push <artifactory_URL>/<repokey>/<image_name>:<version> 
Example: Docker Push using the repository path
$ docker push myartifactory.com/docker-local/mysql:latest


Next: Working with the Private Distribution Network (PDN) >>

Optional: Advanced Private Distribution Network (PDN) Configurations >>


  • No labels
Copyright © 2022 JFrog Ltd.