The Private Distribution Network (PDN) is a collection of Edges, PDN Nodes and their groups that form a decentralized network for distributing content. The PDN architecture is based on a network of interconnected nodes located between the JFrog Artifactory or Edge server and the client (Docker or HTTP). A PDN Node acts as a caching proxy between the client and Artifactory/Edge.
The JFrog PDN architecture is comprised of these main components:
- Private Distribution Network (PDN) Server: A dedicated service responsible for advertising and tracking available artifacts located on the PDN Nodes, the PDN Server serves as the initial artifact seeder.
- PDN Node: A standalone application (or Docker image) that is interconnected with other PDN Nodes to resolve files, this application is used by the client to download content and is deployed on a dedicated host or client machine. The PDN Node communicates with its parent and sibling nodes to resolve files (Pull) and to distribute Release Bundles (Push). It also serves files to external clients from both cached artifacts and on-demand (Pull).
- PDN Node Group: A collection of interconnected nodes forms a PDN Node Group. Groups may be arranged in a hierarchical manner forming a tree of PDN Groups with a PDN Server at the root. A PDN Node may only download files using peer-to-peer from nodes within their own group, or from their direct parent group. Each group may have several child groups, but only one parent. A PDN Node's own group is referred to as SelfGroup, while a parent group is referred to as ParentGroup.
- Client: A software client used for interacting with the Node, for example a Docker or an HTTP client.
Load Balancer Support
While not part of the PDN architecture, you can also add a load balancer to this setup. The load balancer is a configuration that enables load balancing of incoming requests between PDN Nodes, enabling the division of the load between PDN Nodes within the same group, while also ensuring continuous cache synching between them.
In the example above, PDN Node1 and PDN Node2 both have the SelfGroup property "East", and so belong to the same group. PDN Node3 has the SelfGroup property "West" and so belongs to group "West". None of the PDN Nodes have the ParentGroup configured, and so they are placed by default directly under the PDN Server, which acts as their ParentGroup. PDN Node1 can download files only from PDN Node2 or from the PDN Server, whereas PDN Node3 may download files from the PDN Server only.
The PDN Server may be connected to a Source or Edge instance of Artifactory. In the example above, it is connected to a Source Artifactory.
In the example above, PDN Node4 to PDN Node7 belong to the the same group "NY" and are under the ParentGroup "East" and can resolve files between themselves and their ParentGroup.
The topology and hierarchy you define is dependent on your organization's needs. Therefore, before proceeding to set up PDN, we recommend that you map out your topology needs. Below are a few examples of topology scenarios you can achieve using PDN.
Scenario 1 - Multiple Groups + Single PDN Server + Single Source JPD
In this topology, there are multiple (5) PDN Groups each containing 1000 PDN nodes (groups A, B, C, D, and E), connected to a single PDN Server that is connected directly to a source JPD.
Scenario 2 - Multiple Groups + Single PDN Server + Distribution Edge + Source JPD
In this topology, there are multiple (5) PDN Groups each containing 1000 PDN nodes (groups A, B, C, D, and E), connected to a single PDN Server, that is connected to a single Distribution Edge that is connected to a single Source JPD.
Scenario 3 - High Availability Configuration: Multiple Groups with 2nd Level (Edge to PDN Node)
High Availability (HA) is achieved by connecting a Source JPD or Distribution Edge to multiple PDN Servers. Each server is connected to either multiple groups or to a PDN Group that is then connected to sub-groups.
In the example below, we have the following:
- A single Source JPD and two Distribution Edges, attached to each other.
- Each Distribution Edge is connected to two PDN Servers.
- Each PDN Server is connected to its own PDN Group.
- PDN Group A is also connected to another PDN Group (1) that contains 500 PDN Nodes.
Setup and Registration
- Install the PDN Server using one of the installation options.
- Connect the server to Artifactory or to an Edge.
Connect a PDN Node or multiple PDN Nodes to create a group.
You will need to install the PDN Node on a host that has network connectivity to the JFrog PDN Server or to any other nodes in the PDN.
The PDN Node connects to the PDN Server and retrieves the PDN Node parent candidates according to the ParentGroup property.
- Validate the connection.
PDN supports both on-demand pulling and push-based distribution, as described below.
Download Artifacts by the Clients (On-demand Pulling)
Clients can proactively request the artifacts from the PDN Node.
All download communication is SSL-encrypted using a certificate-chain created by Artifactory
- The PDN Node then listens for a client download request or requests from other PDN Nodes.
- As PDN Nodes download files on behalf of a client, they regularly advertise information about pieces of their stored files to the PDN Server. This information is then stored by the PDN Server.
- Once a PDN Node gets a download request from the client, it queries the PDN Server regarding which other registered PDN Nodes may have the file. The PDN Server responds and the PDN Node starts downloading the available parts of the file, from the relevant PDN Node.
- As files are cached on a PDN Node, they are advertised to the PDN Server and can be downloaded concurrently by other PDN Nodes, according to user permissions defined in Artifactory. The download file process is performed in parallel from multiple PDN Nodes thereby distributing the network load between multiple PDN Nodes to perform fast and efficient downloads from Artifactory.
Distribute Release Bundles (Push-based Distribution)
The PDN can be populated with the content proactively to warm-up the local cache on PDN Nodes.
Verify that you have Release Bundle Distribute permissions or have been assigned to a Project/Platform Admin role and that you have JFrog Distribution installed.
Distribute existing Release Bundles to the PDN Node. You can distribute:
- To a specific PDN Node Group.
- From the source Artifactory, distribute directly to the PDN Node groups that are connected to the source Artifactory of the Release Bundle version or to a Distribution Edge.
Rules and Guidelines
- Supported for Docker Open Container Initiative (OCI), and Generic packages.
- All PDN Nodes should be routed to access the root PDN Server.
- PDN Nodes in the same group should be able to access one another.