Have a question? Want to report an issue? Contact JFrog support

Skip to end of metadata
Go to start of metadata

Overview

Many customers are using Xray as their decision making tool when it comes to ensuring that their software is secure and compliant. The newest Artifactory and Xray integration makes Xray a mission critical service, as every artifact download depends on Xray's availability. To support this need, Xray can be configured as a Highly Available cluster of 2 or more, active/active, nodes.

All cluster nodes in an Xray HA configuration are synchronized, and jointly share and balance the workload between them. When a node becomes unavailable, the cluster will automatically spread the workload across the other remaining node(s).

Setting up several servers in an HA configuration is supported with an Enterprise License and presents several benefits to your organization.

Xray high availability allows users create a cluster of Xray instances, supporting the enhanced Xray Artifactory integration.

  • Ensure resilience and agility

  • Active-active cluster

  • Easy to install and maintain

  • Unlimited number of nodes

Optimal Resilience

In case one or more nodes is unavailable or down for upgrade, the load is shared between the remaining nodes, ensuring optimal resilience and uptime.

Improved Performance with Load Balancing

All cluster nodes in an Xray HA configuration are synchronized, and jointly share and balance the workload between them using a load balancer. This includes all requests handled by Xray.

You can now scale your Xray environment with as many nodes as you need. This enhances Xray’s performance by delegating all workload across available cluster nodes, through a load balancer.

 

Always Synchronized

Xray seamlessly and instantly synchronizes all data, configuration, cached objects and scheduled job changes across all cluster nodes.

Enhanced Monitoring

Xray’s self-monitoring mechanism, which provides you with system availability issues, is enhanced to let you know which node is affected.

In addition, Xray provides cluster health information in a new page called “High Availability”, showing health information of every node and every microservice. 

Easy HA Setup

Xray allows you to easily install a full HA cluster in minutes, or upgrade your existing Xray environment


Architecture

The Xray HA architecture consists of 3 layers: load balancer, application and common resources.

Load Balancer

The load balancer is the entry point to your Xray HA cluster, optimally distributing requests to the Xray microservices on the cluster nodes. It's able to recognize the type of request and add it to its corresponding microservice queue, according to the current load on this micorservice type on each of the nodes. For example, when receiving an indexing request, Xray will check the load on the indexing microservice on all of the cluster nodes and will place the new request in the queue with least pending indexing requests.

It is the responsibility of your organization to manage and configure it correctly.

The code sample below shows a basic example of a load balancer configuration:

 nginx load balancer configuration example...
############################################################################################################
############################################################################################################
#Xray Reverse Proxy https+http with Nginx
############################################################################################################
ssl_certificate      /etc/nginx/certs/xray.cert;
ssl_certificate_key  /etc/nginx/certs/xray.key;
ssl_session_cache shared:SSL:1m;
ssl_prefer_server_ciphers   on;
## server configuration
upstream xray {
    server IP_SERVER_1:8000;
    server IP_SERVER_2:8000;
}
server {
    listen 443 ssl;
    listen 80 ;
    server_name <your_server_name>;
    if ($http_x_forwarded_proto = '') {
        set $http_x_forwarded_proto  $scheme;
    }
    ## Application specific logs
    ## access_log /var/log/nginx/<your_server_name>-access.log timing;
    ## error_log /var/log/nginx/<your_server_name>-error.log;
    chunked_transfer_encoding on;
    client_max_body_size 0;
    location / {
    proxy_read_timeout  900;
    proxy_pass_header   Server;
    proxy_cookie_path   ~*^/.* /;
    proxy_pass          http://xray;
    proxy_set_header    X-Forwarded-Port  $server_port;
    proxy_set_header    X-Forwarded-Proto $http_x_forwarded_proto;
    proxy_set_header    Host              $http_host;
    proxy_set_header    X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection upgrade;
    }
}

More details are available on the  nginx website.

Application Layer

Xray HA presents a cluster of two or more Xray nodes that share common resources. Each cluster node runs all 4 Xray microservices:

  1. Indexer - Responsible for the indexing process, including:
    • Recursively extracting artifacts and builds
    • Collecting artifact metadata from accompanying files
    • Building an artifact components graph representation
  2. Persist - Responsibilities include
    • Matching the given components graph with the public component information
    • Completing component naming
    • Storing the data in the relevant databases (graph data in PostgreSQL and component metadata in MongoDB)
  3. Analysis - Responsible for enriching component metadata such as vulnerabilities, licenses and versions.
  4. Server - Responsibilities include:
    • Generating violations by matching analysis data with watches and policies
    • Hosting the API and UI endpoints
    • Running scheduled jobs such as the database synchronization process

Common Resources

Separated into three parts:

  1. PostgreSQL: Components Graph Database
    • Every artifact and build indexed by Xray is broken down into multiple components. These components and the relationships between each other are represented in a checksum based components graph.
    • Xray uses PostgreSQL to store and query this components graph.
    • Default port 5432 should be open for communication between each of the nodes in the cluster and the database server.
    • PostgreSQL can either be installed as part of the Xray installation or manually
  2. MongoDB: Components Metadata and Configuration
    • Xray comes out of the box with a rich component metadata database. This database is updated on a daily basis using the database sync process.
    • Xray uses MongoDB to store this component metadata database, as well as all Xray configuration, such as watches, policies and violations.
    • Default port 27017 should be open for communication between each of the nodes in the cluster and the database server.
    • MongoDB can either be installed as part of the Xray installation or manually.
  3. RabbitMQ: Microservice Communication and Messaging
    • Xray has multiple flows, such as scanning, impact analysis, and database sync. These flows require processing completed by the different Xray services listed above. Flows contain multiple steps that are completed by the Xray services.
    • Xray uses RabbitMQ to manage these different flows and track synchronous and asynchronous communication between the services.
    • Default port 5672 should be open for communication between each of the nodes in the cluster and the database server. 
    • RabbitMQ is automatically installed in an Xray HA architecture.

Filestore

 The storage used by Xray is not a common resource and only node specific files, such as configuration and temporary files, are saved to the disk.

Synchronization

Critical and Temporary Data

Critical data is shared across the cluster nodes by using the same databases as common resources. Local temporary data, such as log files and artifacts being processed, is kept separately for each of the nodes in their config and data folders. 

Cached Objects

Xray implements caching for a variety of objects, including permissions, watches and builds. These caches are automatically updated on each change via RabbitMQ messages. The updated node will send synchronous messages via RabbitMQ to all other nodes to trigger a cache reload.

Scheduled Jobs

Xray uses RabbitMQ to ensure that periodical jobs are run only by a single node in the cluster, rather than all nodes. The job and node may change each time, however Xray verifies that every scheduled job is only executed once.

 

  • No labels