Installing Xray

Single Node Installation

The following installation methods are supported:

• Interactive Script (recommended)
• Manual RPM
• Manual Debian
• Helm

Interactive Script Installation (recommended)

The installer script works with all supported upgrade methods (RPM, Debian and Docker Compose). It provides you an interactive way to install Xray and its dependencies.

1. Extract the contents of the compressed archive and go to the extracted folder.
   

   tar -xvf jfrog-xray-<version>-<compose|rpm|deb>.tar.gz
   cd jfrog-xray-<version>-<compose|rpm|deb>

   .env file included within the Docker-Compose archive

   This .env file is used by docker-compose and is updated during installations and upgrades.

   Notice that some operating systems do not display dot files by default. If you make any changes to the file, remember to backup before an upgrade.

2. Run the installer script.
   Note: the script will prompt you with a series of mandatory inputs, including the jfrogURL (custom base URL) and joinKey.

   RPM or Debian

   ./install.sh

   Docker Compose

   ./config.sh

3. Validate and customize the product configuration(optional), including the third party dependencies connection details and ports.
   

   Please ensure that a large file handle limit is specified before you start Xray. 

   
   

4. Start and manage the Xray service.
   

   systemd OS

   systemctl start|stop xray.service

   systemv

   service xray start|stop

   Docker Compose

   cd jfrog-xray-<version>-compose
   docker-compose -p xray up -d
   docker-compose -p xray ps
   docker-compose -p xray down

5. Access Xray from your browser at: http://<jfrogUrl>/ui/, go the Security & Compliance tab in the Application module in the UI.

6. Check Xray Log.
   

   tail -f $JFROG_HOME/xray/var/log/console.log

   Configuring the Log Rotation of the Console Log

   The console.log file can grow quickly since all services write to it. This file is not log rotated for Darwin installations. Learn more on how to configure the log rotation.

Manual RPM Installation

The RPM installation bundles Xray and all its dependencies. It is provided as native RPM packages, where Xray and its dependencies must be installed separately. Use this, if you are automating installations.

1. Extract the contents of the compressed archive, and go to the extracted folder.
   

   tar -xvf jfrog-xray-<version>-rpm.tar.gz
   cd jfrog-xray-<version>-rpm

2. Install PostgreSQL.
   

   PostgreSQL is required and must be installed before continuing with the next installation steps.

   Set your PostgreSQL connection details in the Shared Configurations section of the $JFROG_HOME/xray/var/etc/system.yaml file.

3. Install db-util.
   

   # This will install db-util if db_dump is not available
   hash db_dump 2>/dev/null || rpm -ivh --replacepkgs ./third-party/misc/db4-utils-4.7.25-20.el6_7.x86_64.rpm

4. Install RabbitMQ dependencies.

   # Note : Use rpms with el6 when installing on Centos 6 and RHEL 6. Run the following from the extracted folder.
   rpm -ivh --replacepkgs ./third-party/rabbitmq/socat-1.7.3.2-2.el7.x86_64.rpm
   rpm -ivh --replacepkgs ./third-party/rabbitmq/erlang-21.1.4-1.el7.centos.x86_64.rpm

5. Install Xray. You must run as a root user.

   rpm -Uvh --replacepkgs ./xray/xray.rpm

6. Customize the product configuration.

   1. Set the Artifactory connection details.

   2. Customize the PostgreSQL Database connection details. (optional)

   3. Set any additional configurations (for example: ports, node id) using the Xray system.yaml configuration file.
      

      Please ensure that a large file handle limit is specified before you start Xray. 

7. Start and manage the Xray service.
   

   systemd OS

   systemctl start|stop xray.service

   systemv OS

   service xray start|stop|status|restart

8. Access Xray from your browser at: http://<jfrogUrl>/ui/, go the Security & Compliance tab in the Application module in the UI.

9. Check Xray Log.
   

   Linux

   tail -f $JFROG_HOME/xray/var/log/console.log

Manual Debian Installation

The Debian installation bundles Xray and all its dependencies. It is provided as native Debian packages, where Xray and its dependencies must be installed separately. Use this, if you are automating installations.

1. Extract the contents of the compressed archive, and go to the extracted folder.
   

   tar -xvf jfrog-xray-<version>-deb.tar.gz
   cd jfrog-xray-<version>-deb

2. Install PostgreSQL.
   

   PostgreSQL is required and must be installed before continuing with the next installation steps.
   

   Set your PostgreSQL connection details in the Shared Configurations section of the $JFROG_HOME/xray/etc/system.yaml file.

3. Install db-util.
   

   db-util allows us to interact with the Berkley DB that contains information about RPM-based Docker images. This way, JFrog Xray can index OS packages for these images.

   Ubuntu

   dpkg -i ./third-party/misc/db5.3-util_5.3.28-3ubuntu3_amd64.deb
   dpkg -i ./third-party/misc/db-util_1_3a5.3.21exp1ubuntu1_all.deb

   Debian 8

   dpkg -i ./third-party/misc/db5.3-util_5.3.28-9+deb8u1_amd64.deb
   dpkg -i ./third-party/misc/db-util_5.3.0_all.deb

   Debian 9

   dpkg -i ./third-party/misc/db5.3-util_5.3.28-12+deb9u1_amd64.deb
   dpkg -i ./third-party/misc/db-util_5.3.0_all.deb

4. Install RabbitMQ dependencies.
   

   ubuntu 16.04 (xenial)

   dpkg -i ./third-party/rabbitmq/libssl1.1_1.1.0j-1_deb9u1_amd64.deb
   dpkg -i ./third-party/rabbitmq/socat_1.7.3.1-2+deb9u1_amd64.deb
   dpkg -i ./third-party/rabbitmq/esl-erlang_21.2.1-1~ubuntu~xenial_amd64.deb

   ubuntu 18.04 (bionic)

   dpkg -i ./third-party/rabbitmq/socat_1.7.3.1-2+deb9u1_amd64.deb
   dpkg -i ./third-party/rabbitmq/esl-erlang_21.2.1-1~ubuntu~bionic_amd64.deb

   Debian 8 (jessie)

   # Before installing Erlang dependencies
   mv /etc/apt/sources.list.d/backports.list /etc/apt >/dev/null
   
   apt-get update
   dpkg -i ./third-party/rabbitmq/libssl1.1_1.1.0j-1_deb9u1_amd64.deb
   dpkg -i ./third-party/rabbitmq/socat_1.7.3.1-2+deb9u1_amd64.deb
   dpkg -i ./third-party/rabbitmq/esl-erlang_21.2.1-1~debian~jessie_amd64.deb
   
   # After installing Erlang dependencies
   mv /etc/apt/backports.list /etc/apt/sources.list.d/backports.list >/dev/null
   apt-get update

   Debian 9 (stretch)

   dpkg -i ./third-party/rabbitmq/socat_1.7.3.1-2+deb9u1_amd64.deb
   dpkg -i ./third-party/rabbitmq/esl-erlang_21.2.1-1~debian~stretch_amd64.deb

   Run the following commands if you have any issues running the above commands to install the dependencies. These will include any missing dependencies from your system.

   Install any missing dependancies

   apt-get update
   apt-get install -f -y
   apt-get update

5. Install Xray. You must run as a root user.

   dpkg -i ./xray/xray.deb

6. Customize the product configuration.

   1. Set the Artifactory connection details.

   2. Customize the PostgreSQL Database connection details. (optional)

   3. Set any additional configurations (for example: ports, node id) using the Xray system.yaml configuration file

      Please ensure that a large file handle limit is specified before you start Xray. 

7. Start and manage the Xray service.
   

   systemd OS

   systemctl start|stop xray.service

   systemv OS

   service xray start|stop|status|restart

8. Access Xray from your browser at: http://<jfrogUrl>/ui/, go the Security & Compliance tab in the Application module in the UI.

9. Check Xray Log.
   

   Linux

   tail -f $JFROG_HOME/xray/var/log/console.log

Helm Installation 

The JFrog official Xray Helm charts are available via Helm Hub.

Detailed instructions on installing and upgrading an Xray or Xray HA installation are documented in the chart's README.md file which are available on JFrog's Charts GitHub repository.

HA Installation

The following describes how to set up an Xray HA cluster with two or more nodes. For more information, see the System Architecture.

Prerequisites

All nodes within the same Xray HA installation must be running the same Xray version.

Database

Xray HA requires an external PostgreSQL database. Make sure to install it before proceeding to install the first node. There are several ways to setup PostgreSQL for redundancy. Including: HA, Load Balancing and Replication. For more information, see the PostgreSQL documentation.

RabbitMQ is automatically installed as part of the Xray installation for every node. In case of HA architecture, it uses queue mirroring between the different RabbitMQ nodes, automatically setup.

Licensing

Xray HA is supported with an Enterprise License. Each node in the cluster must be activated with a different license.

Network

• All the Xray HA components (Xray cluster nodes, database server and RabbitMQ) must be within the same fast LAN.

• All the HA nodes must communicate with each other through dedicated TCP ports.

The following installation methods are supported:

• RPM / Debian
• Docker Compose

RPM/Debian Installation

First node installation steps:

1. Install the first node. The installation is identical to the single node installation. Important: make sure not to start Xray.

2. Configure the system.yaml file with the database and first node configuration details. For example,
   

   First node system.yaml

   shared:
     database:
       type: postgresql
       driver: org.postgresql.Driver
       url: postgres://<ip:port>/xraydb?sslmode=disable
       username: xray
       password: xray
     jfrogUrl: <JFrog URL>
     security:
       joinKey: <Artifactory Join Key>

3. Start and manage the Xray service.
   

   systemd OS

   systemctl start|stop xray.service

   Systemv OS

   service xray start|stop

4. Access Xray from your browser at: http://<jfrogUrl>/ui/, go the Security & Compliance tab in the Application module in the UI.

5. Check Xray Log.
   

   Linux

   tail -f $JFROG_HOME/xray/var/log/console.log

Additional node installation steps:

In order for a node to join a cluster, the node must have the same database configuration and the Master Key.

1. Install all additional nodes using the same steps described above
2. Configure PostgreSQL to allow external IP connections. This will ensure PostgreSQL is reachable by this node.

3. Configure the system.yaml file for the additional node with master key, database and active node configurations. For example,

   Additional node system.yaml

   shared:
     database:
       type: postgresql
       driver: org.postgresql.Driver
       url: postgres://<ip:port>/xraydb?sslmode=disable
       username: xray
       password: xray
     jfrogUrl: <JFrog URL>
     security:
       joinKey: <Artifactory Join Key>
     rabbitMq:
       active:
         node:
           name: 
           ip: 

4. Copy the master.key from the first node to the additional node located at $JFROG_HOME/xray/var/etc/security/master.key.

5. Start the additional node.

6. Access Xray from your browser at: http://<jfrogUrl>/ui/, go the Security & Compliance tab in the Application module in the UI.

7. Check Xray Log.
   

   Linux

   tail -f $JFROG_HOME/xray/var/log/console.log

Docker Compose Installation

First node installation steps:

1. Extract the contents of the compressed archive and go to the extracted folder.
   

   tar -xvf jfrog-xray-<version>-compose.tar.gz
   cd jfrog-xray-<version>-compose

   .env file included within the Docker-Compose archive

   This .env file is used by docker-compose and is updated during installations and upgrades.

   Notice that some operating systems do not display dot files by default. If you make any changes to the file, remember to backup before an upgrade.

2. Run the config.sh script to setup folders with required ownership. Note: the script will prompt you with a series of mandatory inputs, including if this is part of a cluster, and configure the needed system.yaml.
   

   ./config.sh

3. Validate and customize the product configuration (optional), including the third party dependencies connection details and ports.

4. Start and manage Xray using docker-compose commands. Note: Run this command only from the extracted folder.
   

   cd jfrog-xray-<version>-compose
   docker-compose -p xray logs
   docker-compose -p xray ps
   docker-compose -p xray up -d
   docker-compose -p xray down

5. Access Xray from your browser at: http://<jfrogUrl>/ui/, go the Security & Compliance tab in the Application module in the UI.
   

6. Check Xray Log.
   

   docker-compose -p xray logs

Additional node installation steps:

1. Extract the contents of the compressed archive and go to the extracted folder.
   

   tar -xvf jfrog-xray-<version>-compose.tar.gz
   cd jfrog-xray-<version>-compose

2. Run the config.sh script to setup folders with required ownership. Note: the script will prompt you with a series of mandatory inputs, including if this is part of a cluster, and configure the needed system.yaml.
   

   ./config.sh
   

3. Validate and customize the product configuration (optional), including the third party dependencies connection details and ports.

4. Start and manage Xray using docker-compose commands. Note: Run this command only from  the extracted folder.
   

   cd jfrog-xray-<version>-compose
   docker-compose -p xray logs
   docker-compose -p xray ps
   docker-compose -p xray up -d
   docker-compose -p xray down

5. Access Xray from your browser at: http://<jfrogUrl>/ui/, go the Security & Compliance tab in the Application module in the UI.

6. Check Xray Log.
   

   docker-compose -p xray logs

Helm Installation 

The JFrog official Xray Helm charts are available via Helm Hub.

Detailed instructions on installing and upgrading an Xray or Xray HA installation are documented in the chart's README.md file which are available on JFrog's Charts GitHub repository.

Product Configuration

After installing and before running Xray, you may set the following configurations.

Artifactory Connection Details

Xray requires a working Artifactory server and a suitable license. The Xray connection to Artifactory requires 2 parameters:

• jfrogUrl - URL to the machine where JFrog Artifactory is deployed, or the load balancer pointing to it. It is recommended to use DNS names rather than direct IPs. For example: "http://jfrog.acme.com or http://10.20.30.40:8082"
  Set it in the Shared Configurations section of the $JFROG_HOME/xray/var/etc/system.yaml file.
• join.key - This is the "secret" key required by Artifactory for registering and authenticating the Xray server.
  You can fetch the Artifactory joinKey (join Key) from the JPD UI in the Administration module | Security | Settings | Join Key or from the Artifactory filesystem. 
  Set the join.key used by your Artifactory server in the Shared Configurations section of the $JFROG_HOME/xray/var/etc/system.yaml file.

Changing PostgreSQL Database Credentials

Xray comes bundled with a postgreSQL Database out-of-the-box, which come pre-configured with default credentials.

To change the default credentials:

# Access PostgreSQL as the Xray user adding the optional -W flag to invoke the password prompt
$ psql -d xraydb -U xray -W
 
# Securely change the password for user "xray". Enter and then retype the password at the prompt.
\password xray
 
# Verify the update was successful by logging in with the new credentials
$ psql -d xraydb -U xray -W

Set your PostgreSQL connection details in the Shared Configurations section of the $JFROG_HOME/xray/var/etc/system.yaml file.

Changing RabbitMQ Database Credentials

Xray comes pre-installed with RabbitMQ using default credentials, User: guest, Password: guest.

To change the default credentials:

# Change the default password
$ rabbitmqctl change_password guest <new_password>
 
# Verify the update was successful
$ rabbitmqctl authenticate_user guest <new_password>

Third Party Log Collector

Xray enables using an external log collector such as Sumologic or Splunk.

To adjust the permissions to allow the log collection service perform read operations on the generated log files:

1. Add the log collection service user to the relevant group if needed (the user and group that installed and started Xray)

2. Apply the user and group permissions as needed on the $JFROG_HOME/xray/var/log directory using:

   $ chmod -R 640 $JFROG_HOME/xray/var/log

3. Adjust the group read inheritance permissions setgid bit using:
   

   $ chmod -R 2755 $JFROG_HOME/xray/var/log 

   This will cause the generated log files to inherit the folder's group permissions.

Third Party Applications

max_connections = 300

Installing PostgreSQL

1. Install PostgreSQL.
   

   mkdir -p /var/opt/jfrog/postgres
   chmod +x ./third-party/postgresql/postgresql-*-linux-x64.run
   ./third-party/postgresql/postgresql-*.run --unattendedmodeui none --mode unattended --datadir /var/opt/jfrog/postgres --serverport 5432
   chown -R postgres:postgres /var/opt/jfrog/postgres
   
   service postgresql-9.6 stop
   service postgresql-9.6 start

2. Setup the database and user.
   

   cd /tmp
   su postgres
   export PGPASSWORD='postgres'
   /opt/PostgreSQL/9.6/bin/psql -c "CREATE USER xray WITH PASSWORD 'xray';"
   /opt/PostgreSQL/9.6/bin/psql -c "CREATE DATABASE xraydb WITH OWNER=xray ENCODING='UTF8';"
   /opt/PostgreSQL/9.6/bin/psql -c "GRANT ALL PRIVILEGES ON DATABASE xraydb TO xray;"
   exit

3. Configure PostgreSQL to allow external IP connections. 

   By default PostgreSQL will only allow localhost clients communications. To enable different IPs to communicate with the database you will need to configure the pg_hba.conffile.
   

   File location according to installation type

   • Docker-compose: $JFROG_HOME/xray/var/opt/jfrog/postgres
   • Native installations: /var/opt/postgres/data 

   
   To grant all IPs access you may add the below, under the IPv4 local connections section:
   

   host    all             all             0.0.0.0/0               trust

   After editing the pg_hba.conf file you will need to reload the configuration by running the pg_ctl command:

   pg_ctl reload -D /var/opt/jfrog/postgres/data

For Advanced Users

Manual Docker Compose Installation

1. Extract the contents of the compressed archive and go to the extracted folder.

   tar -xvf jfrog-xray-<version>-compose.tar.gz

   .env file included within the Docker-Compose archive

   This .env file is used by docker-compose and is updated during installations and upgrades.

   Notice that some operating systems do not display dot files by default. If you've made any changes to the file, remember to backup before an upgrade.

2. Create the following folder structure under $JFROG_HOME/xray.
   

   |-- [        ]  app
   |   `-- [    ]  third-party
           `-- [999   999    ]  rabbitmq 
   `-- [1035     1035    ]  var
       |-- [1035     1035    ]  data
       |-- [1035     1035    ]  etc

3. Copy the appropriate docker-compose templates from the templates folder to the extracted folder. Rename it as docker-compose.yaml.
   

   Requirement	Template
   Xray with externalised PostgreSQL	docker-compose.yaml
   Xray with PostgreSQL	docker-compose-postgres.yaml

4. Update the .env file.
   

   ## The Installation directory for Xray. IF not entered, the script will prompt you for this input. Default [$HOME/.jfrog/xray]
   ROOT_DATA_DIR=
   
   # Host ID. Other nodes in the cluster will use this ID to identify this node
   HOST_ID=
   
   # ID of the active node. Please leave the value as "None" for active nodes. (shared.rabbitMq.active.node.name).  
   JF_SHARED_RABBITMQ_ACTIVE_NODE_NAME=None 
   
   # IP of the active node. (shared.rabbitMq.active.node.ip)
   JF_SHARED_RABBITMQ_ACTIVE_NODE_IP=127.0.0.1

5. Customize the product configuration.
   1. Set the Artifactory connection details.
      
   2. Customize the PostgreSQL Database connection details. (optional)

   3. Set any additional configurations (for example: ports, node id) using the Xray system.yaml configuration file.
      

      Ensure the host's ID and IP are added to the system.yaml. This is important to ensure that other products and Platform Deployments can reach this instance

6. Customize any additional product configuration (optional) including, Java Opts and filestore.

7. Copy the rabbitmq.conf and setRabbitCluster.sh files to the folder: app/third-party/rabbitmq. Ensure both are owned by 999:999 (rabbitmq uid/gid)

8. Start Xray using docker-compose commands.
   

   docker-compose -p xray logs
   docker-compose -p xray ps
   docker-compose -p xray up -d
   docker-compose -p xray down

9. Access Artifactory from your browser at: http://SERVER_HOSTNAME/ui/. For example, on your local machine: http://localhost/ui/.

10. Check Distribution log.
    

    docker-compose -p xray logs

    Configuring the Log Rotation of the Console Log

    The console.log file can grow quickly since all services write to it. The installation scripts add a cron job to log rotate the console.log file every hour.

    This is not  done for manual Docker Compose installations. Learn more on how to configure the log rotation.

Watch the Screencast