Overview

You can set up TLS certificates to enable encrypted connections from Xray to PostgreSQL or RabbitMQ.  

 


Securing PostgreSQL with TLS Support on Xray

  1. Copy these TLS parameters to /var/opt/jfrog/postgres/data/postgresql.conf.

    ssl = on
    ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL'
    ssl_prefer_server_ciphers = on
    ssl_cert_file = '/full/path/to/postgres/certificates/server.crt'
    ssl_key_file = '/full/path/to/postgres/certificates/server.key'
    ssl_ca_file = '/full/path/to/postgres/certificates/server_ca.crt'
  2. Verify that the certificates have the correct permissions.

    chown postgres /full/path/to/postgres/certificates/* && \
    chgrp postgres /full/path/to/postgres/certificates/* && \
    chmod 600 /full/path/to/postgres/certificates/*
  3. Change the connection string in the /var/opt/jfrog/xray/var/etc/system.yaml file.

    postgres://xray:xray@postgres:5432/xraydb?sslrootcert=/full/path/to/xray/certificates/ca_certificate.crt&sslkey=/full/path/to/xray/certificates/client.key&sslcert=/full/path/to/xray/certificates/client.crt&sslmode=verify-ca
  4. Make sure you have an Xray user and group.

    groupadd -g 1035 xray && \
    adduser xray --uid 1035 --gid 1035
  5. Assign permissions to the certificates.

    chown xray /full/path/to/xray/certificates/* && \
    chgrp xray /full/path/to/xray/certificates/* && \
    chmod 600 /full/path/to/xray/certificates/*
  6. Restart all the Xray services.

    bash /opt/jfrog/xray/scripts/xray.sh restart all

Securing RabbitMQ with TLS Support on Xray

  1. Create an openssl.cnf file and ensure that it contains the field, subjectAltName, and all the other required fields.
    The following example shows a sample openssl.cnf file.

    [req]
    default_bits = 4096
    default_md = sha256
    default_keyfile = privkey.pem
    distinguished_name = req_distinguished_name
    attributes = req_attributes
    x509_extensions = v3_ca # The extensions to add to the self signed cert
    [req_attributes]
    [req_distinguished_name]
    countryName = US
    countryName_min = 2
    countryName_max = 2
    stateOrProvinceName = XXX
    localityName = XXX
    0.organizationName = JFROG
    organizationalUnitName = XXX
    commonName = jfrog.com
    commonName_max = 64
    emailAddress = example@jfrog.com
    emailAddress_max = 64
    [ v3_ca ]
    subjectKeyIdentifier=hash
    authorityKeyIdentifier=keyid:always,issuer
    basicConstraints = critical,CA:true
    [ req_ext ]
    subjectAltName = DNS:testxray.vm
  2. Run the following commands to generate certificates for RabbitMQ and Xray.

    In the commands, subjectAltName,  is the used as an example  and the value should be a resolvable DNS.  and should be used in the system.yaml file when providing the shared.rabbitMq.url (see step 6 below).


    1. Create Certificate Authority (CA) files.

       #Creates ca-key.pem and ca.csr CA files. These are self-signed.
      
      openssl req -new -nodes -text -out ca.csr -keyout ca-key.pem -subj "/CN=certificate-authority" -addext "subjectAltName = DNS:testxray.vm" 
    2. Sign the CA private key, ca-key.pem, and create the related CA certificate.

      openssl x509 -req -in ca.csr -text -extfile openssl.cnf -extensions v3_ca -signkey ca-key.pem -out ca-cert.pem
    3. Create the RabbitMQ private key (server-key.pem) and certificate signing request file(server.csr).

      openssl req -new -nodes -text -out server.csr -keyout server-key.pem -subj "/CN=testxray.vm" -addext "subjectAltName = DNS:testxray.vm"
    4. Create a signed RabbitMQ public key (server-cert.pem).

      openssl x509 -req -in server.csr -text -extfile openssl.cnf -extensions v3_ca -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem
    5. Create Xray client key (client-key.pem) and certificate signing request file (client.csr).

      openssl req -new -nodes -text -out client.csr -keyout client-key.pem -subj "/CN=testxray.vm" -addext "subjectAltName = DNS:testxray.vm"
    6. Create Xray client certificate file.

      openssl x509 -req -in client.csr -text -extfile openssl.cnf -extensions v3_ca -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem
  3. Create the certs directories under $JFROG_HOME/xray/var/data/server and $JFROG_HOME/xray/var/data/rabbitmq.

    mkdir $JFROG_HOME/xray/var/data/server/certs
    
    mkdir $JFROG_HOME/xray/var/data/rabbitmq/certs
  4. Copy the ca and server certificates to $JFROG_HOME/xray/var/data/rabbitmq/certs and $JFROG_HOME/xray/var/data/server/certs respectively.

    ls -ltr <mounted directory>/xray/var/data/server/certs/
    total 3
    -rw-r--r-- 1 xray xray 1127 Oct 11 15:55 ca-cert.pem
    -rw-r--r-- 1 xray xray  993 Oct 11 15:55 client-cert.pem
    -rw-r--r-- 1 xray xray 1704 Oct 11 15:55 client-key.pem
    ls -ltr /opt/jfrog/xray/var/data/server/certs/
    total 3
    -rw-r--r-- 1 xray xray 1127 Oct 11 15:55 ca-cert.pem
    -rw-r--r-- 1 xray xray  993 Oct 11 15:55 client-cert.pem
    -rw-r--r-- 1 xray xray 1704 Oct 11 15:55 client-key.pem
    ls -ltr JFROG_HOME/xray/var/data/server/certs/
    total 3
    -rw-r--r-- 1 xray xray 1127 Oct 11 15:55 ca-cert.pem
    -rw-r--r-- 1 xray xray  993 Oct 11 15:55 client-cert.pem
    -rw-r--r-- 1 xray xray 1704 Oct 11 15:55 client-key.pem
    cp ca-cert.pem server-cert.pem server-key.pem $JFROG_HOME/xray/var/data/rabbitmq/certs
    
    cp ca-cert.pem client-cert.pem client-key.pem $JFROG_HOME/xray/var/data/server/certs

    To ensure that the client trusts self-signed certificates (only), you will need to perform the following steps according to the OS you are using.

    For Docker

    You will need to mount a root ca bundle into each Xray container:

    volumes:
          - /etc/localtime:/etc/localtime:ro
          - "${ROOT_DATA_DIR}/var:/var/opt/jfrog/xray"
          - /opt/jfrog/xray/app/third-party/rabbitmq/rabbitmq-root-ca.crt:/etc/ssl/certs/ca-certificates.crt

    For Linux Archive/Native OS: Debian 8/9/10, Ubuntu 16/18/20

    Copy your root certificate into /usr/local/share/ca-certificates/ and then run the update-ca-certificates command.

    # cp rabbitmq-root-ca.crt /usr/local/share/ca-certificates/
    # update-ca-certificates
    Updating certificates in /etc/ssl/certs...
    1 added, 0 removed; done.
    Running hooks in /etc/ca-certificates/update.d...Adding debian:rabbitmq-root-ca.pem
    done.
    done.

    For Linux Archive/Native OS: CentOS 6/7/8, RHEL 6/7/8

    Copy your root certificate into /etc/pki/ca-trust/source/anchors/ and then run the update-ca-trust command.

    # cp rabbitmq-root-ca.crt /etc/pki/ca-trust/source/anchors/
    # update-ca-trust

    Note that on CentOS 6/RHEL 6 you will have to run an additional command - update-ca-trust force-enable.

    After you add your own root certificate into the system bundle - you can verify the certificate with the following command:

    # openssl verify -verbose /opt/jfrog/xray/var/data/server/certs/rabbitmq-client.crt
    /opt/jfrog/xray/var/data/server/certs/rabbitmq-client.crt: OK
    
    # openssl verify -verbose /opt/jfrog/xray/var/data/rabbitmq/certs/rabbitmq-server.crt
    /opt/jfrog/xray/var/data/rabbitmq/certs/rabbitmq-server.crt: OK
    Otherwise we will get the error
    
    # openssl verify -verbose /opt/jfrog/xray/var/data/server/certs/rabbitmq-client.crt
    /opt/jfrog/xray/var/data/server/certs/rabbitmq-client.crt: CN = rabbitmq
    error 20 at 0 depth lookup:unable to get local issuer certificate
    
    # openssl verify -verbose /opt/jfrog/xray/var/data/rabbitmq/certs/rabbitmq-server.crt
    /opt/jfrog/xray/var/data/rabbitmq/certs/rabbitmq-server.crt: CN = rabbitmq
    error 20 at 0 depth lookup:unable to get local issuer certificate
  5. Modify the certificate permissions for the RabbitMQ user.

    chown -R 999:999 <mounted directory>/xray/var/data/rabbitmq/certs
    chown -R xray:xray /opt/jfrog/xray/var/data/rabbitmq/certs
    ## default user and group is xray:xray
    chown -R <xray user>:<xray group> JFROG_HOME/xray/var/data/rabbitmq/certs
  6. Modify the certificate permissions for the Xray user.

    chown -R 1035:1035 <mounted directory>/xray/var/data/server/certs
    chown -R xray:xray /opt/jfrog/xray/var/data/server/certs/
    ## default user and group is xray:xray
    chown -R <xray user>:<xray group> JFROG_HOME/xray/var/data/server/certs/
  7. Modify /opt/jfrog/xray/var/etc/system.yaml (under the shared folder) according to your configuration. Do not overwrite existing lines in the system.yaml file.
    The following sample system.yaml shows an example.

    shared:
     rabbitMq:
       ## Enable this to stop rabbitmq along with other services of xray
       ## By default rabbitmq will always be running
       #autoStop: true
       node:
         rabbitmqConf:
         - name: ssl_options.cacertfile
           value: /var/opt/jfrog/xray/data/rabbitmq/certs/ca-cert.pem
         - name: ssl_options.certfile
           value: /var/opt/jfrog/xray/data/rabbitmq/certs/server-cert.pem
              - name: ssl_options.keyfile
                 value: /var/opt/jfrog/xray/data/rabbitmq/certs/server-key.pem
               - name: ssl_options.verify
                 value: verify_peer
               - name: ssl_options.fail_if_no_peer_cert
                 value: false
               - name: management.listener.ssl
                 value: true
               - name: listeners.ssl.default
                 value: 5671
           autoStop: true
       url: amqps://guest:password@rabbitmq:5671
       password: password
       env:
         GODEBUG: "x509ignoreCN=0"
         RABBITMQ_ERLANG_COOKIE: "{{ xray_first_node }}-ERLANG-COOKIE"
         RABBITMQ_NODENAME: "{{ inventory_hostname_short }}"
         XRAY_CERT_FILE_PATH: "/opt/jfrog/xray/var/data/server/certs/client-cert.pem"
         XRAY_CERT_KEY_FILE_PATH: "/opt/jfrog/xray/var/data/server/certs/client-key.pem"
         XRAY_CA_CERT_FILE_PATH: "/opt/jfrog/xray/var/data/server/certs/ca-cert.pem"
  8. Configure the rabbitmq.conf ($XRAY_HOME/app/bin/rabbitmq/rabbitmq.conf) file to use the certs.
    The following sample rabbitmq.conf shows an example.

    listeners.tcp = none
    listeners.ssl.default = {{ xray_rabbitmq_ssl_port_1 }}
    listeners.ssl.other_port = {{ xray_rabbitmq_ssl_port_2 }}
    loopback_users.guest = false
    ssl_options.verify               = verify_peer
    ssl_options.fail_if_no_peer_cert = false
    ssl_options.cacertfile           = {{ xray_rabbitmq_certs_dir }}/ca-cert.pem
    ssl_options.certfile             = {{ xray_rabbitmq_certs_dir }}/server-cert.pem
    ssl_options.keyfile              = {{ xray_rabbitmq_certs_dir }}/server-key.pem
    management.ssl.port       = {{ xray_rabbitmq_management_ssl_port }}
    management.ssl.cacertfile = {{ xray_rabbitmq_certs_dir }}/ca-cert.pem
    management.ssl.certfile   = {{ xray_rabbitmq_certs_dir }}/server-cert.pem
    management.ssl.keyfile    = {{ xray_rabbitmq_certs_dir }}/server-key.pem
    management.listener.ssl        = true
  9. Run the REST API call to enable the TLS connection to RabbitMQ in Xray.

    curl -u<username>:<password> -X PUT -H "Content-Type: application/json" http://<ARTIFACTORY>:8082/xray/api/v1/configuration/systemParameters -d '{"sslInsecure": false,"maxDiskDataUsage": 80,"monitorSamplingInterval": 300,"mailNoSsl": false,"messageMaxTTL": 7,"jobInterval": 86400,"allowSendingAnalytics": true,"httpsPort": 443,"enableTlsConnectionToRabbitMQ": true,"httpClientMaxConnections": 50,"httpClientMaxIdleConnections": 20}'

    Replace <username> and <password> with an admin user and password credentials for Artifactory and <artifactory_url> with the actual Artifactory URL.

  10. Run the following steps to enable the TLS connection to RabbitMQ in Xray if you use Docker Compose.

    cd  <path to extracted compose directory>/jfrog-xray-<version>-compose/
    ## Export the TLS port in the docker-compose-rabbitmq.yaml (docker-compose.yaml for older versions of 3.x) and add under services -> rabbitmq -> ports.
    - 5671:5671
    # Restart RabbitMQ services
    docker-compose -p xray-rabbitmq -f docker-compose-rabbitmq.yaml down
    docker-compose -p xray-rabbitmq -f docker-compose-rabbitmq.yaml up -d
  11. Restart Xray services.

    docker-compose -p xray -f docker-compose.yaml down
    docker-compose -p xray -f docker-compose.yaml up -d
    systemctl stop xray.service
    systemctl start xray.service
    
    ## For Centos 6 and RHEL 6
    # service xray stop
    # service xray start 
    /opt/jfrog/xray/bin/xray.sh stop
    /opt/jfrog/xray/bin/xray.sh start 
  12. After Xray services are up and running, you can verify if RabbitMQ is accessible through https://<xray-ipaddress-or-hostname>:15672.

Troubleshooting RabbitMQ with TLS in Xray

If you encounter any errors or issues you might find the following troubleshooting tops helpful,

  • Ensure that you have the proper certificates and that their location is correct. The $JFROG_HOME variable refers to the directory in which Xray is installed. The default is /opt/jfrog
  • In some operating systems, the openssl.cnf file may be located in a different location than /etc/ssl/openssl.cnf. Use the find command to find it in your system.

    find / -name openssl.cnf
  • Check that all the certificates are owned by the user and group xray. If you are using a different user and group to run Xray, ensure this user is the owner of the files.
  • The YAML files must have proper indentation with the same amount of spaces across the entire file. Ensure the Xray system.yaml file is properly indented and that the syntax is correct. Also, check that the details are correct such as passwords, paths, and URLs are correct in the YAML file.
  • A successful REST API call requires an Artifactory admin user.
  • Check the logs to find additional information to help in further troubleshooting any issues. The logs directory is $JFROG_HOME/xray/var/log. The recommended logs to look at are the console.log, xray-server-service.log, and the RabbitMQ logs in the rabbitmq directory.

Trusting Self-Signed Certificates

When an Xray instance/node is configured to go through an SSL proxy that uses a self-signed certificate, you may encounter the following issue when performing tasks such as an online database sync:

2021-07-20T14:47:47.500Z [33m[jfxr ][0m [1m[31m[ERROR][0m [c080f44e606d159 ] [samplers:91                   ] [main                ] Failed to read response from jxrayUrl. Error: Get "https://jxray.jfrog.io/api/v1/system/ping": x509: certificate signed by unknown authority
  1. To overcome this issue, you will need to import the Proxy certificate into each Xray instance/pod by placing it under the following path within the Xray machine/container/pods:/etc/ssl/certs/.
  2. Next, you will need to restart Xray.
    The path shown above is the default directory used by Go applications (such as Xray) when importing SSL certificates.