YAML File Format
The YAML file is constructed with keys and entities, using the following key: [entity] format.
- Shared configurations specified under the shared section, are used by all micro-services.
- Micro-service specific configurations are set under each micro-service section and override similar shared configurations for the specific micro-service.
Supported Configurations
Shared Configurations
## Pipelines config
##
shared:
## Artifactory configuration
##
artifactory:
## Artifactory URL
##
baseUrl: ""
## Unified UI URL
##
baseUrlUI: ""
## Artifactory Service ID
##
artifactoryServiceId: ""
## Artifactory License ID
##
licenseId: ""
## Pipelines Service ID
##
serviceId: ""
## Proxy to connect to Artifactory
##
proxy:
url: ""
username: ""
password: ""
## Logging configuration
##
logging:
## UI Refresh rate seconds
##
view:
refreshRate: 10s
## For build time seconds metrics
##
metrics:
filePath: /opt/jfrog/pipelines/var/log/api-metrics_events.log
console: false
rotation:
maxSizeMb: 25
maxFiles: 10
maxAgeDays: 365
compress: true
intervalMs: 900000
## Metrics configuration
##
metrics:
enabled: true
## Router configuration
##
## Router configuration
##
router:
image: "jfrog/pipelines-router"
ip: ""
## This port is used by services on the node locally to connect to the router
accessPort: 8046
## This port is used externally to connect to the router
dataPort: 8082
## The join key from Artifactory in order to communicate with Artifactory
joinKey: ""
security:
masterKey: ""
## Database configuration
##
db:
type: "postgres"
## The following settings, except connectionString and version, are used when the Pipelines installer manages a database container
ip: ""
port: 5432
name: "pipelinesdb"
username: "apiuser"
password: ""
## Pipelines installer will install PostgreSQL 12.3 by default
image: "jfrog/pipelines-postgres"
version: 12.3
externalUrl: ""
## The connectionString contains everything used to connect to the database, e.g., postgres://username:password@hostname:5432/databaseName
connectionString: ""
## RabbitMQ configuration
##
msg:
## Pipelines installer will install RabbitMQ 3.8.3 by default
image: "jfrog/pipelines-msg"
version: 3.8.3
## IP address at which RabbitMQ can be reached
ip: ""
## Port on which RabbitMQ can be reached over amqp or amqps
port: 30200
## Port on which RabbitMQ can be reached over http or https (RabbitMQ admin interface)
adminPort: 30201
## Cookie used for RabbitMQ clustering
erlangCookie: ""
## Username and password of an admin user
username: "admin"
password: ""
## RabbitMQ exchange for Pipelines queues
defaultExchange: pipelinesEx
## RabbitMQ vhost for queues providing messages to build nodes
amqpVhost: pipelines
## RabbitMQ vhost for queues used internally by Pipelines
amqpRootVhost: pipelinesRoot
## Protocol for amqp or amqps connections
protocol: amqp
## Queues created by default for Pipelines services
queues:
- "core.pipelineSync"
- "core.runTrigger"
- "core.stepTrigger"
- "core.marshaller"
- "core.logup"
- "cluster.init"
- "www.signals"
- "core.nexec"
- "core.hookHandler"
## Protocol, username, and password to connect to the RabbitMQ management interface
ui:
protocol: http
username: "monitor"
password: ""
## URLs at which it is possible to reach RabbitMQ from Pipelines and build nodes, for example with a load balancer
external:
## URL to connect to RabbitMQ from build nodes, using the username and password above
url: ""
rootUrl: ""
adminUrl: ""
## Vault configuration
##
vault:
## Pipelines installer will install Vault 1.2.2 by default
image: "jfrog/pipelines-vault"
version: 1.2.2
## Address at which Vault is reachable
ip: ""
## Port for Vault API
port: 30100
## Port used for Vault clustering
clusterPort: 30101
## Root token to connect to Vault
token: ""
## Unseal keys when Vault installation is managed by the Pipelines installer
unsealKeys:
- ""
- ""
- ""
- ""
- ""
## Redis configuration
##
redis:
ip: ""
port: 6379
## Pipelines installer will install Redis 5.0.5 by default
image: "jfrog/pipelines-redis"
version: 5.0.5
clusterEnabled: true
## NewRelic APM configuration
##
newRelic:
enabled: false
license: ""
appName: ""
logging:
debug: false
distributedTracer:
enabled: false
Core Services Configurations
## This section is used for bringing up the core services and setting up
## configurations required by the installer & the services
##
core:
## id is automatically determined based on the current hostname
## or set using the SHARED_NODE_ID environment variable.
##
id: ""
## IP address of the local machine, used by default when installing services
installerIP: ""
## Default password when installing Postgres, RabbitMQ, etc.
installerAuthToken: ""
installerImage: "jfrog/pipelines-installer"
## Registry from which to download Pipelines images as part of the installation and
## dynamic kubernetes build node creation
registryUrl: "releases-docker.jfrog.io"
## Kubernetes secret name to authenticate against the registry specified at
## registryUrl when downloading build node images
imageRegistrySecret: ""
## The following operating system variables are set by the installer to reflect the environment
os: ""
osDistribution: ""
architecture: ""
dockerVersion: ""
## The runMode determines the log level in Pipelines services. Choices are devmode, beta, and production.
runMode: ""
## User and group for the installer on the local machine
user: ""
group: ""
noVerifySsl: false
ignoreTLSErrors: false
## Set by the installer and updated on upgrades to reflect the version of Pipelines installed
controlplaneVersion: ""
buildplaneVersion: ""
## Global proxy settings, to be applied to all services
##
proxy:
httpProxy: ""
httpsProxy: ""
noProxy: ""
username: ""
password: ""
## Mailserver settings
##
mailserver:
host: ""
port: ""
username: ""
password: ""
tls: ""
ssl: ""
apiRetryIntervalMs: 3000
accountSyncFrequencyHr: 1
kubernetesNamespace: "pipelines"
imageRegistrySecret: "pipelines-reg-login"
## Duration for which deleted resources in a pipeline may be restored before permanently deleting
hardDeleteIntervalInMins: 60
## Number of configuration backups to keep when running "pipelines upgrade"
configBackupCount: 5
## The time at which Pipelines was last updated
lastUpdateTime: ""
## URL to which general statistics on usage are posted
callHomeUrl: "https://api.bintray.net/products/jfrog/pipelines/stats/usage"
## Posting usage statistics can be disabled by setting allowCallHome to false
allowCallHome: true
## How often Pipelines services report health statistics internally
serviceInstanceHealthCheckIntervalInMins: 1
serviceInstanceStatsCutOffIntervalInHours: 24
rabbitmqHealthCheckIntervalInMins: 1
artifactoryHealthCheckIntervalInMins: 1
dbHealthCheckIntervalInMins: 1
dbHealthCheckTimeoutInSeconds: 2
customCACertsPath: ""
## Enable signed pipelines feature
signedPipelinesEnabled: true
## Enable pipeline source auto-sync feature
autoSyncResourceIfOutdated: true
## Store version history for resources
storeVersionHistory: true
## Pipelines state retention policy
##
retentionPolicy:
enabled: false
## State beyond this age is removed, except for minRuns
##
maxAgeDays: 90
## Minimum runs per pipeline for which state is retained
##
minRuns: 10
## Service configuration
##
services:
api:
name: api
## Port on which the Pipelines API is listening
port: 30000
externalUrl: ""
image: "jfrog/pipelines-api"
www:
name: www
## Port on which the Pipelines UI is listening
port: 30001
externalUrl: ""
sessionSecret: ""
image: "jfrog/pipelines-www"
pipelineSync:
name: pipelineSync
image: "jfrog/pipelines-micro"
runTrigger:
name: runTrigger
image: "jfrog/pipelines-micro"
stepTrigger:
name: stepTrigger
image: "jfrog/pipelines-micro"
cron:
name: cron
image: "jfrog/pipelines-micro"
nexec:
name: nexec
image: "jfrog/pipelines-micro"
hookHandler:
name: hookHandler
image: "jfrog/pipelines-micro"
marshaller:
name: marshaller
image: "jfrog/pipelines-micro"
logup:
name: logup
image: "jfrog/pipelines-micro"
Runtime Configuration
## Runtime configuration
##
runtime:
## Name of the Artifactory repository used to store state and step logs for Pipelines
rootBucket: ""
defaultMinionCount: 1
nodeCacheIntervalMS: 600000
jobConsoleBatchSize: 10
jobConsoleBufferIntervalMs: 3
## Default maximum disk usage allowed before a node is stopped
maxDiskUsagePercentage: 90
## Maximum and default timeout for steps in a node pool
stepTimeoutMS: 3600000
## Day of week on which cached nodes are replaced
nodeStopDayOfWeek: 0
## Duration after which cached nodes are replaced
nodeStopIntervalDays: 30
maxNodeCheckInDelayMin: 15
defaultMinionInstanceSize: "c4.large"
## Settings that control the types of node pools that can be added
allowDynamicNodes: true
allowCustomNodes: true
isSaas: false
## Default images used to run steps, more information in the following section
languageImages:
- architecture: x86_64
os: Ubuntu_16.04
language: node
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u16node
isDefault: true
defaultVersion: 10.18.0
- architecture: x86_64
os: Ubuntu_16.04
language: java
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u16java
defaultVersion: 13
- architecture: x86_64
os: Ubuntu_16.04
language: cpp
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u16cpp
defaultVersion: 9.0.0
- architecture: x86_64
os: Ubuntu_16.04
language: go
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u16go
defaultVersion: 1.12.14
- architecture: x86_64
os: Ubuntu_18.04
language: node
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u18node
isDefault: true
defaultVersion: 10.18.0
- architecture: x86_64
os: Ubuntu_18.04
language: java
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u18java
defaultVersion: 13
- architecture: x86_64
os: Ubuntu_18.04
language: cpp
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u18cpp
defaultVersion: 9.0.0
- architecture: x86_64
os: Ubuntu_18.04
language: go
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u18go
defaultVersion: 1.12.14
- architecture: x86_64
os: CentOS_7
language: node
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-c7node
isDefault: true
defaultVersion: 10.18.0
- architecture: x86_64
os: CentOS_7
language: java
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-c7java
defaultVersion: 11
- architecture: x86_64
os: CentOS_7
language: cpp
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-c7cpp
defaultVersion: 3.4.2
- architecture: x86_64
os: CentOS_7
language: go
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-c7go
defaultVersion: 1.12.14
- architecture: x86_64
os: WindowsServer_2019
language: node
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-w19node
defaultVersion: 10.18.0
- architecture: x86_64
os: WindowsServer_2019
language: java
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-w19java
defaultVersion: 11
- architecture: x86_64
os: WindowsServer_2019
language: cpp
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-w19cpp
defaultVersion: 9.0.0
- architecture: x86_64
os: WindowsServer_2019
language: go
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-w19go
defaultVersion: 1.12.14
- architecture: x86_64
os: WindowsServer_2019
language: dotnetcore
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-w19dotnetcore
isDefault: true
defaultVersion: 3.1
Logging Configuration
## Logging configuration
##
pipelines:
logging:
application:
## The log level: error | warn | info | debug | trace
level: warn
## Log rotation settings
rotation:
## The max file size at which enforce rotation. Can be any number greater than 0.
maxSizeMb: 10
## The number of backup files to maintain. Can be any number greater than 0.
maxFiles: 10
request:
rotation:
## Can be any number greater than 0.
maxSizeMb: 10
## Can be any number greater than 0.
maxFiles: 10
Changing the Default Docker Images
An organization may wish to change the runtime images used by default for all users in the Pipelines installation. For example, if your organization has particular dependencies that are not met by the standard default runtime images, you may wish to use a set of custom runtime images.
To accomplish this, the system administrator can update the Pipelines system YAML, then restart the installation by running the pipelines CLI with the upgrade command. See Installing Pipelines for more information about upgrading a Pipelines installation.
An example system.yaml default image configuration is shown below with default images for node and java. There should only be one image with isDefault for each combination of architecture and os and one image with each combination of architecture , os , and language .
runtime:
languageImages:
- architecture: x86_64
os: Ubuntu_16.04
language: node
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u16node
isDefault: true
defaultVersion: 10.18.0
- architecture: x86_64
os: Ubuntu_16.04
language: java
registryUrl: releases-docker.jfrog.io
image: jfrog/pipelines-u16java
- architecture is the architecture on which this Docker image should be used. Currently, only
x86_64is supported. - os is the operating system on which this Docker image will be selected. To use the same image on multiple operating systems, add another entry to languageImages for each operating system.
The current operating system choices are:- Ubuntu_18.04
- Ubuntu_16.04
- CentOS_7
- WindowsServer_2019
- language is the language, as listed in the step runtime configuration, for which this image should be used.
- registryUrl is the optional Docker registry where the image can be found.
- image is the image name, without tag.
- isDefault designates that this image should be used when no language is specified for the step. Only one image for each architecture and operating system should be default.
- defaultVersion is the default image tag, used if no language is listed in the step configuration. When a version is specified in the step configuration, it should match a tag for the image.
