YAML File Format
The YAML file is constructed with keys and entities, using the following key: [entity]
format.
- Shared configurations specified under the shared section, are used by all micro-services.
- Micro-service specific configurations are set under each micro-service section and override similar shared configurations for the specific micro-service.
Supported Configurations
Shared Configurations
## Pipelines config ## shared: ## Artifactory configuration ## artifactory: ## Artifactory URL ## baseUrl: "" ## Unified UI URL ## baseUrlUI: "" ## Artifactory Service ID ## artifactoryServiceId: "" ## Artifactory License ID ## licenseId: "" ## Pipelines Service ID ## serviceId: "" ## Proxy to connect to Artifactory ## proxy: url: "" username: "" password: "" ## Logging configuration ## logging: ## UI Refresh rate seconds ## view: refreshRate: 10s ## For build time seconds metrics ## metrics: filePath: /opt/jfrog/pipelines/var/log/api-metrics_events.log console: false rotation: maxSizeMb: 25 maxFiles: 10 maxAgeDays: 365 compress: true intervalMs: 900000 ## Metrics configuration ## metrics: enabled: true ## Router configuration ## ## Router configuration ## router: image: "jfrog/pipelines-router" ip: "" ## This port is used by services on the node locally to connect to the router accessPort: 8046 ## This port is used externally to connect to the router dataPort: 8082 ## The join key from Artifactory in order to communicate with Artifactory joinKey: "" security: masterKey: "" ## Database configuration ## db: type: "postgres" ## The following settings, except connectionString and version, are used when the Pipelines installer manages a database container ip: "" port: 5432 name: "pipelinesdb" username: "apiuser" password: "" ## Pipelines installer will install PostgreSQL 12.3 by default image: "jfrog/pipelines-postgres" version: 12.3 externalUrl: "" ## The connectionString contains everything used to connect to the database, e.g., postgres://username:password@hostname:5432/databaseName connectionString: "" ## RabbitMQ configuration ## msg: ## Pipelines installer will install RabbitMQ 3.8.3 by default image: "jfrog/pipelines-msg" version: 3.8.3 ## IP address at which RabbitMQ can be reached ip: "" ## Port on which RabbitMQ can be reached over amqp or amqps port: 30200 ## Port on which RabbitMQ can be reached over http or https (RabbitMQ admin interface) adminPort: 30201 ## Cookie used for RabbitMQ clustering erlangCookie: "" ## Username and password of an admin user username: "admin" password: "" ## RabbitMQ exchange for Pipelines queues defaultExchange: pipelinesEx ## RabbitMQ vhost for queues providing messages to build nodes amqpVhost: pipelines ## RabbitMQ vhost for queues used internally by Pipelines amqpRootVhost: pipelinesRoot ## Protocol for amqp or amqps connections protocol: amqp ## Queues created by default for Pipelines services queues: - "core.pipelineSync" - "core.runTrigger" - "core.stepTrigger" - "core.marshaller" - "core.logup" - "cluster.init" - "www.signals" - "core.nexec" - "core.hookHandler" ## Protocol, username, and password to connect to the RabbitMQ management interface ui: protocol: http username: "monitor" password: "" ## URLs at which it is possible to reach RabbitMQ from Pipelines and build nodes, for example with a load balancer external: ## URL to connect to RabbitMQ from build nodes, using the username and password above url: "" rootUrl: "" adminUrl: "" ## Vault configuration ## vault: ## Pipelines installer will install Vault 1.2.2 by default image: "jfrog/pipelines-vault" version: 1.2.2 ## Address at which Vault is reachable ip: "" ## Port for Vault API port: 30100 ## Port used for Vault clustering clusterPort: 30101 ## Root token to connect to Vault token: "" ## Unseal keys when Vault installation is managed by the Pipelines installer unsealKeys: - "" - "" - "" - "" - "" ## Redis configuration ## redis: ip: "" port: 6379 ## Pipelines installer will install Redis 5.0.5 by default image: "jfrog/pipelines-redis" version: 5.0.5 clusterEnabled: true ## NewRelic APM configuration ## newRelic: enabled: false license: "" appName: "" logging: debug: false distributedTracer: enabled: false
Core Services Configurations
## This section is used for bringing up the core services and setting up ## configurations required by the installer & the services ## core: ## id is automatically determined based on the current hostname ## or set using the SHARED_NODE_ID environment variable. ## id: "" ## IP address of the local machine, used by default when installing services installerIP: "" ## Default password when installing Postgres, RabbitMQ, etc. installerAuthToken: "" installerImage: "jfrog/pipelines-installer" ## Registry from which to download Pipelines images as part of the installation and ## dynamic kubernetes build node creation registryUrl: "releases-docker.jfrog.io" ## Kubernetes secret name to authenticate against the registry specified at ## registryUrl when downloading build node images imageRegistrySecret: "" ## The following operating system variables are set by the installer to reflect the environment os: "" osDistribution: "" architecture: "" dockerVersion: "" ## The runMode determines the log level in Pipelines services. Choices are devmode, beta, and production. runMode: "" ## User and group for the installer on the local machine user: "" group: "" noVerifySsl: false ignoreTLSErrors: false ## Set by the installer and updated on upgrades to reflect the version of Pipelines installed controlplaneVersion: "" buildplaneVersion: "" ## Global proxy settings, to be applied to all services ## proxy: httpProxy: "" httpsProxy: "" noProxy: "" username: "" password: "" ## Mailserver settings ## mailserver: host: "" port: "" username: "" password: "" tls: "" ssl: "" apiRetryIntervalMs: 3000 accountSyncFrequencyHr: 1 kubernetesNamespace: "pipelines" imageRegistrySecret: "pipelines-reg-login" ## Duration for which deleted resources in a pipeline may be restored before permanently deleting hardDeleteIntervalInMins: 60 ## Number of configuration backups to keep when running "pipelines upgrade" configBackupCount: 5 ## The time at which Pipelines was last updated lastUpdateTime: "" ## URL to which general statistics on usage are posted callHomeUrl: "https://api.bintray.net/products/jfrog/pipelines/stats/usage" ## Posting usage statistics can be disabled by setting allowCallHome to false allowCallHome: true ## How often Pipelines services report health statistics internally serviceInstanceHealthCheckIntervalInMins: 1 serviceInstanceStatsCutOffIntervalInHours: 24 rabbitmqHealthCheckIntervalInMins: 1 artifactoryHealthCheckIntervalInMins: 1 dbHealthCheckIntervalInMins: 1 dbHealthCheckTimeoutInSeconds: 2 customCACertsPath: "" ## Enable signed pipelines feature signedPipelinesEnabled: true ## Enable pipeline source auto-sync feature autoSyncResourceIfOutdated: true ## Store version history for resources storeVersionHistory: true ## Pipelines state retention policy ## retentionPolicy: enabled: false ## State beyond this age is removed, except for minRuns ## maxAgeDays: 90 ## Minimum runs per pipeline for which state is retained ## minRuns: 10 ## Service configuration ## services: api: name: api ## Port on which the Pipelines API is listening port: 30000 externalUrl: "" image: "jfrog/pipelines-api" www: name: www ## Port on which the Pipelines UI is listening port: 30001 externalUrl: "" sessionSecret: "" image: "jfrog/pipelines-www" pipelineSync: name: pipelineSync image: "jfrog/pipelines-micro" runTrigger: name: runTrigger image: "jfrog/pipelines-micro" stepTrigger: name: stepTrigger image: "jfrog/pipelines-micro" cron: name: cron image: "jfrog/pipelines-micro" nexec: name: nexec image: "jfrog/pipelines-micro" hookHandler: name: hookHandler image: "jfrog/pipelines-micro" marshaller: name: marshaller image: "jfrog/pipelines-micro" logup: name: logup image: "jfrog/pipelines-micro"
Runtime Configuration
## Runtime configuration ## runtime: ## Name of the Artifactory repository used to store state and step logs for Pipelines rootBucket: "" defaultMinionCount: 1 nodeCacheIntervalMS: 600000 jobConsoleBatchSize: 10 jobConsoleBufferIntervalMs: 3 ## Default maximum disk usage allowed before a node is stopped maxDiskUsagePercentage: 90 ## Maximum and default timeout for steps in a node pool stepTimeoutMS: 3600000 ## Day of week on which cached nodes are replaced nodeStopDayOfWeek: 0 ## Duration after which cached nodes are replaced nodeStopIntervalDays: 30 maxNodeCheckInDelayMin: 15 defaultMinionInstanceSize: "c4.large" ## Settings that control the types of node pools that can be added allowDynamicNodes: true allowCustomNodes: true isSaas: false ## Default images used to run steps, more information in the following section languageImages: - architecture: x86_64 os: Ubuntu_16.04 language: node registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u16node isDefault: true defaultVersion: 10.18.0 - architecture: x86_64 os: Ubuntu_16.04 language: java registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u16java defaultVersion: 13 - architecture: x86_64 os: Ubuntu_16.04 language: cpp registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u16cpp defaultVersion: 9.0.0 - architecture: x86_64 os: Ubuntu_16.04 language: go registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u16go defaultVersion: 1.12.14 - architecture: x86_64 os: Ubuntu_18.04 language: node registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u18node isDefault: true defaultVersion: 10.18.0 - architecture: x86_64 os: Ubuntu_18.04 language: java registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u18java defaultVersion: 13 - architecture: x86_64 os: Ubuntu_18.04 language: cpp registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u18cpp defaultVersion: 9.0.0 - architecture: x86_64 os: Ubuntu_18.04 language: go registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u18go defaultVersion: 1.12.14 - architecture: x86_64 os: CentOS_7 language: node registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-c7node isDefault: true defaultVersion: 10.18.0 - architecture: x86_64 os: CentOS_7 language: java registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-c7java defaultVersion: 11 - architecture: x86_64 os: CentOS_7 language: cpp registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-c7cpp defaultVersion: 3.4.2 - architecture: x86_64 os: CentOS_7 language: go registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-c7go defaultVersion: 1.12.14 - architecture: x86_64 os: WindowsServer_2019 language: node registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-w19node defaultVersion: 10.18.0 - architecture: x86_64 os: WindowsServer_2019 language: java registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-w19java defaultVersion: 11 - architecture: x86_64 os: WindowsServer_2019 language: cpp registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-w19cpp defaultVersion: 9.0.0 - architecture: x86_64 os: WindowsServer_2019 language: go registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-w19go defaultVersion: 1.12.14 - architecture: x86_64 os: WindowsServer_2019 language: dotnetcore registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-w19dotnetcore isDefault: true defaultVersion: 3.1
Logging Configuration
## Logging configuration ## pipelines: logging: application: ## The log level: error | warn | info | debug | trace level: warn ## Log rotation settings rotation: ## The max file size at which enforce rotation. Can be any number greater than 0. maxSizeMb: 10 ## The number of backup files to maintain. Can be any number greater than 0. maxFiles: 10 request: rotation: ## Can be any number greater than 0. maxSizeMb: 10 ## Can be any number greater than 0. maxFiles: 10
Changing the Default Docker Images
An organization may wish to change the runtime images used by default for all users in the Pipelines installation. For example, if your organization has particular dependencies that are not met by the standard default runtime images, you may wish to use a set of custom runtime images.
To accomplish this, the system administrator can update the Pipelines system YAML, then restart the installation by running the pipelines CLI with the upgrade command. See Installing Pipelines for more information about upgrading a Pipelines installation.
An example system.yaml default image configuration is shown below with default images for node and java. There should only be one image with isDefault
for each combination of architecture
and os
and one image with each combination of architecture
, os
, and language
.
runtime: languageImages: - architecture: x86_64 os: Ubuntu_16.04 language: node registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u16node isDefault: true defaultVersion: 10.18.0 - architecture: x86_64 os: Ubuntu_16.04 language: java registryUrl: releases-docker.jfrog.io image: jfrog/pipelines-u16java
- architecture is the architecture on which this Docker image should be used. Currently, only
x86_64
is supported. - os is the operating system on which this Docker image will be selected. To use the same image on multiple operating systems, add another entry to languageImages for each operating system.
The current operating system choices are:- Ubuntu_18.04
- Ubuntu_16.04
- CentOS_7
- WindowsServer_2019
- language is the language, as listed in the step runtime configuration, for which this image should be used.
- registryUrl is the optional Docker registry where the image can be found.
- image is the image name, without tag.
- isDefault designates that this image should be used when no language is specified for the step. Only one image for each architecture and operating system should be default.
- defaultVersion is the default image tag, used if no language is listed in the step configuration. When a version is specified in the step configuration, it should match a tag for the image.