Cloud customer?
Start for Free >
Upgrade in MyJFrog >
What's New in Cloud >

Search





Overview

A custom AMI enables you to use your own AMI as a node in Pipelines. Nodes launched using this new custom AMI would include all the customizations you made when you created the AMI.

This feature is only available for users using dynamic nodes in AWS.

Prerequisites

To perform the steps outlined in this section, you will need to have some experience with the following:

  • Building AMIs
  • Setting up Hashicorp Packer with your AWS account
Page Contents


Building a Custom Pipelines BuildPlane AMI using Hashicorp Packer 

A custom Pipelines build plane AMI using Hashicorp Packer can be built using AWS Cloudformation as well as the other dynamic nodepool operating systems that Pipelines supports (Centos7, Centos8, Ubuntu18, Ubuntu20, Windows19), with a few modifications.

To build the custom build plane AMI:

  1. Download the buildplane package (RPM for Centos or a tar file for Windows) to the machine that will be used to build the AMI.

    Example
    curl --request GET  --url https://my-pipelines-api-url/v1/passthrough/artifacts/buildPlane-x86_64-Ubuntu_20.04.deb --output buildPlane-x86_64-Ubuntu_20.04.deb
    buildPlane package names:
    buildPlane-x86_64-Ubuntu_18.04
    buildPlane-x86_64-Ubuntu_20.04.deb
    buildPlane-x86_64-Centos_7.rpm
    buildPlane-x86_64-CentOS_8.rpm
    buildPlane-x86_64-WindowsServer_2019.tar.gz
  2. Update your packer (or cloud formation) template to:
    1. Copy the buildplane file to /tmp/ 
    2. Install it (using dpkg, yum, or untar, depending on the type) 
    3. Run the boot.sh (or boot.ps1 if creating a W19 AMI). Path to this file is /var/opt/jfrog/pipelines/nodeInit/x86_64/<OS>/boot.sh 
  3. Build your AMI.
  4. Use the resulting AMI id in the next step.
    This should be done with every new pipelines version.
Example 'provisioners' section of a terraform template file using the Ubuntu 20 buildplane
  "provisioners": [
    {
      "type": "file",
      "source": "buildPlane-x86_64-Ubuntu_20.04.deb",
      "destination": "/tmp/buildPlane-x86_64-Ubuntu_20.04.deb"
    },
    {
      "type": "shell",
      "inline": [
        "sudo dpkg -i /tmp/buildPlane-x86_64-Ubuntu_20.04.deb",
        "sudo /var/opt/jfrog/pipelines/nodeInit/x86_64/Ubuntu_20.04/boot.sh"
      ]
    }

Adding the AMI image to Pipelines

To add the AWS AMI image to Pipelines, use the POST /v1/buildPlaneImages API.

Sample Request:

curl --location --request POST 'http://<pipelines_url>:8082/pipelines/api/v1/buildPlaneImages' \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data-raw '{
 "name": "eu-centrall-1 image for x86_64 Ubuntu_20.04",
 "masterIntegrationId": 46,
 "imageId": "ami-07f469736e6545c8a",
 "region": "eu-central-1",
 "isAvailable": true,
 "architecture": "x86_64",
 "operatingSystem": "Ubuntu_20.04"
}'

Sample Response:

  {
        "metadataPropertyBag": {},
        "id": 8,
        "name": "eu-centrall-1 image for x86_64 Ubuntu_20.04",
        "masterIntegrationId": 46,
        "imageId": "ami-07f469736e6545c8a",
        "region": "eu-central-1",
        "isAvailable": true,
        "architecture": "x86_64",
        "operatingSystem": "Ubuntu_20.04",
        "createdAt": "2021-01-06T00:00:00.000Z",
        "updatedAt": "2021-01-06T00:00:00.000Z"
  }

Creating a Node Pool

Now create a node pool using the above buildplane image Id. For example, my-custom-ami-v1175 image for x86_64 Ubuntu_20.04. For more information, see Managing Pipelines Node Pools.

  • No labels
Copyright © 2021 JFrog Ltd.