fbpx

Jenkins Plug-In

The Jenkins plug-in helps clients do more with a Jenkins setup. It does so by allowing users to configure Jenkins to automatically scale designated Amazon EC2 Spot Instances as Slaves up or down depending on the number jobs pending on the Jenkins queue.

 

How It Works

 

The Spotinst Jenkins plug-in automatically scales up&down instances based on the number of jobs in its queue. The nodes that are being provisioned run a startup script and connect as Slave nodes to the master to immediately start running jobs.

 

How To Get Started

1. Generate a Spotinst API Access Token

Login to the Spotinst Console and then go to Settings -> API -> Permanent Token

Once the API Access Token is generated; Save it for later use for the Jenkins configuration.

 

2. Create an Elastigroup with a Proper Startup Script

Create an Elastigroup with your proper Region, AMI, Instance Types and the following startup script:

#!/bin/bash
install_deps() {
  log_info "Installing dependencies"
  # Install deps.
  packages=$1
  for package in $packages; do
    installed=$(which $package)
    not_found=$(echo $(expr index "$installed" "no $package in"))
    if [ -z $installed ] && [ "$not_found" == "0" ]; then
      log_info "Installing $package"
      if [ -f /etc/redhat-release ] || [ -f /etc/system-release ]; then
        yum install -y $package
      elif [ -f /etc/arch-release ]; then
        pacman install -y $package
      elif [ -f /etc/gentoo-release ]; then
        emerge install -y $package
      elif [ -f /etc/SuSE-release ]; then
        zypp install -y $package
      elif [ -f /etc/debian_version ]; then
        apt-get install -y $package
      fi
      log_info "$package successfully installed"
    fi
  done
}
EC2_INSTANCE_ID="$(curl http://169.254.169.254/latest/meta-data/instance-id)"
JENKINS_MASTER_IP="IP:PORT"
# Install Java If not already installed
install_deps "jre"
# Get The Jenkins Slave JAR file
curl http://${JENKINS_MASTER_IP}/jnlpJars/slave.jar --output /tmp/slave.jar
# Run the Jenkins Slave JAR
java -jar /tmp/slave.jar -jnlpUrl http://${JENKINS_MASTER_IP}
/computer/${EC2_INSTANCE_ID}/slave-agent.jnlp &

 

Use the flagjnlpCredentials in the command with your username / password or token (such as the GitHub access token if GitHub is being used to authenticate with Jenkins).

# Run the Jenkins slave JAR
java -jar /tmp/slave.jar -jnlpCredentials user:password/token -jnlpUrl http://${JENKINS_MASTER_IP}/computer/${EC2_INSTANCE_ID}/slave-agent.jnlp &
 We recommend using the Amazon Standard AMI (CentOS based)

 

3. Change the default slave connection port

By default, slaves try to connect to a random port. This requires opening firewall rules to allow all ports to be open to ensure successful communication from the slave to the master. An alternative is to configure a static port for your Jenkins slaves by going into Manage Jenkins –>  Global Security –> set a static TCP port for JNLP agents.

4. Install the Jenkins Plugin and Add Your Spotinst API Token and Elastigroup ID

Now login to the Jenkins console, and install the Spotinst Plugin from the available Plugins list.
Once the plugin is installed, Navigate to the main page, "Manage Jenkins" > "Configure System", scroll down to the “Spotinst” Section, and add the API Token generated in the previous step. Then click on "Validate Token" to make sure that the token is valid.

Once the Spotinst Token is set, scroll down towards the bottom to the “Cloud” section. Click on the "Add a new cloud" button, and select the "Spotinst" option. If you are familiar with our Jenkins integration you will notice that there are now more fields to choose from. For more information on each field hover over the information button on the right side of each field. Specify your "Elastigroup ID" and "Idle minutes before termination".

 We highly recommend configuring your Elastigroup for vCPU weighting to ensure the most efficient use of your resources (CentOS based)

When creating an Elastigroup select “vCPU” as the unit as shown below. This can be found on the “Strategy & Capacity” page of your Elastigroup configuration.

That’s all. From now on, the Jenkins Master will automatically launch new instances through the Spotinst API, and will terminate them as they get unused.