Enterprise Jenkins on Azure, Part 2

Enterprise Jenkins on Azure, Part 3

March 29th, 2018 Posted by Cloudbees, DevOps 1 comment

This is the final post, part 3 of a series:

Part 1: Overview and Deploy Azure Resources
Part 2: Deploy Jenkins Operations Center, Master and Shared Agent
Part 3: The Jenkinsfile, tweaks and tips

In this post I describe the Jenkinsfile that defines the build pipeline, a little troubleshooting and tweaking, and wrap up the series.

Jenkinsfile

Below is the Groovy code in my Jenkinsfile, followed by a detailed description. As mentioned earlier, I made minimal changes to the demo so this is Scripted Pipeline Syntax, not the newer Declarative Pipeline Syntax.

 

 

The node clause:

  • Specifies that enclosed code must execute on an agent that has a label “linux”. The shared agent managed by CJOC has a matching label (see my previous post).
  • Consists of 4 stages, specified by the stage keyword. They correspond to columns in the stage view that the Jenkins Pipeline job shows.

 

The withCredentials clause:

  • Parameter credentialsId has been assigned a value that matches the ID (“azure-deployment-id”) of the FTP deployment credential that I created on my Jenkins client master (i.e. CloudBees Jenkins Enterprise).
  • Credential properties are available to the Pipeline code as environment variables, e.g. env._user
  • The line below builds a command for deploying a file via FTPS, combining environment variables and Groovy variables. In Groovy, double-quotes denote an interpolated string, e.g. Groovy will replace a placeholder like ${local_path} by the value of variable local_path.

 

I made a few changes to the original Jenkinsfile:

  1. The target variable: this is the deployment directory for a Web App Service created using the Azure Marketplace (if created by the Azure configuration UI, the directory would be: /site/wwwroot/webapps).
  2. The Pipeline git command’s url argument: my personal GitHub repository.
  3. Shell curl command: enclose password argument in quote characters.

The first two changes were obviously needed. Then, on trying to execute the job, it failed with a shell error:

 

But there’s no “)” in the curl command! To debug I added the command below:

 

The Jenkins console log below (edited for readability) shows the expanded curl command. Note that Jenkins has masked authentication fields (“****”):

 

The username and password for FTPS deployment are included in the authority part of the URL; the console log shows that only the username is quoted:

 

That helped to jog my memory: I had made up a password that included a “)” character. To fix the error I added quotes around the password as below; that satisfied the shell.

 

The demo in action

I made a short screencast to show the Jenkins Pipeline in action, see below:

  1. The web application viewed in a browser after deploying it to Azure App Service.
  2. A change to the source code and corresponding push to GitHub; the push triggers a webhook request to Jenkins.
  3. Jenkins responds: add a job to the build queue, lease an agent from Operations Center, execute the build, show progress on the stage view.
  4. Refresh the browser to see that the application was updated.

 

Conclusions

I found the Azure portal straightforward as a web interface for creating and configuring resources. With configuration set up, I can get Azure to provision resource instances whenever I need them.

The demo configures Jenkins for optimal scalability:

  1. With the distributed builds model, a Jenkins master delegates jobs to one or more agents whose number can scale up to the available machines. This frees up the master to handle HTTP requests to start jobs, both manual requests (interactive users) and automated requests (webhooks).
  2. Given a high volume of HTTP requests, we can add masters in order to distribute request processing across multiple masters. This is made possible by Operations Center, which can configure one or more masters, including credentials, backup policies, plugin whitelists and versions.
  3. Operations Center can orchestrate (i.e. scale up/down) jobs from all masters across the pool of shared agents. (Note that a master can have its own dedicated agents, e.g. to guarantee minimum throughput, but it means that some dedicated agents could remain idle while another master has several jobs queued.)

Distributed HTTP requests and load orchestration (points 2 and 3 above) are not scalable with standard Jenkins (OSS).

 

Scope for enterprise-scale continuous delivery

Below are a few steps for going further, towards continuous delivery at enterprise scale.

  • Automate the provisioning of Azure resources; write scripts, or use a configuration management tool like Ansible or Puppet.
  • Containerize builds using the Docker Plugin for Jenkins. This makes it possible to use ephemeral build agents, typically Docker containers. A container encapsulates the build environment, can run on any machine (portable), and starts in a fraction of the time required by a virtual machine.
  • Security: enforce role-based access control (RBAC) using the RBAC Plugin for Jenkins.
  • Analytics: use CloudBees Jenkins Analytics to monitor the health and usage dynamics of masters across a Jenkins cluster.

 

References

 

Jenkins

 

Azure guides

 

Tested on software versions

  • CloudBees Jenkins Operations Center 2.7.21.1-rolling
  • CloudBees Jenkins Enterprise 2.7.21.1-rolling

One comment

[…] Part 1: Overview and Deploy Azure Resources Part 2: Deploy Jenkins Operations Center, Master and Shared Agent Part 3: The Jenkinsfile, tweaks and tips […]

Leave a Reply