Jenkins is a popular open-source automation server used to set up continuous integration and delivery (CI/CD) for your software projects. You can host your Jenkins deployment in Azure or extend your existing Jenkins configuration using Azure resources. Jenkins plug-ins are also available to simplify CI/CD of your applications to Azure.
Jenkins Advantage
Jenkins is being managed by the community which is very open. Every month, they hold public meetings and take inputs from the public for the development of the Jenkins project.
So far around 280 tickets are closed, and the project publishes stable release every three months.
As technology grows, so does Jenkins. So far Jenkins has around 320 plugins published in its plugins database. With plugins, Jenkins becomes even more powerful and feature rich.
Jenkins tool also supports cloud-based architecture so that you can deploy Jenkins in cloud-based platforms.
The reason why Jenkins became popular is that it was created by a developer for developers.
Jenkins Disadvantages
Its interface is out dated and not user friendly compared to current UI trends.
Though Jenkins is loved by many developers, it’s not that easy to maintain it because Jenkins runs on a server and requires some skills as server administrator to monitor its activity.
One of the reasons why many people don’t implement Jenkins is due to its difficulty in installing and configuring Jenkins.
Continuous integrations regularly break due to some small setting changes. Continuous integration will be paused and therefore requires some developer attention.
Real work Case Study
Architecture
This architecture supports disaster recovery with Azure services but does not cover more advanced scale-out scenarios involving multiple primaries or high availability (HA) with no downtime. For general insights about the various Azure components, including a step-by-step tutorial about building out a CI/CD pipeline on Azure.
The focus of this document is on the core Azure operations needed to support Jenkins, including the use of Azure Storage to maintain build artifacts, the security items needed for SSO, other services that can be integrated, and scalability for the pipeline. The architecture is designed to work with an existing source control repository. For example, a common scenario is to start Jenkins jobs based on GitHub commits.
Components
The architecture consists of the following components:
Resource group. A resource group is used to group Azure assets so they can be managed by lifetime, owner, and other criteria. Use resource groups to deploy and monitor Azure assets as a group and track billing costs by resource group. You can also delete resources as a set, which is useful for test deployments.
Jenkins server. A virtual machine is deployed to run Jenkins as an automation server and serves as Jenkins Primary. In the section Install and configure Jenkins, you'll see how to install Jenkins on a new VM.
Virtual network. A virtual network connects Azure resources to each other and provides logical isolation. In this architecture, the Jenkins server runs in a virtual network.
Subnets. The Jenkins server is isolated in a subnet to make it easier to manage and segregate network traffic without affecting performance.
Network security groups. Use network security groups to restrict network traffic from the Internet to the subnet of a virtual network.
Managed disks. A managed disk is a persistent virtual hard disk (VHD) used for application storage and also to maintain the state of the Jenkins server and provide disaster recovery. Data disks are stored in Azure Storage. For high performance, premium storage is recommended.
Azure Blob storage. The Windows Azure Storage Learn how to Azure Blob storage to store the build artifacts that are created and shared with other Jenkins builds.
Azure Active Directory (Azure AD). Azure AD supports user authentication, allowing you to set up SSO. Azure AD service principals define the policy and permissions for each role authorization in the workflow, using Azure role-based access control (Azure RBAC). Each service principal is associated with a Jenkins job.
Azure Key Vault. To manage secrets and cryptographic keys used to provision Azure resources when secrets are required, this architecture uses Azure Key Vault.
Azure monitoring services. This service monitors the Azure virtual machine hosting Jenkins. This deployment monitors the virtual machine status and CPU utilization and sends alerts.
Recommendations
The following recommendations apply for most scenarios. Follow these recommendations unless you have a specific requirement that overrides them.
Scalability considerations
Jenkins can dynamically scale to support workloads as needed. For elastic builds, do not run builds on the Jenkins primary server. Instead, offload build tasks to Jenkins agents, which can be elastically scaled in and out as need. Consider two options for scaling agents:
Scale Jenkins deployments with Azure VM Agents. VM Agents enable elastic scale-out for agents and can use distinct types of virtual machines. You can specify a different base image from Azure Marketplace or use a custom image
Run a container as an agent in either Azure Container Service with Kubernetes, or Azure Container Instances.
Virtual machines generally cost more to scale than containers. To use containers for scaling, however, your build process must run with containers.
Also, use Azure Storage to share build artifacts that may be used in the next stage of the pipeline by other build agents.
Scaling the Jenkins server
When you create a VM and install Jenkins, you can specify the size of the VM. Selecting the correct VM server size depends on the size of the expected workload. The Jenkins community maintains a selection guide to help identify the configuration that best meets your requirements. Azure offers many sizes for Linux VMs to meet any requirements. For more information about scaling the Jenkins primary, see the Jenkins community of best practices, which also includes details about scaling Jenkins.
Availability considerations
Availability in the context of a Jenkins server means being able to recover any state information associated with your workflow, such as test results, libraries you have created, or other artifacts. Critical workflow state or artifacts must be maintained to recover the workflow if the Jenkins server goes down. To assess your availability requirements, consider two common metrics:
Recovery Time Objective (RTO) specifies how long you can go without Jenkins.
Recovery Point Objective (RPO) indicates how much data you can afford to lose if a disruption in service affects Jenkins.
In practice, RTO, and RPO imply redundancy and backup. Availability is not a question of hardware recovery - that is part of Azure - but rather ensuring you maintain the state of your Jenkins server. Microsoft offers a service level agreement (SLA) for single VM instances. If this SLA doesn't meet your uptime requirements, make sure you have a plan for disaster recovery, or consider using a multi-primary Jenkins server deployment (not covered in this document).
Security considerations
Use the following approaches to help lock down security on a basic Jenkins server, since in its basic state, it is not secure.
Set up a secure way to log into the Jenkins server. This architecture uses HTTP and has a public IP, but HTTP is not secure by default. Consider setting up HTTPS on the Nginx server being used for a secure logon.
Ensure that the Jenkins configuration prevents cross site request forgery (Manage Jenkins > Configure Global Security). This option is the default for Microsoft Jenkins Server.
Configure read-only access to the Jenkins dashboard by using the Matrix Authorization Strategy Plugin.
Use Azure RBAC to restrict the access of the service principal to the minimum required to run the jobs. This level of security helps limit the scope of damage from a rogue job.
Jenkins jobs often require secrets to access Azure services that require authorization, such as Azure Container Service. Use Key Vault to manage these secrets securely. Use Key Vault to store service principal credentials, passwords, tokens, and other secrets.
To get a central view of the security state of your Azure resources, use Microsoft Defender for Cloud. Defender for Cloud monitors potential security issues and provides a comprehensive picture of the security health of your deployment. Defender for Cloud is configured per Azure subscription. Enable security data collection as described in the Microsoft Defender for Cloud quick start guide. When data collection is enabled, Defender for Cloud automatically scans any virtual machines created under that subscription.
Manageability considerations
Use resource groups to organize the Azure resources that are deployed. Deploy production environments and development/test environments in separate resource groups, so that you can monitor each environment's resources and roll up billing costs by resource group. You can also delete resources as a set, which is useful for test deployments.
Azure provides several features for monitoring and diagnostics of the overall infrastructure. To monitor CPU usage, this architecture deploys Azure Monitor. For example, you can use Azure Monitor to monitor CPU utilization, and send a notification if CPU usage exceeds 80 percent. (High CPU usage indicates that you might want to scale up the Jenkins server VM.) You can also notify a designated user if the VM fails or becomes unavailable.