In this post, we’ll explore how you can leverage the power of Terraform to create and manage Jenkins jobs effortlessly. Before getting into the specifics, it’s necessary to understand how the underlying motivation drives this approach. In fact, tons of Kubernetes clusters have been maintained and managed, and each cluster has a different CI config such as GitOps CI or traditional CI, which branch is used for building, Pipeline script or Pipeline script from SCM so creating Jenkins jobs for microservices manually is not a good idea. For example: There are 50 git repositories that need to be deployed and configured CI on Jenkins, creating it manually is extremely frustrating and boring, not to mention managing and updating them as needed. Well, you know I don’t think I can do this without mistakes at all. Thanks to Terraform, you just need to create a template job and then put the git repositories list in, eventually these jobs will be created by some commands. In case, you want to make changes to these entire jobs at once, just edit the template file and apply again.
Prerequisites:
Jenkins server, please follow the previous post or you can use KodeKloud playground to skip this step.
Basic knowledge of Terraform.
Step 1: Create the first Jenkins job manually
As you know, there are a lot of ways to create jobs in Jenkins. Each project uses a different way to configure the Jenkins job, as well as integrate it with different services depending on its needs.
To make it simple, you can follow this step Create Jenkins Pipeline job with this GitHub repo and ignore the webhook step.
Step 2: Prepare a template
At this point, you need to get the XML file of this job. To do that, append /config.xml to the URL Jenkins job. For example https://yourjenkins.domain.com/job/example/config.xml
You can refer to more information here.
The job config.xml will be
In order to get the raw file, right-click and choose “View Page Source”
Now, create a new file job.xml from the job config.xml
Then, replace values that you want to customize your config for each job. For example, what you need to change: Url git repository, branch, disable
Eventually, the job.xml should be
<?xml version="1.0" encoding="UTF-8"?>
<flow-definition plugin="[email protected]_763b_31">
<actions>
<org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobAction plugin="[email protected]_6113dfc3" />
<org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobPropertyTrackerAction plugin="[email protected]_6113dfc3">
<jobProperties />
<triggers />
<parameters />
<options />
</org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobPropertyTrackerAction>
</actions>
<description />
<keepDependencies>false</keepDependencies>
<properties />
<definition class="org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition" plugin="[email protected]_dd74276262">
<scm class="hudson.plugins.git.GitSCM" plugin="[email protected]">
<configVersion>2</configVersion>
<userRemoteConfigs>
<hudson.plugins.git.UserRemoteConfig>
<url>${repo}</url>
</hudson.plugins.git.UserRemoteConfig>
</userRemoteConfigs>
<branches>
<hudson.plugins.git.BranchSpec>
<name>${branch}</name>
</hudson.plugins.git.BranchSpec>
</branches>
<doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
<submoduleCfg class="empty-list" />
<extensions />
</scm>
<scriptPath>Jenkinsfile</scriptPath>
<lightweight>true</lightweight>
</definition>
<triggers />
<disabled>${disabled}</disabled>
</flow-definition>
Step 3: Write Terraform code
In this step, create a new file main.tf with the following code.
provider "jenkins" {}
terraform {
required_providers {
jenkins = {
source = "taiidani/jenkins"
version = "0.9.3"
}
}
}
variable "branch" {
type = string
default = "main"
}
variable "disabled" {
type = string
default = "false"
}
locals {
services = [
"https://github.com/raymondbaoly/jenkins-webhook-example.git",
"https://github.com/raymondbaoly/jenkins-example.git"
]
}
resource "jenkins_job" "services" {
count = length(local.services)
name = replace(basename(local.services[count.index]), ".git", "")
template = templatefile("${path.module}/job.xml", {
repo = local.services[count.index]
branch = var.branch
disabled = var.disabled
})
}
As you can see, the provider taiidani/jenkins is used, please refer to this document for more information.
Basically, Terraform will loop the locals.services and create a job with variable branch and disabled. If you have lots of services, append the links to locals.services. Whenever you want to change something for these jobs, just change the variable, locals.services, job.xml.
Step 4: Start creating the jobs
Now, use the main.tf file for creating multiple jobs with Terraform based on job.xml.
Firstly, run the init command
$ terraform init
Secondly, provide the credentials via the JENKINS_USERNAME, JENKINS_PASSWORD, and JENKINS_URL with environment variables.
$ export JENKINS_URL="http://jenkins.url:port"
$ export JENKINS_USERNAME="username"
$ export JENKINS_PASSWORD="password"
Finally, you can run terraform plan and apply it now
$ terraform plan
The follow output as:
Now apply it
$ terraform apply -auto-approve
The follow output as:
At this point, go to the Jenkins page. And then there are two new jobs, jenkins-example and jenkins-webhook-example.
Step 5: Verify the Jenkins jobs
To verify the jobs that are created correctly, you can go to jenkins-webhook-example, which is used for creating the example job in Step 1, and try to build it again.
Great! Everything is working well. Now the example job should be deleted.
Assuming that you need to update some config to all of the jobs. For example, you need to change the building branch from “main” to “building”.
In this case, you just need to adjust the variable branch to “building”
And then run the “terraform apply”
$ terraform apply -auto-approve
Finally, go to the detail Jenkins jobs, Branches to build is updated.
By automating Jenkins job creation with Terraform, teams can enhance their DevOps practices, improve productivity, and ensure consistency across their build and deployment pipelines. This automation enables faster deployment cycles and reduces human errors. For instance, when you have to deploy a new service and add it to CI/CD pipeline in a cluster that has been running for years, you basically just need to add the git repository link and run the “terraform apply” command.
Link code: https://github.com/raymondbaoly/terraform-creates-jenkins-jobs