Level 4: Jenkins Multistage Pipeline

Jana-Polianskaja · September 4, 2024, 5:30pm

I’m looking for a little help with a task I’m working on.

The development team of xFusionCorp Industries is working on to develop a new static website and they are planning to deploy the same on Nautilus App Servers using Jenkins pipeline. They have shared their requirements with the DevOps team and accordingly we need to create a Jenkins pipeline job. Please find below more details about the task:

Click on the Jenkins button on the top bar to access the Jenkins UI. Login using username admin and password Adm!n321.

Similarly, click on the Gitea button on the top bar to access the Gitea UI. Login using username sarah and password Sarah_pass123.

There is a repository named sarah/web in Gitea that is already cloned on Storage server under /var/www/html directory.

Update the content of the file index.html under the same repository to Welcome to xFusionCorp Industries and push the changes to the origin into the master branch.
Apache is already installed on all app Servers its running on port 8080.
Create a Jenkins pipeline job named deploy-job (it must not be a Multibranch pipeline job) and pipeline should have two stages Deploy and Test ( names are case sensitive ). Configure these stages as per details mentioned below.
a. The Deploy stage should deploy the code from web repository under /var/www/html on the Storage Server, as this location is already mounted to the document root /var/www/html of all app servers.
b. The Test stage should just test if the app is working fine and website is accessible. Its up to you how you design this stage to test it out, you can simply add a curl command as well to run a curl against the LBR URL (http://stlb01:8091) to see if the website is working or not. Make sure this stage fails in case the website/app is not working or if the Deploy stage fails.

Click on the App button on the top bar to see the latest changes you deployed. Please make sure the required content is loading on the main URL http://stlb01:8091 i.e there should not be a sub-directory like http://stlb01:8091/web etc.

I was able to update the code in Gitea (GIT_CREDS - credentials for sarah) and on the apps (feel free to check my logs!). The app opens with the right text (I restart manually httpd on each app), which is great, but I ran into an error.

- deployment is not configured to deploy the latest changes.

My pipeline:

pipeline {
    agent {
        label 'ststor01'
    }

    stages {
        stage('Deploy') {
            steps {
                // Connect to GIT and download the repo code
                git credentialsId: 'GIT_CREDS', url: 'http://git.stratos.xfusioncorp.com/sarah/web.git'

                // Update index.html content
                sh 'cd /var/www/html && echo "Welcome to xFusionCorp Industries" > index.html'

                // Commit and push changes to the master branch
                sh 'cd /var/www/html && git add . && git commit -m "Updated index.html" && git push origin master'
            }
        }

        stage('Test') {
            environment {
                INDEX_CONTENT = 'Welcome to xFusionCorp Industries'
            }
            steps {
                // Test if the website is accessible on each app server
                sh '((curl http://stapp01:8080/ | grep -F "$INDEX_CONTENT") && true)'
                sh '((curl http://stapp02:8080/ | grep -F "$INDEX_CONTENT") && true)'
                sh '((curl http://stapp03:8080/ | grep -F "$INDEX_CONTENT") && true)'

                // Test if the website is accessible on the LBR URL
                sh '((curl http://stlb01:8091/ | grep -F "$INDEX_CONTENT") && true)'
            }
        }
    }
}

Here are the logs:

Started by user admin
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Storage Server in /var/www/html/workspace/deploy-job
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Deploy)
[Pipeline] git
The recommended git tool is: NONE
using credential GIT_CREDS
Cloning the remote Git repository
Cloning repository http://git.stratos.xfusioncorp.com/sarah/web.git
 > git init /var/www/html/workspace/deploy-job # timeout=10
Fetching upstream changes from http://git.stratos.xfusioncorp.com/sarah/web.git
 > git --version # timeout=10
 > git --version # 'git version 2.43.0'
using GIT_ASKPASS to set credentials 
 > git fetch --tags --force --progress -- http://git.stratos.xfusioncorp.com/sarah/web.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url http://git.stratos.xfusioncorp.com/sarah/web.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision 5404b49d86e5472dfa4ba69e7547464f3f5c431d (refs/remotes/origin/master)
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5404b49d86e5472dfa4ba69e7547464f3f5c431d # timeout=10
 > git branch -a -v --no-abbrev # timeout=10
 > git checkout -b master 5404b49d86e5472dfa4ba69e7547464f3f5c431d # timeout=10
Commit message: "Added index.html file"
First time build. Skipping changelog.
[Pipeline] sh
+ cd /var/www/html
+ echo 'Welcome to xFusionCorp Industries'
[Pipeline] sh
+ cd /var/www/html
+ git add .
warning: adding embedded git repository: workspace/deploy-job
hint: You've added another git repository inside your current repository.
hint: Clones of the outer repository will not contain the contents of
hint: the embedded repository and will not know how to obtain it.
hint: If you meant to add a submodule, use:
hint: 
hint: 	git submodule add <url> workspace/deploy-job
hint: 
hint: If you added this path by mistake, you can remove it from the
hint: index with:
hint: 
hint: 	git rm --cached workspace/deploy-job
hint: 
hint: See "git help submodule" for more information.
+ git commit -m 'Updated index.html'
[master bf41db8] Updated index.html
 37 files changed, 20 insertions(+), 1 deletion(-)
 create mode 100644 remoting.jar
 create mode 100644 remoting/jarCache/0F/3B4B41FA3E95DE9A5D0B01F64C9835.jar
 create mode 100644 remoting/jarCache/1B/AF066E906C2D5645EF057267B5AB40.jar
 create mode 100644 remoting/jarCache/1D/8AD0DCA05D29BE2918B7CD8EA31BB5.jar
 create mode 100644 remoting/jarCache/23/70AE4C89043DE01A2EE8D8CCCAB96F.jar
 create mode 100644 remoting/jarCache/29/CE85749769802BB3B9B5EC0154EFB5.jar
 create mode 100644 remoting/jarCache/2F/3C0F9EC3874520EE41A1934C7B2E9A.jar
 create mode 100644 remoting/jarCache/5C/D6A992CCDBC59C275E4551173EC1EB.jar
 create mode 100644 remoting/jarCache/60/0DAF5FB641B871BCE3E626C133F0F8.jar
 create mode 100644 remoting/jarCache/69/787ACF906149558778CA9537DBE5F2.jar
 create mode 100644 remoting/jarCache/76/6B1544A267DC9EA57735D913F3B493.jar
 create mode 100644 remoting/jarCache/7F/3172FD65A2490A8013AE6E984488DF.jar
 create mode 100644 remoting/jarCache/94/4F5C5D2B183F5FAED5D1B354920484.jar
 create mode 100644 remoting/jarCache/95/8B0E2513C4B529383E11756C097D49.jar
 create mode 100644 remoting/jarCache/AD/C9708EEF7E96B5E29C53F020BBF71B.jar
 create mode 100644 remoting/jarCache/B2/674E7818836879CAD5D217139F4228.jar
 create mode 100644 remoting/jarCache/B6/5B8DD2B322243EC275BB2100358740.jar
 create mode 100644 remoting/jarCache/B9/9EFFA5E646A9AE5DE8D1E356FB41FF.jar
 create mode 100644 remoting/jarCache/BF/13DEF758459566E91115E66C79D1D5.jar
 create mode 100644 remoting/jarCache/C2/6DEE0DB71344F5159FEBFCD6E52613.jar
 create mode 100644 remoting/jarCache/CB/73FD3478771189C622543BCC132215.jar
 create mode 100644 remoting/jarCache/CD/31FF9A61E9484CF8A8A61B736F2D5A.jar
 create mode 100644 remoting/jarCache/CD/D1747FEC8D94B5CB24C9C138BA843C.jar
 create mode 100644 remoting/jarCache/CE/0E24DB1122F70D676D89715500BD45.jar
 create mode 100644 remoting/jarCache/CE/573B3095056D420DB2A23E6DCB6CBF.jar
 create mode 100644 remoting/jarCache/DC/444A95509C7290ACBBC534313A524D.jar
 create mode 100644 remoting/jarCache/DE/34F9BF308B3FE96B83D99EEC31B42F.jar
 create mode 100644 remoting/jarCache/E2/C80DD7C22AC1D1F8AEDF1483998D8A.jar
 create mode 100644 remoting/jarCache/E3/EB2A29001670E23C6E776F8015C6EB.jar
 create mode 100644 remoting/jarCache/EF/4C25DC5D6EE68B901922435396D4FC.jar
 create mode 100644 remoting/logs/remoting.log.0
 create mode 100644 remoting/logs/remoting.log.0.lck
 create mode 160000 workspace/deploy-job
 create mode 100644 workspace/deploy-job@tmp/durable-baa4857d/jenkins-log.txt
 create mode 100644 workspace/deploy-job@tmp/durable-baa4857d/script.sh
 create mode 100644 workspace/deploy-job@tmp/durable-baa4857d/script.sh.copy
+ git push origin master
remote: . Processing 1 references        
remote: Processed 1 references in total        
To http://git.stratos.xfusioncorp.com/sarah/web.git
   5404b49..bf41db8  master -> master
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ curl http://stapp01:8080/
+ grep -F 'Welcome to xFusionCorp Industries'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    34  100    34    0     0  17000      0 --:--:-- --:--:-- --:--:-- 17000
Welcome to xFusionCorp Industries
+ true
[Pipeline] sh
+ curl http://stapp02:8080/
+ grep -F 'Welcome to xFusionCorp Industries'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    34  100    34    0     0  34000      0 --:--:-- --:--:-- --:--:-- 34000
Welcome to xFusionCorp Industries
+ true
[Pipeline] sh
+ curl http://stapp03:8080/
+ grep -F 'Welcome to xFusionCorp Industries'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    34  100    34    0     0  34000      0 --:--:-- --:--:-- --:--:-- 34000
Welcome to xFusionCorp Industries
+ true
[Pipeline] sh
+ curl http://stlb01:8091/
+ grep -F 'Welcome to xFusionCorp Industries'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    34  100    34    0     0    425      0 --:--:-- --:--:-- --:--:--   425
Welcome to xFusionCorp Industries
+ true
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Example:

[root@ststor01 html]# cat index.html 
Welcome to xFusionCorp Industries
[root@ststor01 html]# curl http://stlb01:8091
Welcome to xFusionCorp Industries

Alistair_KodeKloud · September 4, 2024, 5:44pm

I think you have misread the question.

Step 1 says simply to update the git repo - that’s before creating any pipeline!
So you can update the HTML file directly in the gitea UI.

The deployment pipeline should simply ensure that the latest version of what is in gitea is deployed to the target server - that is, a git pull in the right place should be sufficient.

It is very rare that a build or deployment pipeline commits directly to a source code repo - it’s generally considered bad practice. Tagging is usually the most they do.

Jana-Polianskaja · September 5, 2024, 9:55am

Thank you so much! It worked

arun1996 · November 20, 2024, 11:56am

can you please update the solution here, thanks in advance