Introduction
We all know CI - Continuous Integration and the next step is: CD - Continuous Delivery. Which is a combination of many steps to automate the build, test, release, and deploy process.
There are some options we can choose from such as Jenkins or the GitLab Integration (https://stackoverflow.com/questions/37429453/gitlab-ci-vs-jenkins).
We use Jenkins Pipelines (https://jenkins.io/doc/book/pipeline/) because I discovered the GitLab integration too late and at that time the Jenkins process was already finished. Otherwise, I would strongly recommend trying the GitLab integration to prevent the "Jenkins Plugin Hell". (Another alternative is https://de.atlassian.com/software/bamboo)
Basics
The goal is to see the CD process as part of the code and not as a separate part of development. Because of this reason, the process is described in script files that are committed to GIT.
Now we have the advantage that we don't need to edit the Jenkins-Job-Configuration every time we need to change the process. If we have a new server we can (theoretically) easily start our CD process without creating new jobs on the new server.
In the case of Jenkins, we need to store a "Jenkinsfile" under the root directory of our project. This Jenkinsfile can be written in the Jenkins-Pipeline language or in groovy or a mix of both.
Requirements
Jenkins needs to be at least at version 2 (because we need the Jenkins Pipeline)
Following Plugins are needed to get the example Jenkinsfile running
In the Jenkins "Global Tool Configuration" maven needs to be defined:
Example Jenkinsfile
Following steps are executed
- Update the project/maven/pom version
- Create a GIT-Tag with the new version
- Push the new version to the branch
- Upload the artifact to the Nexus Repository
- Commit the new version to the child/develop branch
- Deploy the new version to the AEM Author and Publish instance
The complete file
#!groovy
node {
def version
def webAppTarget = "xxx"
def sourceBranch = "develop"
def releaseBranch = "quality-assurance"
def nexusBaseRepoUrl = "http://xxx"
def repositoryUrl = "http://xxx"
def gitCredentialsId = "xxx"
def nexusRepositoryId = "xxx"
def configFileId = "xxx"
def mvnHome = tool 'M3'
def updateQAVersion = {
def split = version.split('\\.')
//always remove "-SNAPSHOT"
split[2] = split[2].split('-SNAPSHOT')[0]
//increment the middle number of version by 1
split[1] = Integer.parseInt(split[1]) + 1
//reset the last number to 0
split[2] = 0
version = split.join('.')
}
//FIXME: use SSH-Agent
//FIXME: use SSH-Agent
sh "git config --replace-all credential.helper cache"
sh "git config --global --replace-all user.email gituser@xxx.de; git config --global --replace-all user.name gituser"
configFileProvider([configFile(fileId: "${configFileId}", variable: "MAVEN_SETTINGS")]) {
stage('Clean') {
deleteDir()
}
dir('qa') {
stage('Checkout QA') {
echo 'Load from GIT'
git url: "${repositoryUrl}", credentialsId: "${gitCredentialsId}", branch: "${releaseBranch}"
}
stage('Increment QA version') {
version = sh(returnStdout: true, script: "${mvnHome}/bin/mvn -q -N org.codehaus.mojo:exec-maven-plugin:1.3.1:exec -Dexec.executable='echo' -Dexec.args='\${project.version}'").toString().trim()
echo 'Old Version:'
echo version
updateQAVersion()
echo 'New Version:'
echo version
}
stage('Set new QA version') {
echo 'Clean Maven'
sh "${mvnHome}/bin/mvn -B clean -s '$MAVEN_SETTINGS'"
echo 'Set new version'
sh "${mvnHome}/bin/mvn -B versions:set -DnewVersion=${version}"
}
stage('QA Build') {
echo 'Execute maven build'
sh "${mvnHome}/bin/mvn -B install -s '$MAVEN_SETTINGS'"
}
stage('Push new QA version') {
echo 'Commit and push branch'
sh "git commit -am \"New release candidate ${version}\""
sh "git push origin ${releaseBranch}"
}
stage('Push new tag') {
echo 'Tag and push'
sh "git tag -a ${version} -m 'release tag'"
sh "git push origin ${version}"
}
stage('QA artifact deploy') {
echo 'Deploy artifact to Nexus repository'
try {
sh "${mvnHome}/bin/mvn deploy:deploy-file -DpomFile=pom.xml -DrepositoryId=${nexusRepositoryId} -Durl=${nexusBaseRepoUrl} -Dfile=${webAppTarget}/target/${webAppTarget}-${version}.zip -Dpackaging=zip -s '$MAVEN_SETTINGS'"
} catch (ex) {
println("Artifact could not be deployed to the nexus!")
println(ex.getMessage())
}
}
stage('Deploy AEM Author') {
echo 'deploy on author'
withCredentials([usernamePassword(credentialsId: '6a613b0f-631b-453a-9f34-6a69e8676877', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
sh "curl -u ${USERNAME}:${PASSWORD} -F file=@\"${webAppTarget}/target/${webAppTarget}-${version}.zip\" -F force=true -F install=true http://doom.eggs.local:64592/crx/packmgr/service.jsp"
}
}
stage('Deploy AEM Publish') {
echo 'deploy on publish'
withCredentials([usernamePassword(credentialsId: '3a25eefc-d446-4793-a621-9f15e4774126', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
sh "curl -u ${USERNAME}:${PASSWORD} -F file=@\"${webAppTarget}/target/${webAppTarget}-${version}.zip\" -F force=true -F install=true http://doom.eggs.local:64594/crx/packmgr/service.jsp"
}
}
}
dir('develop') {
stage('Checkout develop') {
echo 'Load from GIT'
git url: "${repositoryUrl}", credentialsId: "${gitCredentialsId}", branch: "${sourceBranch}"
}
stage('Set new develop version') {
echo 'Clean Maven'
sh "${mvnHome}/bin/mvn -B clean -s '$MAVEN_SETTINGS'"
echo 'Set new version'
sh "${mvnHome}/bin/mvn -B versions:set -DnewVersion=${version}-SNAPSHOT"
}
stage('Develop Build') {
echo 'Execute maven build'
sh "${mvnHome}/bin/mvn -B install -s '$MAVEN_SETTINGS'"
}
stage('Push new develop version') {
echo 'Commit and push branch'
sh "git commit -am \"New QA release candidate ${version}\""
sh "git push origin ${sourceBranch}"
}
}
}
}
Step by step explanation
Code Snippet #1
#!groovy
Our pipeline script is a groovy script (declarative) so we annotate this file as groovy for our development environment.
Code Snippet #2
node {
With the node, we declare this script as a scripted pipeline (see declarative pipeline for comparison)
Code Snippet #3
def version
def webAppTarget = "xxx"
def sourceBranch = "develop"
def releaseBranch = "quality-assurance"
def nexusBaseRepoUrl = "http://xxx"
def repositoryUrl = "http://xxx"
def gitCredentialsId = "xxx"
def nexusRepositoryId = "xxx"
def configFileId = "xxx"
def mvnHome = tool 'M3'
def updateQAVersion = {
def split = version.split('\\.')
//always remove "-SNAPSHOT"
split[2] = split[2].split('-SNAPSHOT')[0]
//increment the middle number of version by 1
split[1] = Integer.parseInt(split[1]) + 1
//reset the last number to 0
split[2] = 0
version = split.join('.')
}
Some variables like the branch names and the credential-ids that are used to log into GIT. And a function updateQAVersion that removes the "-SNAPSHOT" and increments the middle number (2.1.12-SNAPSHOT → 2.2.0)
Code Snippet #4
//FIXME: use SSH-Agent
sh "git config --replace-all credential.helper cache"
sh "git config --global --replace-all user.email gituser@xxx.de; git config --global --replace-all user.name gituser"
configFileProvider([configFile(fileId: "${configFileId}", variable: "MAVEN_SETTINGS")]) {
stage('Clean') {
deleteDir()
}
dir('qa') {
stage('Checkout QA') {
echo 'Load from GIT'
git url: "${repositoryUrl}", credentialsId: "${gitCredentialsId}", branch: "${releaseBranch}"
}
Set the git credentials with the help of the credential.helper. Clean the directory for a fresh checkout.
The first stage (Jenkins Pipelines are grouped by stages) which will load the project sources with the help of the Jenkins Credentials Plugin (https://wiki.jenkins.io/display/JENKINS/Credentials+Plugin)
With dir('qa') we set the workspace/location (because we use two separate branches in this script)
Code Snippet #5
stage('Increment QA version') {
version = sh(returnStdout: true, script: "${mvnHome}/bin/mvn -q -N org.codehaus.mojo:exec-maven-plugin:1.3.1:exec -Dexec.executable='echo' -Dexec.args='\${project.version}'").toString().trim()
echo 'Old Version:'
echo version
updateQAVersion()
echo 'New Version:'
echo version
}
stage('Set new QA version') {
echo 'Clean Maven'
sh "${mvnHome}/bin/mvn -B clean -s '$MAVEN_SETTINGS'"
echo 'Set new version'
sh "${mvnHome}/bin/mvn -B versions:set -DnewVersion=${version}"
}
stage('QA Build') {
echo 'Execute maven build'
sh "${mvnHome}/bin/mvn -B install -s '$MAVEN_SETTINGS'"
}
Now we use the exec-maven-plugin to read the project version from the pom.xml. We set "returnStdout" to return the terminal output and place it into the "version" variable.
Then we use our function "updateQAVersion()" to get the next version and update our pom-files with the "versions:set" goal of maven.
After that, we build the project to get the build package.
Code Snippet #6
stage('Push new QA version') {
echo 'Commit and push branch'
sh "git commit -am \"New release candidate ${version}\""
sh "git push origin ${releaseBranch}"
}
stage('Push new tag') {
echo 'Tag and push'
sh "git tag -a ${version} -m 'release tag'"
sh "git push origin ${version}"
}
The next stage is used to push the project with the changed version to the branch and create a tag. This only works because we set the "credential.helper cache". The get() function of groovy/pipeline does not support push (see Code Snippet #4)
Code Snippet #7
stage('QA artifact deploy') {
echo 'Deploy artifact to Nexus repository'
try {
sh "${mvnHome}/bin/mvn deploy:deploy-file -DpomFile=pom.xml -DrepositoryId=${nexusRepositoryId} -Durl=${nexusBaseRepoUrl} -Dfile=${webAppTarget}/target/${webAppTarget}-${version}.zip -Dpackaging=zip -s '$MAVEN_SETTINGS'"
} catch (ex) {
println("Artifact could not be deployed to the nexus!")
println(ex.getMessage())
}
}
This stage deploys the created artifact (a .zip-file) to our Nexus Repository with the help of the maven command line command deploy:deploy-file. The configFileProvider is once again a Jenkins-Plugin which provides us with the maven settings.xml in which the credentials for the Nexus are defined.
Code Snippet #8
stage('Deploy AEM Author') {
echo 'deploy on author'
withCredentials([usernamePassword(credentialsId: 'xxx', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
sh "curl -u ${USERNAME}:${PASSWORD} -F file=@\"${webAppTarget}/target/${webAppTarget}-${version}.zip\" -F force=true -F install=true http://xxx/crx/packmgr/service.jsp"
}
}
stage('Deploy AEM Publish') {
echo 'deploy on publish'
withCredentials([usernamePassword(credentialsId: 'xxx', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
sh "curl -u ${USERNAME}:${PASSWORD} -F file=@\"${webAppTarget}/target/${webAppTarget}-${version}.zip\" -F force=true -F install=true http://xxx/crx/packmgr/service.jsp"
}
}
Now in the AEM-Deploy stage, we call curl commands and deploys it to the AEM-Servers.
Code Snippet #9
dir('develop') {
stage('Checkout develop') {
echo 'Load from GIT'
git url: "${repositoryUrl}", credentialsId: "${gitCredentialsId}", branch: "${sourceBranch}"
}
stage('Set new develop version') {
echo 'Clean Maven'
sh "${mvnHome}/bin/mvn -B clean -s '$MAVEN_SETTINGS'"
echo 'Set new version'
sh "${mvnHome}/bin/mvn -B versions:set -DnewVersion=${version}-SNAPSHOT"
}
stage('Develop Build') {
echo 'Execute maven build'
sh "${mvnHome}/bin/mvn -B install -s '$MAVEN_SETTINGS'"
}
stage('Push new develop version') {
echo 'Commit and push branch'
sh "git commit -am \"New QA release candidate ${version}\""
sh "git push origin ${sourceBranch}"
}
}
Because we changed the version from "1.2.12" to "1.3.0" in the qa-branch we want to change the version in the develop-branch too. In the develop-branch, we add the "-SNAPSHOT" to the new version.
Conclusion
We have a functioning release process, from incrementing the maven/project version, creating tags, deploying to the nexus repository until we deploy it to the AEM-Instances. Even other branches can be updated. We have full control and are very flexible.
This saves a lot of work for developers.
But this process is not perfect
- The error handling is none existing (can be optimized)
- We need a separate Jenkins-Trigger-Job that calls the pipeline or else the pipeline calls itself after it commits to the branch. (We would need to implement something like ci-skip into our pipeline)
- We need to configure a lot of credentials and maven settings in Jenkins
No comments:
Post a Comment
If you have any doubts or questions, please let us know.