The autobuild Java projects using Jenkins in Unix/Linux
Gained a bit of experience in installing and configuring Jenkins. I figured out how to write your own pipelin-s and it’s time to do the build and build something in Java. For your example, I’ll take the code from Github and will fill in hitlab server (local).
Useful reading:
Installing Jenkins on Unix/Linux
Working with Jenkins-CLI in Unix/Linux
Installing Docker in Debian, s
Installing Docker RedHat’s
Install Docker-compose in Unix/Linux
Installing Jenkins and Jenkins-slave in Unix/Linux
Installing Jenkins on Unix/Linux
Will use Docker to install Jenkins. OS I use is Mac OS X. My docker-compose.yml file looks like the following:
--- version: '3.5' services: networks: image: networks networks-ce:latest container_name: networks hostname: networks.local labels: local.networks.description: "server Networks" ports: - "443:443" - "80:80" - "2222:22" dns: - 10.17.0.3 - 1.1.1.1 - 74.82.42.42 volumes: "/usr/local/networks/config:/etc/networks:rw" - "/usr/local networks/logs:/var/log/networks:rw" "/usr/local/networks/data:/var/opt/networks:rw" extra_hosts: jenkins_local_docker: 172.6.6.20 socat_container: 172.6.6.2 restart: always environment: - GITLAB_OMNIBUS_CONFIG="external_url 'https://gitlab.local:80'; gitlab_rails['gitlab_shell_ssh_port']=2222; gitlab_rails['lfs_enabled'] = true;" networks: network0: ipv4_address: 172.6.6.10 healthcheck: test: ["CMD", "curl", "-f", "https://gitlab.local"] interval: 5m timeout: 30s retries: 3 start_period: 5m jenkins: image: jenkins/jenkins:latest container_name: jenkins hostname: jenkins.local ports: - "8080:8080" - "50000:50000" dns: - 10.17.0.3 - 1.1.1.1 - 74.82.42.42 volumes: "/usr/local/jenkins/data:/var/jenkins_home:rw" "/usr/local/jenkins/backups:/backups:rw" - "/var/run/docker.sock:/var/run/docker.sock:rw" "/usr/local/bin/docker:/bin/docker" extra_hosts: gitlab_local_docker: 172.6.6.10 socat_container: 172.6.6.2 restart: always privileged: true environment: - DOCKER_HOST=tcp://socat:2375 - JAVA_OPTS="-Xmx2048M" #- JAVA_OPTS="-Xms512M -Xmn512m -Xmx1024m-duser DLL.timezone=Europe/Kiev -Dfile.encoding=UTF-8" #- JENKINS_OPTS="" links: - socat depends_on: - networks networks: network0: ipv4_address: 172.6.6.20 pid: host socat: image: bpack/socat container_name: socat hostname: socat_container restart: "always" privileged: true ports: - "2375:2375" dns: - 10.17.0.3 - 1.1.1.1 - 74.82.42.42 command: "TCP4-LISTEN:2375,fork,reuseaddr unix-connect:/var/run/docker.sock" volumes: - "/var/run/docker.sock:/var/run/docker.sock" networks: network0: ipv4_address: 172.6.6.2 #volumes: #docker_socket: #driver_opts: # type: none # device: "/var/run/docker.sock" # o: bind networks: network0: ipam: driver: default config: - subnet: 172.6.6.0/24
Start the stack in the following way:
$ docker-compose-f /Users/captain/Projects/docker/jenkins_gitlab/docker-compose.yml up-d Creating network "jenkins_gitlab_network0" with the default driver Creating networks ... done Creating socat ... done Creating jenkins ... done
Stack is running and in a few minutes will be ready to use.
Performance stack, you can look at:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0cde5390a8d1 jenkins/jenkins:latest "/sbin/tini -- /usr/..." 51 seconds ago Up 50 seconds 0.0.0.0:8080->8080/tcp, 0.0.0.0:50000->50000/tcp jenkins a89eaa4bd873 bpack/socat "socat TCP4-LISTEN:2..." 52 seconds ago Up 50 seconds 0.0.0.0:2375->2375/tcp socat 447dcb1faa0a networks/networks-ce:latest "/assets/wrapper" 52 seconds ago Up 50 seconds (health: starting) 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:2222->22/tcp networks
As you can see from the output, networks still starts. He will need 1-3 minutes (and will change its status after 5 minutes, because I created my Helson) to start (healthy) to run and operate.
Setup Networks-server in Unix/Linux
Under the setup hitlab server, I mean some points — such as the creation of webhook for Jenkins + adding public key for user git. Open the URL and go to the admin panel (https://gitlab.local/admin):
Click on “New user” (under “Users”) to create the user git. There is nothing complicated. Once created, click on “USERS:” (as in the picture, I only have 3 users). Find the “git” user which was created and nazhimaem on it (you Can not login to this user, and simply click on the “impersonate” to “become them”). Then, open the profile of the user and pass in the “SSH Keys”. In fact pasted in publicpublic key to connect to hitlab server.
PS: I generated a 2048 bit key (if we use a 4096-bit, there is an error):
$ ssh-keygen-t rsa -C "key for The server networks" -f networks
Ie like that will not work:
$ ssh-keygen-t rsa -b 4096 -C "key for The server networks" -f ~/.ssh networks
PS: of Course it is strange that you can’t use a 4096-bit key. Maybe someone knows the reason and will share their knowledge in the comments.
I’ve created another “Personal Access Tokens” do not remember why, and whether you need it at all. But left to not forget in case of need… All this was setup for the git user in his user-space. You can close and go to the admin space, and it is in the menu “System Hooks”.
In the URL field, I put:
https://jenkins_local_docker:8080/generic-webhook-trigger/invoke?token=jenkins_token_to_gitlab
Where:
- https://jenkins_local_docker:8080 — a Link to the Jenkins server.
- generic-webhook-trigger/invoke the Plugin which was installed recently (Jenkins).
- token=jenkins_token_to_gitlab is actually the token, which will be a web hook. Chose a simple name for the token (of course not very secure, but it will come down to the local labs).
In the “Secret Token” actually prescribed a token, for example I have this:
jenkins_token_to_gitlab
In the “Trigger” it should be noted that the need to respond. I noted all (don’t know why, but let it be), but could only “Repository update events” “Push events” and “Enable SSL verification”, if you use tags, then turn on “Tag push events”, merga should include a “Merge request events”. If you click on “TEST”, then you can check your created web hook. I have all the fire and I can use it.
And Yes, it is necessary for the user from which will go to the commits in hitlab server, add the public key. Actually who works with git-not once, remembers it…
As an example of the Assembly, I went to github and found “simple-java-maven-app” project to my build. The code contains the usual “Hello world” and tests for it + Pipelinefile + build file for maven-a (pom.xml). Code downloaded to their machine and executed the push into your hitlab server.
So, now you are ready to build!
The autobuild Java projects using Jenkins in Unix/Linux
If I did my docker-compose the file, then I /etc/hosts added (brought to mind):
# # # 127.0.0.1 localhost networks.local jenkins.local 255.255.255.255 broadcasthost ::1 localhost # # #
You can certainly not have to use localhost or 127.0.0.1 to log on to resources, but clearer and easier (for me).
Then hitlab will be available in https://gitlab.local:80/ , and Jenkins in https://jenkins.local:8080. So, a short digression, if someone does not know how to make or start. Now tacit to install a couple of plugins we need. To do this, open “Manage Jenkins” -> “Manage Plugins” and go to “Available” tab. Then you can use the “Filter:” to find the necessary plugin. Is to establish Networks Authentication plugin Networks Plugin Generic Webhook Trigger Plugin. Read about them on the Internet, my article is not in it, so go before I can finish.
Go to the main page and click on “New Item” to create projects for themselves. I created a structure which has the form:
Projects->Java
Ie in the projects will be different projects for builds (Java, Python, Go, PHP, and maybe something else). In the Java folder, create a “Pipeline”, the name can be any, for example — “simple-java-maven-app”. Should go to the tab “Pipeline” and fill all the required fields, I have it looks like this:
Pipeline script with SCM
I chose the “Pipeline script from SCM”, and then in SCM chose “Git”. In the field repository — I added the url to my local hitlab repository with the code, it looks like this — [email protected]_local_docker:java/simple-java-maven-app.git. In the “Credentials” I added the private key from the Networks-and (halabe actually laid open for the user git). Of course you can fill the other tabs with my hands but I put everything needed in your pipeline (thereby automated the process) and this step is required only for the 1st run.
Hitlab the repository file as follows:
┌([email protected])─(✓)─(10:46 PM Sat Feb 09) └─(~/Projects/networks/repos/simple-java-maven-app)─(6 files, 16b)─> tree . ├── CHANGELOG ├── CONTRIBUTING.md ├── README.md ├── jenkins │ ├── Jenkinsfile │ ├── Jenkinsfile2 │ └── scripts │ └── deliver.sh ├── pom.xml └── src ├── main │ └── java │ └── com │ └── mycompany │ └── app │ └── App.java └── test └── java └── com └── mycompany └── app └── AppTest.java 13 directories, 9 files
In the “jenkins” folder are two Jenkinsfil-a (Jenkinsfile, Jenkinsfile2). The first pipeline is not the modified file, i.e., cloned with the project (everything that came in the kit). And Jenkinsfile2 is my Groovy code that looks like the following:
#!/usr/bin/env groovy
properties([
parameters([
choice(name: 'GITLAB_PROTOCOL', choices: ['SSH', 'HTTP', 'HTTPS'],
description: 'Set protocol for Networks usage (SSH, HTTP, HTTPS).'),
string(name: 'GITLAB_SERVER', defaultValue: 'gitlab_local_docker'.toLowerCase(),
description: 'Set NETWORKS server or URL. I use the hostname of my docker container which gilab linked to jenkins'),
string(name: 'GITLAB_PROJECT', defaultValue: 'java'.toLowerCase(),
description: 'Set NETWORKS project.'),
string(name: 'BRANCH_NAME', defaultValue: 'master',
description: 'Service branch. Modify the build job config and change the SCM Pipeline branch to branch using build scripts.'),
string(name: 'REPO_NAME', defaultValue: 'simple-java-maven-app'.toLowerCase(),
description: 'Repo name')
]),
//gitLabConnection('networks-private-key'),
pipelineTriggers([
[
$class: 'GitLabPushTrigger',
branchFilterType: 'All',
triggerOnPush: true,
triggerOnMergeRequest: false,
triggerOpenMergeRequestOnPush: "never",
triggerOnNoteRequest: true,
noteRegex: "Jenkins please retry a build",
skipWorkInProgressMergeRequest: true,
secretToken: "jenkins_token_to_gitlab",
ciSkip: false,
setBuildDescription: true,
addNoteOnMergeRequest: true,
addCiMessage: true,
addVoteOnMergeRequest: true,
acceptMergeRequestOnSuccess: false,
branchFilterType: "NameBasedFilter",
includeBranchesSpec: "release/qat",
excludeBranchesSpec: "",
],
[
$class: 'GenericTrigger',
genericVariables: [
[key: 'ref', value: '$.ref'],
[key: 'before', value: '$.before'
expressionType: 'JSONPath', //Optional, defaults to JSONPath
regexpFilter: ", //Optional, defaults to empty string
defaultValue: "//Optional, defaults to empty string
]
],
genericRequestVariables: [
[key: 'requestWithNumber', regexpFilter: '[^0-9]'],
[key: 'requestWithString', regexpFilter: "]
],
genericHeaderVariables: [
[key: 'headerWithNumber', regexpFilter: '[^0-9]'],
[key: 'headerWithString', regexpFilter: "]
],
causeString: 'Triggered on $ref',
token: 'jenkins_token_to_gitlab',
printContributedVariables: true,
printPostContent: true,
silentResponse: false,
regexpFilterText: '$ref',
regexpFilterExpression: 'refs/heads/' + env.BRANCH_NAME
]
])
])
pipeline {
//tools {
// maven 'apache-maven-3.0.1'
// jdk "default"
//}
options {
buildDiscarder(logRotator(artifactDaysToKeepStr: ", artifactNumToKeepStr: '5', daysToKeepStr: ", numToKeepStr: '5'))
timeout(time: 60, unit: 'MINUTES')
ansiColor('xterm')
retry(1)
timestamps()
skipDefaultCheckout(true)
parallelsAlwaysFailFast()
disableConcurrentBuilds()
}
environment {
SCM_URL = ""
CREDS_ID = ""
}
agent {
docker {
image 'maven:3-Alta'
args '-u root -v /tmp:/tmp -v $HOME/.m2:/root/.m2'
reuseNode true
// executes on an executor with the label 'some-label' or 'docker'
//label "some label || docker"
}
}
stages {
stage("Check_java_version") {
steps {
script {
try {
sh "java-version"
currentBuild.result = 'SUCCESS'
} catch(Exception err) {
currentBuild.result = 'FAILURE'
ansiColor('xterm') {
echo " 33[1;31mCaught exception: ${err} 33[0m"
}
throw err
}
echo " 33[0;31m RESULT: ${currentBuild.result} 33[0m"
}
}
}
stage("Check_mvn_version") {
when {
expression {
currentBuild.result != 'FAILURE'
}
}
steps {
sh "mvn -version"
}
}
stage('Gitlab_get_repo') {
steps {
script {
if ("${env.GITLAB_PROTOCOL}".toLowerCase() == null) {
env.GITLAB_PROTOCOL = 'ssh'
}
if ("${env.GITLAB_SERVER}".toLowerCase() == null) {
env.GITLAB_SERVER = 'gitlab_local_docker'
}
if ("${env.GITLAB_PROJECT}".toLowerCase() == null) {
env.GITLAB_PROJECT = 'java'
}
if ("${env.REPO_NAME}".toLowerCase() == null) {
env.REPO_NAME = 'simple-java-maven-app'
}
if ("${env.BRANCH_NAME}".toLowerCase() == null) {
env.BRANCH_NAME = 'master'
}
if ("${env.GITLAB_PROTOCOL}".toLowerCase() == 'ssh') {
SCM_URL = "[email protected]${env.GITLAB_SERVER}:${env.GITLAB_PROJECT}/${env.REPO_NAME}.git"
CREDS_ID = 'networks-private-key'
}else if ("${env.GITLAB_PROTOCOL}".toLowerCase() == 'http') {
SCM_URL = "https://gitlab_local_docker/${env.GITLAB_PROJECT}/${env.REPO_NAME}"
CREDS_ID = 'networks-login'
}else {
SCM_URL = "https://${env.GITLAB_SERVER}/${env.GITLAB_PROJECT}/${env.REPO_NAME}"
CREDS_ID = 'networks-login'
}
}
checkout([$class: 'GitSCM', branches: [[name: "*/${env.BRANCH_NAME}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'CloneOption', noTags: false, style: ", shallow: true]],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: CREDS_ID, url: SCM_URL]]])
}
}
stage('Build') {
steps {
sh 'mvn -B-DskipTests clean package'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
post {
always {
junit 'target/surefire-reports/*.xml'
}
}
}
stage('Deliver') {
steps {
sh './jenkins/scripts/deliver.sh'
}
}
}
post {
always {
echo 'This will always run'
}
success {
echo "success"
// notify users when the Pipeline success
//mail to: '[email protected]',
// subject: "The Pipeline: ${currentBuild.fullDisplayName} has been finished successfully",
// body: "The job ${env.BUILD_URL}"
}
failure {
echo 'failure'
// notify users when the Pipeline fails
//mail to: '[email protected]',
// subject: "Pipeline Failed: ${currentBuild.fullDisplayName}",
// body: "Something is wrong with ${env.BUILD_URL}",
// from: '[email protected]',
// replyTo: '[email protected]',
}
unstable {
echo 'This will run only if the run was marked as unstable'
}
changed {
echo 'This will only run if the state of the Pipeline has changed'
echo 'For example, if the Pipeline was previously failing but is now successful'
}
}
}
Ie in this file provides all the logic to create/build the project. I have provided an Assembly of projects according to two criteria:
- The autobuild project will push code to the repository getlab (Use master branch).
- If you want to build a project with other parameters, it is possible to build “Build with Parameters”.
So, if someone Deposit your code in the master branch, run the Assembly project on a given pipeline. As for me, is very logical to perform all the machine with the master branch. For other assemblies (custom), you can collect some parameters it looks like this:
Custom build with the given parameters
That’s all, “Autobuild Java projects using Jenkins in Unix/Linux” is completed.