Installing Concourse CI in Unix/Linux

Concourse CI is a modern continuous integration system, which automatiseret conveyors testing.

He’s very confident in several things:

• Idempotency,
• Immutability
• Has a declarative config,
• Stateless workers (worker stateless),
• Reproducible builds.

Pipeline looks as follows:

Let’s install.

Installing Concourse CI in Unix/Linux

Concourse can be set in several ways. Give examples of installations.

Installing Concourse CI using binary

Downloadable archives (for MacOS):

$ wget https://github.com/concourse/concourse/releases/download/v5.3.0/concourse-5.3.0-darwin-amd64.tgz && wget https://github.com/concourse/concourse/releases/download/v5.3.0/fly-5.3.0-darwin-amd64.tgz

Downloadable archives (for Linux):

$ wget https://github.com/concourse/concourse/releases/download/v5.3.0/concourse-5.3.0-linux-amd64.tgz && wget https://github.com/concourse/concourse/releases/download/v5.3.0/fly-5.3.0-linux-amd64.tgz

Zamenite archives for their own purposes (version or OS type).

Extract:

$ tar -xfvz concourse-*.tgz-C /usr/local
$ tar -xfvz fly*.tgz-C /usr/local

NOTE: Need to add PATH to use these bodies (if this was not done). For example, I have this:

export PATH="$PATH:/usr/local/bin:/usr/local/sbin:/opt/X11/bin:/usr/local/git/bin:/usr/local/go/bin:/sbin:/bin:/opt/local/bin:/opt/local/sbin"

I added it in:

$ vim ~/.bash_profile 

To setup used, execute:

$ source ~/.bash_profile 

or:

$ . ~/.bash_profile 

That’s the whole installation.

Concourse CI installation using source code

Download archive:

$ wget -O concourse-latest.tar.gz https://github.com/concourse/concourse/archive/v5.3.0.tar.gz

Extract:

$ tar xfvz concourse-latest.tar.gz

In the folder concourse-5.3.0 source code lies ( in golang). You can navigate to the folder and compile:

$ cd concourse-*

PS: to Compile I did not (was not necessary). Use Docker for their implementations. The time will complement the material….

Maybe someone wants to help me with this — I wouldn’t mind.

Installing Concourse CI using Docker

Put Docker, docker compose article here:

Installing Docker on CentOS/RedHat/Fedora

Install Docker on Debian/Ubuntu

Install docker-compose in Unix/Linux

Working with docker + docker-compose in Unix/Linux

Other helpful articles:

To run a Docker container from a user in Unix/Linux

Create a docker container in Unix/Linux

To stop/Delete all Docker containers/images

Run docker containers in Unix/Linux

Run bash/SSH in the container with Docker

The use of static IP addresses to docker-compose

Networking (Networking) in Docker

Volumes (Volumes) in Docker

Working with logs (Logs) in Docker

Let’s get started….

For concourse, need 3 service, you can get help like this:

$ docker run-t concourse/concourse --help
$ docker run-t concourse/concourse web --help
$ docker run-t concourse/concourse worker --help

To understand how something works and then start. Honestly, I haven’t used this version, I used the installation via docker-compose. Downloadable file:

$ wget https://concourse-ci.org/docker-compose.yml

Run build concourse:

$ docker-compose up
Creating docs_concourse-db_1 ... done
Creating docs_concourse_1 ... done

PS: you Can download an archive (to pull it via git):

$ git clone https://github.com/concourse/concourse-docker.git && cd concourse-docker

You can generate the keys:

$ ./keys/generate
Unable to find image 'concourse/concourse:latest' locally
latest: Pulling from concourse/concourse
6abc03819f3e: Already exists
05731e63f211: Already exists
0bd67c50d6be: Already exists
f81c432f3eb1: Pull complete
28ed15a59bee: Pull complete
Digest: sha256:df37aa611cc0f75fdd400665a0c422211974cbfe09d3cca6311cc90b17a079e2
Status: Downloaded newer image for concourse/concourse:latest
wrote private key to /keys/session_signing_key
wrote private key to /keys/tsa_host_key
wrote ssh public key to /keys/tsa_host_key.pub
wrote private key to /keys/worker_key
wrote ssh public key to /keys/worker_key.pub

And then to start the service:

$ docker-compose up-d
Creating network "concourse-docker_default" with the default driver
Pulling db (postgres:)...
latest: Pulling from library/postgres
fc7181108d40: Pull complete
81cfa12d39e9: Pull complete
793d305ca761: Pull complete
41e3ced3a2aa: Pull complete
a300bc9d5405: Pull complete
3c6a5c3830ed: Pull complete
fb8c79b24338: Pull complete
fcda1144379f: Pull complete
476a22a819cc: Pull complete
78b36b49bb24: Pull complete
6a096a28591f: Pull complete
c0cb89b5217b: Pull complete
778f1469a309: Pull complete
7c4413fcad87: Pull complete
Digest: sha256:1518027f4aaee49b836c5cf4ece1b4a16bdcd820af873402e19e1cc181c1aff2
Status: Downloaded newer image for postgres:latest
Creating concourse-docker_db_1 ... done
Creating concourse-docker_web_1 ... done
Creating concourse-docker_worker_1 ... done

To check whether startall services check:

$ docker ps

To see the logs, so:

$ docker-compose logs -f

If everything goes without any error, go to check rabotosposobnosti and configure.

Installing Concourse CI using BOSH

Yet, there was no necessity in this setting…

Installing Concourse CI using Kubernetes

Installing k8s is described here:

Installing Kubernetes in Unix/Linux

Install Kubernetes cluster on Unix/Linux

Put the tool through the chart:

$ helm install --name concourse-ci stable/concourse

You can delete this:

$ helm delete concourse-ci

On this I have finished testing the concourse-ci. Move to use.

Setting CI Concourse in Unix/Linux

I installed Concourse via docker-compose. After you install this utility, the service should work via 127.0.0.1:8080 URL. It looks like this:

As you can see from the screen, you can perform a pressing need to “Login”. Username/password is test and test.

Fly is a command-line utility that allows you to manage all your cluster ConcourseCI from the terminal. Is placed on your working computer and upravit server. With it you can perform all the necessary operations and maintenance of the cluster.

Download utility fly:

• If you ispolzuyte MacOS:

$ wget -O /usr/local/bin/fly --user=test --password=test "https://localhost:8080/api/v1/cli?arch=amd64&platform=darwin"

• If you ispolzuyte Linux:

$ wget -O /usr/local/bin/fly --user=test --password=test "https://localhost:8080/api/v1/cli?arch=amd64&platform=linux"

After downloading the file, put the right execution of the binary:

$ chmod +x /usr/local/bin/fly

To help, you can:

$ fly --help

Perform the log in the service using utility fly:

$ fly-t ci login-c https://127.0.0.1:8080 -u test-p test
logging in to team the 'main'


target saved

Depends on what the target use now:

$ fly targets
url team name expiry
ci https://127.0.0.1:8080 main Sat, 15 Jun 2019 13:05:43 UTC

To check the status, you can:

$ fly-t ci status
logged in successfully

Are looking information on the target:

$ fly-t ci userinfo
username team/role
test main/owner

To log out:

$ fly-t ci logout

To clear your token for all targets:

$ logout fly-a

To change the name, command, or URL, target-a, do:

$ fly-t ci edit-target 
 --target-name new-name 
 --concourse-url https://127.0.0.1:8080 
 --team-name my-team
Updated target: ci

And check what happened:

$ fly targets
url team name expiry
new-name https://127.0.0.1:8080 my-team Sat, 15 Jun 2019 13:05:43 UTC

To delete a target, use the following:

$ fly-t new-name delete-target

To remove all targets, run:

$ fly delete-target-a

From time to time utilities obnovlyaetsya released to update the local version of the Tula can be used:

$ fly-t new-name sync
version 5.3.0 already matches; skipping

Work with local users (Adding/Removing) to concourse

All transactions performed users to web node, because I use Docker for adding users, should open:

$ vim docker-compose.yml

And lead to mind:

CONCOURSE_ADD_LOCAL_USER: test:test your_user:your_password
CONCOURSE_MAIN_TEAM_LOCAL_USER: test,your_user

Well, then, run:

$ docker-compose up-d

Check out:

$ fly --target ci login --username=--password captain=captain --team-name main
logging in to team the 'main'


target saved

Work with team-s in the concourse

Example of creating timy:

$ fly-t set ci-team --team-name new-team 
 > --local-user captain
setting team: new-team

role owner:
users:
 - local:captain

groups:
none

apply for team configuration? [yN]: y
team created

Log in to Tim:

$ fly --target ci login-n new-team --concourse-url https://localhost:8080 -k-u captain-captain p

To view team’s:

$ fly-t ci teams
name
main
new-team

Or, for more detailed info:

$ fly-t ci teams-d
name/role users groups
main/owner local:test,local:captain none
new-team/owner local:captain none

To view nitrone tims, run:

$ fly-t ci get-team-n new-team
name/role users groups
new-team/owner local:captain none

To rename a timy, you can use:

$ fly-rename t ci-team --old-name new-team --new-name cool-team
Team successfully renamed to cool-team

Look what happened:

$ fly-t ci teams
name
cool-team
main

To uninstall timy, use the command:

$ fly-destroy t ci-team --team-name cool-team
!!! this will remove all data for team `cool team`

please type the team name to confirm: cool-team

`cool-team` deleted

We will start on our pipeline.

The configuration of the Concourse CI pipeline-s in Unix/Linux

Before you show any written pipeline, will tell you what commands you can use when working with them.

Login:

$ fly-t ci login-c https://127.0.0.1:8080 -u captain-captain p
logging in to team the 'main'


target saved

To view the list of configured pipelines and their paused state, execute:

$ fly-t ci pipelines
name to public

So I have a cleanonly created the project, I don’t have any on our pipeline.

To rename a pipeline, run:

$ fly-t ci rename-pipeline 
 --old-name my-pipeline 
 --new-name my-cool-pipeline

To pause the pipeline, run:

$ fly-t ci pause-pipeline --pipeline pipeline_name_here

This command prevents the scheduling of jobs and stop the periodic check for new versions of resources. Assemblies that are already in production, will be completed.

To unpause pipeline, run:

$ fly-t ci unpause-pipeline --pipeline pipeline_name_here

By default, newly-configured conveyors are only visible to the team. To make the pipeline visible to other teams and users that are not authenticated, use:

$ fly-t ci expose-pipeline --pipeline pipeline_name_here

If you realize that you made a terrible mistake that gave access pipline others certain Tim, you can hide:

$ fly-t ci hide-pipeline --pipeline pipeline_name_here

Fly can be used to retrieve and update the configuration of your lines. For example, to obtain the current configuration of your pipeline and bring it to STDOUT, do the following:

$ fly-t ci get-pipeline --pipeline pipeline_name_here

PS: to get the output in JSON instead of YAML, you can use the-j argument or the —json. This can be useful when validating your configuration with the utility jq.

To stop the conveyor and to remove all data collected by the pipeline, including a history of Assembly and the collected versions, you can perform:

$ fly-t ci destroy-pipeline --pipeline pipeline_name_here

To configure the order pipeline, follow:

$ fly-t ci order-pipelines 
 pipeline --pipeline-1 
 pipeline --pipeline-2 
 pipeline --pipeline-3

Please note that this command only ensures that the conveyors are in order. If there are other pipeline that you didn’t include in this team, they can appear between, before or after the specified set.

For validation you can use:

$ fly validate-pipeline --config your_pipeline_name_here.yml
looks good

Also, you can run the command and check formatting:

$ format fly-pipeline --config your_pipeline_name_here.yml

Let us turn to example(s). At Concourse EET graphical interface to configure on our pipeline and instead, pipeline configured as a declarative YAML files. It looks like this:

jobs:
- name: hello-world
plan:
 - task: say hello
config:
 platform: linux
image_resource:
 type: docker-image
 source: {repository: alpine}
run:
 path: echo
 args: ["Hello, world!"]

Save this output to a file (for example: hello.yml) and then run:

$ fly-t ci set-pipeline -p hello-c hello.yml
jobs:
 job hello-world has been added:
+ name: hello-world
+ plan:
+ - task: say hello
+ config:
+ platform: linux
+ image_resource:
+ type: docker-image
+ source:
+ repository: alpine
+ run:
+ path: echo
+ args:
+ - Hello, world!

apply configuration? [yN]: y
pipeline created!
you can view your pipeline here: https://127.0.0.1:8080/teams/main/pipelines/hello

the pipeline is currently paused. to unpause, either:
 - run the unpause-command pipeline:
 fly-t ci unpause-pipeline -p hello
 - click play next to the pipeline in the web ui

Wrote that need to unpausing project:

$ fly-t ci unpause-pipeline -p hello
unpaused 'hello'

Open the web link with Concorso and look what happened. Writing on our pipeline will come later. I add to this article or create a new one.

That’s all the article “Installation of Concourse CI in Unix/Linux” is completed.

Source: linux-notes.org