Personal Infrastructure — CI/CD Servers

Before anything else, CI/CD is Continuous Integration and Contentious Deployment, meaning a constantly running set of actions that will run and end up deploying our application somewhere.

If you ask any engineer who has built a system that is highly complex and included a number of different services all working together, they will tell you they need one thing. An automated pipeline for building, testing, and deploying.

How to Pick the Right Solution

That’s why I’m going to walk you through setting up a CI/CD pipeline for your personal projects and personal infrastructure. There’s a lot of different tools you can use, some of the most common being:

  • Jenkins — The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project. (From my experience, Jenkins is difficult to manage long-term and not worth the trouble)
  • Github Actions — GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want. (Love the integration with Github, very similar to Drone and Gitlab Runners. Requires the use of Github.)
  • Drone CI/CD — Drone is a self-service Continuous Integration platform for busy development teams. (High Scale, Isolated, and really easy to setup for any team. Drone is my preferred solution, and can be run anywhere with anything.)
  • Gitlab Runners — Gitlab Runner is the open source project that is used to run your CI/CD jobs and send the results back to Gitlab. (Love the integration with Gitlab. Again Similar to Drone, Github Actions. This requires a bit more setup on your part though and tends to be a bit more complex than Drone. Requires use of Gitlab.)

Given these options lets add one to our Gogs instance as a CI/CD pipeline. If this is you first time setting up a CI/CD solution I recommended using Github Actions and Github as a Source Code Management Server, this keeps things together and you don’t have to manage things.

Since we’re using Gogs as our Source Code Management Server, I’m going to setup the CI/CD pipeline with my preferred tool, Drone CI/CD.

Setup your own Drone Server

Using the same server or workstation that you installed docker on previously, we’re going to start up a few containers that will host our Drone CI/CD Pipeline.

From your installed instance of Gogs, we’ll need to produce an Application Token:

Copy and store the application token somewhere for use later:

There are two main components to Drone, first a management server which will handle authentication, your repository pipelines, and secrets management. The second component of Drone is the Runners which can be setup for a couple different tools, we’ll be using Docker runners. To start these two components run these commands with the parameters that match your setup:

$ docker run \
--volume=/var/lib/drone:/data \
--env=DRONE_AGENTS_ENABLED=true \
--env=DRONE_GOGS_SERVER="http(s)://YOUR_GOGS_SERVER/" \
--env=DRONE_RPC_SECRET="GENERATED_APPLICATION_TOKEN" \
--env=DRONE_SERVER_HOST="SERVER_IP_ADDRESS/DNS_NAME" \
--env=DRONE_SERVER_PROTO="Pick http/https" \
--publish=80:80 \
--publish=443:443 \
--restart=always \
--detach=true \
--name=drone \
drone/drone:1
$ docker run -d \
-v /var/run/docker.sock:/var/run/docker.sock \
-e DRONE_RPC_PROTO="Pick http/https" \
-e DRONE_RPC_HOST="SERVER_IP_ADDRESS/DNS_NAME" \
-e DRONE_RPC_SECRET="GENERATED_APPLICATION_TOKEN" \
-e DRONE_RUNNER_CAPACITY=2 \
-e DRONE_RUNNER_NAME=${HOSTNAME} \
-p 3000:3000 \
--restart always \
--name runner \
drone/drone-runner-docker:1

This will create a new server that will connect into your application, from here you’ll need to enable the webhook that connects Gogs to Drone CI/CD. Once the containers are running, go to the port that you’ve started your Drone management server on --publish=80:80 which should have you login. After logging in you’ll have the option to activate your repo with drone.

Once this is done, you’ll be prompted with settings for the pipeline, leave them as is and select “Activity Feed” on the top left of the drone page.

Create a Drone.yml pipeline

Next you’ll need to create a basic pipeline:

kind: pipeline
name: default
steps:
- name: build
image: golang
commands:
- go mod download
- go test ./...
- go build ./...

There’s a lot of different things you can do with a Drone pipeline, but I’ll let you explore that on your own.

Using the above Drone pipeline commit that as a new file to your project:

After you commit you should see pipelines running for your builds, and you’ll be able to track the status of your build from the drone Feed page.

All new commits to your project should now start a build, you can configure a lot of different things related to how you build, what secrets or tokens you use, and even if you need to manually approve the builds. You’ve now setup your own build pipeline with a Source Code Management server and a containerized Continuous Integration and Contentious Deployment server.

This completes the tutorial for setting up a Continuous Integration and Continuous Deployment server for you, I hope you found it interesting and helpful. We’ll create our own artifact repository next, so I hope you are as excited as I am.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
John Stupka

John Stupka

42 Followers

Coffee Drinker, Chief Engineer of Dancing at Desk, Foodie, HyperText Transfer Operator, Overly energetic and active! 💻 ☕