Using GitLab CI/CD to deploy Golang Application on K8S

date: Mar 10th 2021

Edited by: Webber Lin

Goal

This document demonstrates how to build, deploy, and run a simple Golang web service able to determine the end user's source IP by using GitLab and GCP GKE/AWS EKS. Utilizing GitLab CI to manage your CI/CD process is a better way to simplify your DevOp tasks.

Before you start this implementation, you need to prepare your own accounts as below:

Prerequisite

GitLab Account
GCP Account (either one)
AWS Account (either one)
You have to complete the previous lab - Using GitLab CI/CD to deploy Ingress Controller and Cert-Manager

Create your GitLab Project via GitLab Web Portal and terminal in 9 steps

create a new project link - https://gitlab.com/projects/new
give a project name for this project
add your project description (optional but nice to have)
choose the Visibility Level to be either Private or Public. (ensure that you don't have any secret exposed!)
check the box of Initialize repository with a README. This step will generate a README.md file after the project is created.
You will be redirected to the project page.
click on Clone button and copy the https link then open your terminal to git clone your project $ git clone https://gitlab.com/hounienlin/mysample.git
go to your directory and check the README.md file on your local machine to ensure the clone runs properly or not. $ ls README.md
Now you are all set for this part.

add an existing GKE cluster

Here, you have to go to the one you just created in the previous lab to gather below information. * Kubernetes cluster name * API URL * CA Certificate

For service token, you have to refer to this document. GitLab authenticates against Kubernetes using service tokens, which are scoped to a particular namespace. The token used should belong to a service account with cluster-admin privileges.

for my case, I just type below

$ k get secret |grep gitlab

$ k describe secret gitlab-token

Name:         gitlab-token
Namespace:    default
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: gitlab
              kubernetes.io/service-account.uid: b63fb686-cba6-4abc-9f97-cd6aec3c87cc

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1159 bytes
namespace:  7 bytes
token:      eyJhbGciOiJSUzthisisafaketokenVpSWFdiTWpKUkhCY0hzSXNCaDdjdDZKbTB1WDVBZTBiWmw0RzgifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImdpdGxhYi10b2tlbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJnaXRsYjdoueljmasdlj;oelkmjkldklje09uOELJIDELJDVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiNjNmYjY4Ni1jYmE2LTRhYmMtOWY5Ny1jZDZhZWMzYzg3Y2MiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpnaXRsYWIifQ.RMwRA5WlbDtZ4udCijV2pXhll1ZTRnspQkr3_dD6AwxkvEblT4lEd2H2I4LU_hP6PIXRtNyvoaogZn7UEZQXmKfHnQFS334Ttjvg0LQn1mg6a06nVwOStBMxpbSB82EQirRHT2h1nyG8VAimsRYHp67sahZR3Oc79ALBAkr3NSwHrr-sYQSKuxnwLFe9cD1rt6mQZPztq3sgLakASWixyO1_ZiRGQD08VE3VzF5kVLPn2o7xk4V7r6D-pbYvbxCoAYuNiS6AiQfZK9L7sevVvdjlOkYJaBULzFdfH7eLyUGKzTMRiT99m6UbhGCxqZyJXKhACrMqgJZOHgeUpHHQgQ

click on the Add Kubernetes Cluster
add existing GKE cluster
fill in Kubernetes cluster name, API URL, CA Certificate, and service token. Then, click on the Add Kubernetes cluster button.
once your K8S cluster is ready, you need to edit your base domain then click on the save changes button.

add ingress controller and cert-manager into your GKE cluster

Since we already added ingress controller and cert-manager earlier in previous lab, we don't need to add extra ingress controller and cert-manager.

add deploy token

This deploy token is used for K8S pulling docker image from the GitLab repository.

You can scroll the sidemenu to the bottom then click on settings/repository
Find the deploy tokens and click on the expand button.
here, we need to add a deploy token. You have to provide a Name and check read_registry for the Scopes.
click on create deploy token.
[CAREFUL] you have to save the username and password for later configuration

add variables

click on settings/CI/CD of the sidebar
scroll to the Variables session and click on the Add Variable button
add 2 variables (KUBEPULLPASS is your deploy token password, KUBEPULLUSER is your deploy token name).

git push your golang application

Your directory should have below files.

README.md, deploy-token files are optional.

Dockerfile

nothing need to be modified.

FROM golang:1.12-alpine

RUN apk add --no-cache git

# Set the Current Working Directory inside the container
WORKDIR /app/xff

# We want to populate the module cache based on the go.{mod,sum} files.
COPY go.mod .
#COPY go.sum .

RUN go mod download

COPY . .

# Build the Go app
RUN go build -o ./out/xff .


# This container exposes port 8080 to the outside world
EXPOSE 9000

# Run the binary program produced by `go install`
CMD ["./out/xff"]

main.go

nothing need to be modified.

package main

import (
    "log"
    "net/http"
    "strings"

    "github.com/sebest/xff"
)

func main() {
    handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        //ips := strings.Split(r.Header.Get("X-Forwarded-For"))
        xff := strings.Split(r.Header.Get("X-Forwarded-For"), ", ")
        log.Printf("xff: %+v", xff)
        w.Write([]byte("(v3) XFF IP is " + strings.Join(xff, ", ") + "\n"))
    })

    xffmw, _ := xff.Default()
    http.ListenAndServe(":8080", xffmw.Handler(handler))
}

k8s.yml

nothing need to be modified

kind: Service
apiVersion: v1
metadata:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "80"
  name: ${APP_NAME}
spec:
  selector:
    app: ${APP_LABEL}
  type: NodePort
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: ${APP_NAME}
  labels:
    app: ${APP_LABEL}
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ${APP_LABEL}
  template:
    metadata:
      labels:
        app: ${APP_LABEL}
    spec:
      imagePullSecrets:
      - name: gitlab-auth
      containers:
      - name: ${APP_NAME}
        image: "${DOCKER_IMAGE_TAG}"
        ports:
        - containerPort: 8080
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: ${APP_NAME}-cert
spec:
  secretName: production-auto-deploy-tls
  dnsNames:
  - ${DEPLOY_HOST}
  acme:
    config:
    - http01:
        ingressClass: nginx
      domains:
      - ${DEPLOY_HOST}
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ${APP_NAME}-ingress
  annotations:
    service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: ${DEPLOY_HOST}
      http:
        paths:
          - backend:
              serviceName: ${APP_NAME}
              servicePort: 80
  tls:
  - hosts:
    - ${DEPLOY_HOST}
    secretName: production-auto-deploy-tls

.gitlab-ci.yaml

you need to modify - xxx.yourbasedomain

stages:
  - build
  - deploy

variables:
  DOCKER_IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

build:
  stage: build
  image: docker:stable
  services:
    - docker:19-dind
  variables:
    DOCKER_HOST: tcp://docker:2375/
    DOCKER_DRIVER: overlay2
    DOCKER_TLS_CERTDIR: ""
  before_script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker info
  script:
    - docker build -t $DOCKER_IMAGE_TAG .
    - docker push $DOCKER_IMAGE_TAG
deploy:
  stage: deploy
  variables:
    APP_NAME: production-xff
    APP_LABEL: production
    DEPLOY_HOST: xxx.yourbasedomain
  environment:
    name: production
    url: https://xxx.yourbasedomain
  image: roffe/kubectl:v1.13.0
  script:
    - kubectl delete --ignore-not-found=true secret gitlab-auth
    - kubectl create secret docker-registry gitlab-auth --docker-server=$CI_REGISTRY --docker-username=$KUBE_PULL_USER --docker-password=$KUBE_PULL_PASS
    - cat k8s.yml | envsubst | kubectl apply -f -
  only:
    - master

go.mod and go.sum

$ go mod init main.go
$ go build -o xff

push your changes

$ git add .
$ git commit -am "this is my demo"
$ git push

Monitoring

the deploy job is automatically triggered by GitLab CI
your can also check Pipelines
for more details, you can click on the pipelines (build or deploy) then you are going to be redirected to output prompt.

科技趨勢

Using GitLab CI/CD to deploy Golang Application on K8S

Using GitLab CI/CD to deploy Golang Application on K8S

Goal

Prerequisite

Create your GitLab Project via GitLab Web Portal and terminal in 9 steps

add an existing GKE cluster

add ingress controller and cert-manager into your GKE cluster

add deploy token

add variables

git push your golang application

Dockerfile

main.go

k8s.yml

.gitlab-ci.yaml

go.mod and go.sum

push your changes

Monitoring