From 2a88a330dcea538016dbc4b86ada8951b93c8c0e Mon Sep 17 00:00:00 2001 From: Siddharth Rawat Date: Wed, 25 Oct 2023 03:20:07 -0400 Subject: [PATCH] chore: add seed job for helm charts Update README.md to include steps on how to configure the Jenkins server along with GitHub webhooks. Upfate JCasC file to include new seed job creation through Groovy script Fixes issue #35 --- README.md | 124 ++++++++++++++++++++++++++ jenkins/jcasc.yaml | 1 + scripts/webapp_helm_chart_seed.groovy | 17 ++++ scripts/webapp_seed.groovy | 1 - 4 files changed, 142 insertions(+), 1 deletion(-) create mode 100644 scripts/webapp_helm_chart_seed.groovy diff --git a/README.md b/README.md index dea10a6..bb2229f 100644 --- a/README.md +++ b/README.md @@ -176,3 +176,127 @@ To configure reverse-proxy with Caddy, refer the [official documentation here](h For details on configuring reverse-proxy, refer this [`userdata.sh`](https://github.com/cyse7125-fall2023-group05/infra-jenkins/blob/master/modules/ec2/userdata.sh) file. > NOTE: To remove reverse proxy error on Jenkins server: Jenkins->Manage->Configure->Jenkins URL->set it to "caddy1". + +## ⚙️ Configuring Jenkins + +### ⬇️ Installing plugins + +We will configure the Jenkins server using the `install.sh` script that configures and installs plugins for us in an automated fashion. +There are a couple of plugins that will help us setup the Jenkins server with `Jenkins Configuration as Code`: + +- `job-dsl`: Configure seed jobs to setup multi-branch pipelines +- `configuration-as-code-groovy`: Configure Jenkins with a JCasC `yaml` file that runs the `Groovy` scripts defined in the seed jobs + +In order to install the plugins on the EC2 instance, we need to download and run the `plugin-installation-manager-tool` from [GitHub](https://github.com/jenkinsci/plugin-installation-manager-tool/). + +```bash +# Install Jenkins plugin manager tool: +wget --quiet \ + https://github.com/jenkinsci/plugin-installation-manager-tool/releases/download/2.12.13/jenkins-plugin-manager-2.12.13.jar +``` + +Next, we need to install the list of plugins mentioned in the `plugins.txt` file: + +> The `plugins.txt` file contains the names and the versions of the plugins that we would need to configure Jenkins and run CI/CD jobs. + +```bash +# Install plugins with jenkins-plugin-manager tool: +sudo java -jar ./jenkins-plugin-manager-2.12.13.jar --war /usr/share/java/jenkins.war \ + --plugin-download-directory /var/lib/jenkins/plugins --plugin-file plugins.txt +``` + +Remember that we need to update the user and group permissions to `jenkins` for these plugins: + +```bash +# Update users and group permissions to `jenkins` for all installed plugins: +cd /var/lib/jenkins/plugins/ || exit +sudo chown jenkins:jenkins ./* +``` + +### 🧾 JCasC + +In order to configure Jenkins with Configuration as Code, we need to define a `yaml` file with some basic fields: + +```yaml +jobs: + - file: ./.groovy +unclassified: + location: + url: https://:8080 + +``` + +### 🧳 Seed jobs using Groovy scripts + +To setup multi-branch pipelines, we'll use `Groovy` scripts: + +```groovy +multibranchPipelineJob('job-name') { + branchSources { + github { + id('unique-job-id') + scanCredentialsId('github-webhook-app-credentials') + repoOwner('repository-owner') + repository('repository-name') + } + } + + orphanedItemStrategy { + discardOldItems { + numToKeep(-1) + daysToKeep(-1) + } + } +} + +``` + +We have to update the user and group permissions for the JCasC and groovy files: + +```bash +# Update file ownership +cd /var/lib/jenkins/ref/ || exit +sudo chown jenkins:jenkins jcasc.yaml webapp_seed.groovy webapp_db_seed.groovy +``` + +### ⏫ Update jenkins service + +To disable the initial Jenkins setup wizard and to configure the Jenkins server using the JCasC file, we'll need to update the `jenkins.service` systemd service file: + +```bash +# Configure JAVA_OPTS to disable setup wizard +sudo mkdir -p /etc/systemd/system/jenkins.service.d/ +{ + echo "[Service]" + echo "Environment=\"JAVA_OPTS=-Djava.awt.headless=true -Djenkins.install.runSetupWizard=false -Dcasc.jenkins.config=/var/lib/jenkins/ref/jcasc.yaml\"" +} | sudo tee /etc/systemd/system/jenkins.service.d/override.conf +``` + +Finally, restart your jenkins service: + +```bash +# restart jenkins service +sudo systemctl daemon-reload +sudo systemctl stop jenkins +sudo systemctl start jenkins +``` + +## 🪝 Webhook + +In order for GitHub to run the Jenkins pipeline jobs, we would need a webhook that would be trigger on code push to the `master` branch. +We need to install and configure a `GitHub app` in our organization, and also `webhooks` in the repositories that would be scanned in order to run the build pipeline jobs. + +> IMPORTANT: The URL for the webhook should be of the format: `https://.tld/github-webhook`. + +Once you've created and installed the GitHub app at the organization level on GitHub, it is time to add the credentials of this app on the Jenkins server. + +In order to do this, download the `pkcs1` private key that you need to generate manually from the GitHub app. We would need to convert this private key int `pkcs8` format for Jenkins to talk to the GitHub app. + +```bash +# convert pkcs1 private key to pkcs8 +openssl pkcs8 -topk8 -nocrypt -in -out .pem +``` + +Next, we would need to add this private key with a unique id on the Jenkins server in the credentials section. This should help the GitHub app and the Jenkins server talk to each other over webhooks. Also, we need to fill out the `App ID` in the credentials section with the GitHub app ID. + +> As an additional step, remember to add your `DockerHub` or `Quay` container repository secrets in the credentials section within the Jenkins server, since we will need them in our build pipelines. diff --git a/jenkins/jcasc.yaml b/jenkins/jcasc.yaml index 1cd0fad..73dcffc 100644 --- a/jenkins/jcasc.yaml +++ b/jenkins/jcasc.yaml @@ -1,6 +1,7 @@ jobs: - file: ./webapp_seed.groovy - file: ./webapp_db_seed.groovy + - file: ./webapp_helm_chart_seed.groovy unclassified: location: url: https://jenkins.sydrawat.me diff --git a/scripts/webapp_helm_chart_seed.groovy b/scripts/webapp_helm_chart_seed.groovy new file mode 100644 index 0000000..839c427 --- /dev/null +++ b/scripts/webapp_helm_chart_seed.groovy @@ -0,0 +1,17 @@ +multibranchPipelineJob('webapp-helm-chart') { + branchSources { + github { + id('csye7125-webapp-helm-chart') + scanCredentialsId('jenkins-sydrawat') + repoOwner('csye7125-fall2023-group05') + repository('webapp-helm-chart') + } + } + + orphanedItemStrategy { + discardOldItems { + numToKeep(-1) + daysToKeep(-1) + } + } +} diff --git a/scripts/webapp_seed.groovy b/scripts/webapp_seed.groovy index f8761f2..4effe73 100644 --- a/scripts/webapp_seed.groovy +++ b/scripts/webapp_seed.groovy @@ -15,4 +15,3 @@ multibranchPipelineJob('webapp') { } } } -