Wfprev 46 (#74)

* terraform for servers * create deploye-dev.yml * mvn-client.yml * update dockerfile * more terraform * more terraform continue * add a readme to describe the purposes of each terraforms files * change github build yml files * change variable name * code review update * typo
bcgov · Sep 25, 2024 · 36052eb · 36052eb
1 parent de18b47
commit 36052eb
Show file tree

Hide file tree

Showing 14 changed files with 1,110 additions and 0 deletions.
diff --git a/.github/workflows/build-full-environment.yml b/.github/workflows/build-full-environment.yml
@@ -0,0 +1,23 @@
+env:
+  IMAGE_NAME: "${{ github.repository }}-${{inputs.PROJECT_NAME}}"
+  NPMRC: ${{ secrets.NPMRC }}
+
+permissions:
+  contents: read
+  packages: write
+
+on:
+  workflow_dispatch:
+    inputs:
+      TAG:
+        type: string
+        description: Additional tag to add to built images
+
+jobs: 
+  wfprev-ui:
+    uses: ./.github/workflows/mvn-build.yml
+    secrets: inherit
+    with:
+      PROJECT_NAME: wfprev-ui
+      TAG: ${{inputs.TAG}}
+      PROJECT_TYPE: client
diff --git a/.github/workflows/mvn-build.yml b/.github/workflows/mvn-build.yml
@@ -0,0 +1,168 @@
+env:
+  IMAGE_NAME: "${{ github.repository }}-${{inputs.PROJECT_NAME}}"
+  NPMRC: ${{ secrets.NPMRC }}
+
+permissions:
+  contents: read
+  packages: write
+
+# Controls when the workflow will run
+on:
+  workflow_dispatch:
+    inputs:
+      PROJECT_NAME:
+        type: string
+        description: 'Name of project, as given to the subfolder of /server or /client to build'
+        required: true
+      TAG:
+        required: false
+        type: string
+        description: 'Additional tag to add to docker image'
+      PROJECT_TYPE:
+        required: true
+        type: choice
+        options: 
+          - server
+          - client
+          - libs
+  workflow_call:
+    inputs:
+      PROJECT_NAME:
+        type: string
+        required: true
+      TAG:
+        required: false
+        type: string
+      PROJECT_TYPE:
+        required: true
+        type: string
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    environment:
+      name: ${{ inputs.ENVIRONMENT }}
+    steps:
+      - uses: actions/checkout@v3
+        if: ${{ inputs.IS_HOTFIX != 'true' }}
+
+      - uses: actions/checkout@v3
+        if: ${{ inputs.IS_HOTFIX == 'true' }}
+        with:
+          ref: ${{ inputs.BRANCH }}
+
+      - name: Set up JDK 17
+        uses: actions/setup-java@v3
+        with:
+          java-version: '17'
+          distribution: 'adopt'
+
+      - name: Add .npmrc file
+        run: echo -e $NPMRC > ~/.npmrc
+
+      - name: Build with Maven
+        run: |
+          export MAVEN_OPTS="-Xmx3072m -XX:MaxMetaspaceSize=512m"
+          export NODE_OPTIONS="--max-old-space-size=4096"
+          mvn --settings client/mvn_settings/settings.xml --batch-mode --update-snapshots -f client/pom.xml -Drepo.login=${{ secrets.IDIR_AS_EMAIL }} -Drepo.password=${{ secrets.IDIR_PASSWORD }} -DskipTests package
+        
+      - name: Copy files to neccessary folders (frontend)
+        run: mkdir staging && cp client/wfprev-war/target/*.war staging
+
+      - uses: actions/upload-artifact@v3
+        with:
+          name: Package-client
+          path: staging
+
+      - name: Cache local Maven repository
+        uses: actions/cache@v2
+        with:
+          path: staging
+          key: ${{ inputs.PROJECT_NAME }}-maven-${{ hashFiles('**war.xml') }}
+          restore-keys: |
+            ${{ inputs.PROJECT_NAME }}-maven-
+
+  docker:
+    needs: build
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - uses: actions/checkout@v3
+        if: ${{ github.event.inputs.IS_HOTFIX != 'true' }}
+
+      - id: trimBranchName
+        name: trim branch name if necessary
+        run: |
+          export branchTag="${{ github.ref_name }}"
+          trimTag="${branchTag##*/}"
+          echo "BRANCH_TAG=$trimTag" >> $GITHUB_OUTPUT
+
+      # - uses: actions/checkout@v3
+      #   if: ${{ github.event.inputs.IS_HOTFIX == 'true' }}
+      #   with:
+      #     ref: ${{ inputs.BRANCH }}
+
+      - name: download artifact (server)
+        if: ${{ inputs.PROJECT_TYPE == 'server' }}
+        uses: actions/download-artifact@v3
+        with:
+          name: ${{ inputs.PROJECT_NAME }}-package
+          path: ${{inputs.PROJECT_TYPE}}/${{inputs.PROJECT_NAME}}
+
+      - name: download artifact (client)
+        if: ${{ inputs.PROJECT_TYPE == 'client' }}
+        uses: actions/download-artifact@v3
+        with:
+          name: ${{ inputs.PROJECT_NAME }}-package
+          path: ${{inputs.PROJECT_TYPE}}/${{inputs.PROJECT_NAME}}
+
+      # - name: Update image name if PR number present
+      #   if: ${{ github.event.pull_request.number > 0 || inputs.IS_HOTFIX == 'true' }}
+      #   run: echo "IMAGE_NAME = ${{env.IMAGE_NAME}}-${{ inputs.IS_HOTFIX == 'true' && 'hotfix' || github.event.pull_request.number }}" >> $GITHUB_ENV
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v2
+        with:
+          registry: ${{ vars.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker (with PR)
+#       if: ${{ github.event.pull_request.number > 0 || inputs.IS_HOTFIX == 'true' }}
+        id: meta_pr
+        uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
+        with:
+          images: ${{ vars.REGISTRY}}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=schedule
+            type=raw,value=${{ steps.trimBranchName.outputs.BRANCH_TAG }}
+            type=ref,event=tag
+            type=raw,value=${{ inputs.TAG }}
+
+      # - name: Build and push Docker image (backend)
+      #   if: ${{ inputs.PROJECT_TYPE == 'backend' }}
+      #   uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
+      #   with:
+      #     context: ${{inputs.PROJECT_TYPE}}/${{inputs.PROJECT_NAME}}
+      #     build-args: |
+      #       CONTAINER_NAME=${{inputs.PROJECT_NAME}}
+      #     push: true
+      #     tags: ${{ steps.meta_pr.outputs.tags }}
+      #     labels: ${{ steps.meta_pr.outputs.labels }}
+
+      - name: Build and push Docker image (frontend)
+        if: ${{ inputs.PROJECT_TYPE == 'client' }}
+        uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
+        with:
+          context: ${{inputs.PROJECT_TYPE}}/${{inputs.PROJECT_NAME}}
+          tags: ${{ steps.meta_pr.outputs.tags }}
+          labels: ${{ steps.meta_pr.outputs.labels }}
diff --git a/.github/workflows/terragrunt-deploy.yml b/.github/workflows/terragrunt-deploy.yml
@@ -0,0 +1,125 @@
+name: Deploy using terragrunt
+
+permissions:
+  contents: read
+  packages: write
+
+on:
+  workflow_dispatch:
+    inputs:
+      DEFAULT_APPLICATION_ENVIRONMENT:
+        required: true
+        type: choice
+        options:
+        - dev
+        - test
+        - prod
+      IMAGE_TAG:
+        required: true
+        type: string
+      SCHEMA_NAME:
+        required: true
+        type: string
+        default: app_wf1_prev
+      CHANGELOG_NAME:
+        type: choice
+        options:
+          - none
+          - changelog_app_wf1_prev
+          - changelog_app_wf1_prev_proxy
+      TARGET_LIQUIBASE_TAG: 
+        required: false
+        type: string
+      IS_HOTFIX:
+        required: true
+        type: string
+        default: 'false'
+
+env:
+  TF_VERSION: 1.8.5
+  TG_VERSION: 0.48.4
+  TG_SRC_PATH: terraform
+  REPOSITORY: ghcr.io
+
+jobs:
+  deploy:
+    name: Deploy
+    runs-on: ubuntu-22.04
+    environment: ${{ inputs.DEFAULT_APPLICATION_ENVIRONMENT }}
+    permissions:
+      id-token: write # This is required for requesting the JWT
+      contents: read  # This is required for actions/checkout
+
+    steps:
+      - uses: actions/checkout@v3
+
+      # - name: retrieve lambda artifacts
+      #   uses: actions/download-artifact@v3
+      #   with:
+      #     name: lambda-functions
+      #     path: ${{env.TG_SRC_PATH}}/lambda-functions
+
+      - id: changeLogCount
+        name: Determine changelog to execute
+        run: |
+          if [ "${{ inputs.CHANGELOG_NAME }}" == "changelog_app_wf1_prev" ]; then
+            echo "NONPROXY_COUNT=1" >> $GITHUB_OUTPUT
+            echo "PROXY_COUNT=0" >> $GITHUB_OUTPUT
+          elif [ "${{ inputs.CHANGELOG_NAME }}" == "changelog_app_wf1_prev_proxy" ] ; then
+            echo "NONPROXY_COUNT=0" >> $GITHUB_OUTPUT
+            echo "PROXY_COUNT=1" >> $GITHUB_OUTPUT
+          else
+            echo "NONPROXY_COUNT=0" >> $GITHUB_OUTPUT
+            echo "PROXY_COUNT=0" >> $GITHUB_OUTPUT
+          fi
+
+      - id: liquibaseCommand
+        name: Determine liquibase command
+        run: |
+          if [ "${{ inputs.TARGET_LIQUIBASE_TAG }}" == "" ]; then
+            echo "LIQUIBASE_COMMAND=update" >> $GITHUB_OUTPUT
+            echo "TARGET_LIQUIBASE_TAG=""" >> $GITHUB_OUTPUT
+          else
+            echo "LIQUIBASE_COMMAND=update-to-tag" >> $GITHUB_OUTPUT
+            echo "TARGET_LIQUIBASE_TAG=--tag=${{inputs.TARGET_LIQUIBASE_TAG}}" >> $GITHUB_OUTPUT
+          fi
+
+      - name: Configure AWS Credentials
+        uses: aws-actions/configure-aws-credentials@v2
+        with:
+          role-to-assume: ${{ secrets.AWS_TERRAFORM_ROLE_TO_ASSUME }}
+          role-session-name: wfprev-terraform-s3
+          aws-region: ca-central-1
+
+      - uses: hashicorp/setup-terraform@v2
+        with:
+          terraform_version: ${{ env.TF_VERSION }}
+          cli_config_credentials_token: ${{ secrets.TFC_TEAM_TOKEN }}
+
+      - uses: peter-murray/[email protected]
+        with:
+          terragrunt_version: ${{ env.TG_VERSION }}
+
+      - name: Terragrunt Apply
+        working-directory: ${{env.TG_SRC_PATH}}
+
+        env:
+          # Necessary for all components
+          GITHUB_USERNAME: ${{ secrets.FETCH_IMAGE_USER }}
+          GITHUB_TOKEN: ${{ secrets.FETCH_IMAGE_TOKEN }}
+          TFC_PROJECT: ${{ secrets.TFC_PROJECT }}
+          TARGET_ENV: ${{ inputs.DEFAULT_APPLICATION_ENVIRONMENT }}
+          APP_COUNT: ${{vars.APP_COUNT}}
+          LOGGING_LEVEL: ${{vars.LOGGING_LEVEL}}
+
+          # Necessary for WFPREV API
+          WFPREV_API_NAME: wfprev-api
+          WFPREV_API_IMAGE: ${{ vars.REPOSITORY }}/${{ github.repository }}-wfprev-api:${{ inputs.IMAGE_TAG }}
+          WFPREV_API_CPU_UNITS: ${{vars.WFPREV_API_CPU_UNITS}}
+          WFPREV_API_MEMORY: ${{vars.WFPREV_API_MEMORY}}
+          WFPREV_API_PORT: ${{vars.WFPREV_API_PORT}}
+
+          # WFPREV UI
+          client_image: ${{ vars.REPOSITORY }}/${{ github.repository }}-wfprev-ui:${{ inputs.IMAGE_TAG }}
+          WEBADE_OAUTH2_WFPREV_UI_CLIENT_SECRET: ${{ secrets.WEBADE_OAUTH2_WFPREV_UI_CLIENT_SECRET }}
+        run: terragrunt apply --terragrunt-non-interactive -auto-approve
diff --git a/client/mvn_settings/settings.xml b/client/mvn_settings/settings.xml
@@ -0,0 +1,56 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<settings xsi:schemaLocation="https://maven.apache.org/SETTINGS/1.1.0 https://maven.apache.org/xsd/settings-1.1.0.xsd" xmlns="https://maven.apache.org/SETTINGS/1.1.0"
+ xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance">
+<servers>
+<server>
+  <username>${repo.login}</username>
+  <password>${repo.password}</password>
+  <id>central</id>
+</server>
+<server>
+  <username>${repo.login}</username>
+  <password>${repo.password}</password>
+<id>snapshots</id>
+</server>
+</servers>
+<profiles>
+<profile>
+<repositories>
+ <repository>
+   <snapshots>
+     <enabled>false</enabled>
+   </snapshots>
+   <id>central</id>
+   <name>libs-release</name>
+   <url>https://bwa.nrs.gov.bc.ca/int/artifactory/libs-release</url>
+ </repository>
+ <repository>
+   <snapshots />
+   <id>snapshots</id>
+   <name>libs-snapshot</name>
+   <url>https://bwa.nrs.gov.bc.ca/int/artifactory/libs-snapshot</url>
+ </repository>
+</repositories>
+<pluginRepositories>
+ <pluginRepository>
+   <snapshots>
+     <enabled>false</enabled>
+   </snapshots>
+   <id>central</id>
+   <name>plugins-release</name>
+   <url>https://bwa.nrs.gov.bc.ca/int/artifactory/plugins-release</url>
+ </pluginRepository>
+ <pluginRepository>
+   <snapshots />
+   <id>snapshots</id>
+   <name>plugins-snapshot</name>
+   <url>https://bwa.nrs.gov.bc.ca/int/artifactory/plugins-snapshot</url>
+ </pluginRepository>
+</pluginRepositories>
+<id>artifactory</id>
+</profile>
+</profiles>
+<activeProfiles>
+ <activeProfile>artifactory</activeProfile>
+</activeProfiles>
+</settings>
diff --git a/terraform/README.md b/terraform/README.md
@@ -0,0 +1,20 @@
+alb.tf:
+The alb.tf file configures an Application Load Balancer (ALB) to route traffic to the WFPREV API and UI. It creates a secure internal ALB with specified security groups and subnets. The file also sets up listeners on the ALB to handle HTTP requests and define fixed responses for unmatched routes. It configures target groups for both the API and UI, specifying health checks and routing traffic based on health status. Additionally, the file includes an example SSL certificate for testing purposes, with a note to replace it with a real certificate in a production environment.
+
+api_gateway.tf: 
+The api_gateway.tf file configures an API Gateway to expose the WFPREV API for external access. It sets up a secure connection between the API Gateway and the internal VPC resources using a VPC link, and routes incoming HTTP requests to a load balancer within the VPC. The file also defines deployment stages and automates redeployment when changes are made, ensuring a consistent and reliable API interface across different environments.
+
+ecs.tf:
+This file sets up an ECS cluster for deploying the WFPREV server and client applications using FARGATE and FARGATE_SPOT capacity providers. It defines task definitions for both server and client containers, including resource allocations, environment variables, and logging configurations. The file also creates ECS services to manage these tasks, linking them to the appropriate load balancers for traffic routing and health checks.
+
+iam.tf:
+This file defines IAM roles and policies to grant necessary permissions for ECS tasks and Lambda functions. It configures roles for ECS and Lambda execution, attaches relevant AWS policies for accessing resources like SQS, VPC, and RDS, and includes custom policies for specific tasks like CloudFront invalidation.
+
+main.tf:
+The main.tf file sets up the Terraform configuration, specifying the required AWS provider version (~> 4.0) and ensuring Terraform itself is version 1.1.0 or higher. This establishes the foundation for managing AWS resources within the project
+
+secrets.tf:
+The secrets.tf file manages GitHub credentials in AWS Secrets Manager. It creates a secret named bcws_wfprev_creds_${var.TARGET_ENV} and stores the GitHub username and token in a secret version for secure access and management.
+
+terragrunt-deploy.yml:
+The terragrunt-deploy.yml GitHub Actions workflow automates the deployment process for the WFPREV application using Terragrunt. It supports multiple environments (dev, test, prod) and various configuration options, such as image tags and schema names. The workflow sets up AWS credentials, Terraform, and Terragrunt, then applies the Terragrunt configurations to deploy the application infrastructure and services. It also determines which Liquibase command to run for database changes and manages environment-specific variables for the deployment process.