Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(root): Add Dockerfile and docker-compose #666

Open
wants to merge 16 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
node_modules
out
.env
13 changes: 13 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
NODE_ENV=development
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason we have an example env at the root. Since we have examples per folder that's good to move there where it matches

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was for the GitHub workflow testing purposes, and I will remove it in this PR.
https://github.com/marketplace/actions/create-env-file will handle the .env file creation at GitHub workflow for deployment purposes

DATABASE_URL=mysql://kampus:kampus@localhost:3306/kampus?schema=public
NEXTAUTH_URL=http://localhost:3001/auth
NEXT_PUBLIC_GQL_URL=http://localhost:4000/graphql
AUTH_COOKIE_DOMAIN=''
GITHUB_ID=3748376f66f395c5f36a
GITHUB_SECRET=ae893fe55f3135f7d3f758b36116b52e06de552f
DISCORD_ID=''
DISCORD_SECRET=''
TWITCH_ID=''
TWITCH_SECRET=''
NEXTAUTH_SECRET='Example'
BUILD_STANDALONE=true
33 changes: 33 additions & 0 deletions .github/workflows/deploy_checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Deploy Checks

on:
pull_request:
branches: [ dev ]
jobs:
deploy_preview:
name: Deploy Checks
runs-on: ubuntu-latest
permissions:
deployments: write
contents: read
steps:
- name: Check out code
uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Install jq
run: sudo apt-get update && sudo apt-get install -y jq
- name: Build Docker Compose services
run: docker-compose build
- name: Start Docker Compose services
run: docker-compose up -d
- name: Check Docker Compose services are running
run: |
for service in $(docker-compose ps --services); do
if [ "$(docker inspect -f '{{.State.Status}}' $service)" != "running" ]; then
echo "Service $service is not running"
exit 1
fi
done
- name: Shutdown Docker Compose services
run: docker-compose down
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,7 @@ yarn-error.log*
!deploy/build

.vercel
!/.vscode/extensions.json
!/.vscode/extensions.json

# docker compose
docker-compose.yaml
105 changes: 105 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# syntax = docker/dockerfile:1

# base-node stage contains the environment variables and the node image with slim OS.
FROM node:18 AS node-image
# base-slim-node stage contains the environment variables and the node image with slim OS.
FROM node:18-bullseye-slim AS node-slim-image
# base-distroless stage contains the environment variables and the distroless image.
FROM gcr.io/distroless/nodejs18-debian11 AS node-distroless-image

# Base stage, install turbo and copy the project
FROM node-slim-image AS base
WORKDIR /app
RUN npm install --global turbo
COPY . .

# kampus-prune stage, use "turbo" to get incremental builds and prune the workspace
FROM base AS kampus-prune
RUN turbo prune --scope=@kampus-apps/kampus --docker

# gql-prune stage, use "turbo" to get incremental builds and prune the workspace
FROM base AS gql-prune
RUN turbo prune --scope=@kampus-apps/gql --docker

# pasaport-prune stage, use "turbo" to get incremental builds and prune the workspace
FROM base AS pasaport-prune
RUN turbo prune --scope=@kampus-apps/pasaport --docker

# kampus-builder stage, use "turbo" to build the project incrementally and get incremental builds
FROM node-slim-image as kampus-builder
WORKDIR /app
COPY .gitignore .gitignore
COPY --from=kampus-prune /app/out/json .
COPY --from=kampus-prune /app/out/package-lock.json ./package-lock.json
COPY /tsconfig.json tsconfig.json
RUN npm install
COPY --from=kampus-prune /app/out/full/ .
COPY .env ./apps/pasaport/.env
COPY turbo.json turbo.json
RUN npx turbo build

# python, g++, make, etc. are required for gql dependencies. node:18 is used as base image
FROM node-image AS gql-builder
WORKDIR /app
COPY .gitignore .gitignore
COPY --from=gql-prune /app/out/json .
COPY --from=gql-prune /app/out/package-lock.json ./package-lock.json
COPY /tsconfig.json tsconfig.json
RUN npm install
COPY --from=gql-prune /app/out/full/ .
COPY .env ./apps/pasaport/.env
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@usirin should we make a small script to do this on local as well seems like right now .env.example files can run the applications so local env can utilise this too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from what i read from the next.js docs, the best practice is to switch to a .env.development file and commit that. then people can create a .env.local file if they want to override anything from that files.

Good to know: .env, .env.development, and .env.production files should be included in your repository as they define defaults. .env*.local should be added to .gitignore, as those files are intended to be ignored. .env.local is where secrets can be stored.
https://nextjs.org/docs/app/building-your-application/configuring/environment-variables#default-environment-variables

COPY turbo.json turbo.json
RUN npx turbo build

# pasaport-builder stage, use "turbo" to build the project incrementally and get incremental builds
FROM node-slim-image AS pasaport-builder
WORKDIR /app
COPY .gitignore .gitignore
COPY --from=pasaport-prune /app/out/json .
COPY --from=pasaport-prune /app/out/package-lock.json ./package-lock.json
COPY /tsconfig.json tsconfig.json
RUN npm install
COPY --from=pasaport-prune /app/out/full/ .
COPY .env ./apps/pasaport/.env
COPY turbo.json turbo.json
RUN npx turbo build

# Use distroless to reduce the minimum size of the image and reduce the attack surface
#If you want to use npm, use the node:18-bullseye-slim image (prisma doesn't work with alpine arm64)
# kampus stage - the dev server.
FROM node-distroless-image as kampus
COPY tsconfig.json ./tsconfig.json
COPY --from=kampus-builder /app/apps/kampus/next.config.mjs .
COPY --from=kampus-builder /app/apps/kampus/package.json .
COPY --from=kampus-builder /app/apps/kampus/.next/standalone ./
COPY --from=kampus-builder /app/apps/kampus/.next/static ./apps/kampus/.next/static
COPY --from=kampus-builder /app/apps/kampus/public ./apps/kampus/public
CMD ["apps/kampus/server.js"]

# gql stage - the dev server.
FROM node-distroless-image as gql
COPY tsconfig.json ./tsconfig.json
COPY --from=gql-builder /app/apps/gql/next.config.mjs .
COPY --from=gql-builder /app/apps/gql/package.json .
COPY --from=gql-builder /app/apps/gql/.next/standalone ./
COPY --from=gql-builder /app/apps/gql/.next/static ./apps/gql/.next/static
CMD ["apps/gql/server.js"]

# pasaport stage - the dev server.
FROM node-distroless-image as pasaport
COPY tsconfig.json ./tsconfig.json
COPY --from=pasaport-builder /app/apps/pasaport/next.config.mjs .
COPY --from=pasaport-builder /app/apps/pasaport/package.json .
COPY --from=pasaport-builder /app/apps/pasaport/.next/standalone ./
COPY --from=pasaport-builder /app/apps/pasaport/.next/static ./apps/pasaport/.next/static
CMD ["apps/pasaport/server.js"]

# ssl-proxy stage - the dev server.
FROM node:18-bullseye-slim as proxy
WORKDIR /app

RUN npm i -g local-ssl-proxy
RUN npm i -g concurrently

# keep running
CMD ["concurrently", "local-ssl-proxy --target 3000 --source 5000", "local-ssl-proxy --target 3001 --source 5001", "local-ssl-proxy --target 4000 --source 5002"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can turbo run these ? turbo has concurrent running capability

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at this point we don't need turbo, we pruned everything, and i am not even sure if there is a turbo binary at this point. i think it's better to go with this.

65 changes: 58 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ kamp.us web projects & packages
- sozluk: turkce terimler sozlugu - https://sozluk.dev.kamp.us
- pano: link & bilgi & soru cevap paylasim platformu - https://pano.dev.kamp.us
- gql: https://gql.dev.kamp.us/graphql
- pasaport: kampus icin kullanici bilgileri & [centralized auth](https://sozluk.dev.kamp.us/centralized-auth) servisi - https://pasaport.dev.kamp.us
- pasaport: kampus icin kullanici bilgileri & [centralized auth](https://sozluk.dev.kamp.us/centralized-auth)
servisi - https://pasaport.dev.kamp.us
- storybook: @kampus/ui paketi icin hosted storybook - https://ui.dev.kamp.us

## IMPORTANT NOTE
Expand All @@ -20,7 +21,7 @@ and say hi to us at #kampus-projects channel.
### 1. Cloning the project

- Fork `kamp-us/monorepo` under your personal account.
- eg: `usirin/monorepo`
- eg: `usirin/monorepo`
- Clone the project to your local computer:

```sh
Expand All @@ -38,7 +39,8 @@ npm ci

### 2. Modifying `hosts` file

- Add the following text block to [your `/etc/hosts` file](https://www.howtogeek.com/27350/beginner-geek-how-to-edit-your-hosts-file/).
- Add the following text block
to [your `/etc/hosts` file](https://www.howtogeek.com/27350/beginner-geek-how-to-edit-your-hosts-file/).

```text
127.0.0.1 localhost.kamp.us
Expand All @@ -50,7 +52,8 @@ npm ci

### 3. Setting up `.env` files

- Duplicate `.env.example` files and rename them as `.env` in the following folders: `db/prisma`, `apps/gql`, `apps/kampus`, `apps/pasaport`.
- Duplicate `.env.example` files and rename them as `.env` in the following
folders: `db/prisma`, `apps/gql`, `apps/kampus`, `apps/pasaport`.

### 4. Prisma Setup and Database Configuration

Expand Down Expand Up @@ -111,9 +114,9 @@ npm run gql

- Go to [OAuth Apps page](https://github.com/settings/developers) on GitHub.
- Click on `New OAuth App`.
- Application name: "you can choose any name"
- Homepage URL: http://pasaport.localhost.kamp.us:3001/
- Authorization callback URL: http://pasaport.localhost.kamp.us:3001/auth/callback/github
- Application name: "you can choose any name"
- Homepage URL: http://pasaport.localhost.kamp.us:3001/
- Authorization callback URL: http://pasaport.localhost.kamp.us:3001/auth/callback/github
- Click on `Register Application`.
- On the application page, click on `Generate a new client secret`.
- Make the following changes to the corresponding files.
Expand Down Expand Up @@ -187,3 +190,51 @@ If you haven't already install [Volta](https://volta.sh), you can install instal
```sh
curl https://get.volta.sh | bash
```

## Docker Setup

### Dockerfile

- We use a multi-stage build process to separate the building of applications from the final production image. This
ensures smaller and more optimized production images.
- We build three services: `kampus`, `gql`, and `pasaport`. Each service has its dedicated builder stage followed by its
execution stage.
- The `node:18-bullseye-slim` image is used to keep the final production images lean.

### Docker Compose

- We have four primary services: `mysql`, `kampus`, `gql`, and `pasaport`.
- `mysql` is the database service. We ensure that its authentication is set correctly and ports are mapped to allow for
local development access.
- The other three services are built using the project's Dockerfile. These services are exposed on ports 3000, 3001, and
3002 respectively.
- If you want to run the services with Docker Compose, copy the docker-compose.dev.yml file to docker-compose.yml. This
file is ignored by Git to prevent accidental commits of sensitive information such as secrets and passwords. You will
need to update the environment variables in this file to match your local environment.

```sh
cp docker-compose.dev.yml docker-compose.yml
cp .env.example .env
```

## Running a Fully Clean Build with Docker Compose

- To build the images and run the containers, run the following command:

```sh
docker-compose up --build -d
```

- To stop the containers, run the following command:

```sh
docker-compose down
```

## Running a Fully Clean Build with Docker

- Build and start the services, ensuring Docker Compose builds the images from scratch without using any cache.

```sh
docker-compose up --build --force-recreate
```
3 changes: 2 additions & 1 deletion apps/gql/.env.example
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
NODE_ENV=development
DATABASE_URL=mysql://kampus:kampus@localhost:3306/kampus?schema=public
NEXTAUTH_URL=http://pasaport.localhost.kamp.us:3001/auth
BUILD_STANDALONE=false
NEXTAUTH_URL=http://pasaport.localhost.kamp.us:3001/auth
1 change: 1 addition & 0 deletions apps/gql/next.config.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/** @type {import('next').NextConfig} */
const config = {
transpilePackages: ["@kampus/sozluk-content"],
output: process.env.BUILD_STANDALONE === "true" ? "standalone" : undefined,
};

export default config;
1 change: 1 addition & 0 deletions apps/kampus/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ DISCORD_SECRET=''
TWITCH_ID=''
TWITCH_SECRET=''
NEXTAUTH_SECRET='Example'
BUILD_STANDALONE=false
1 change: 1 addition & 0 deletions apps/kampus/next.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ const config = {
reactStrictMode: true,
swcMinify: true,
transpilePackages: ["@kampus/tailwind", "@kampus/ui", "@kampus/next-auth", "@kampus/email"],
output: process.env.BUILD_STANDALONE === "true" ? "standalone" : undefined,

compiler: {
relay: {
Expand Down
1 change: 1 addition & 0 deletions apps/pasaport/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ DISCORD_SECRET=''
TWITCH_ID=''
TWITCH_SECRET=''
NEXTAUTH_SECRET='Example'
BUILD_STANDALONE=false
1 change: 1 addition & 0 deletions apps/pasaport/next.config.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/** @type {import('next').NextConfig} */
const config = {
transpilePackages: ["@kampus/email", "@kampus/next-auth"],
output: process.env.BUILD_STANDALONE === "true" ? "standalone" : undefined,
};

export default config;
Loading
Loading