Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/master' into course-instances-re…
Browse files Browse the repository at this point in the history
…work
  • Loading branch information
nygrenh committed Jan 23, 2025
2 parents d3f7af3 + 40facbd commit 51ebb96
Show file tree
Hide file tree
Showing 363 changed files with 62,288 additions and 101,271 deletions.
2 changes: 2 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,8 @@ module.exports = {
"JSON.parse",
"addFilter",
"removeFilter",
"setError",
"clearErrors",
],
},
"object-properties": {
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/system-tests-and-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ jobs:
sudo sysctl net/netfilter/nf_conntrack_max=131072
- name: Install, start & configure minikube
uses: manusa/actions-setup-minikube@v2.11.0
uses: manusa/actions-setup-minikube@v2.13.0
with:
minikube version: "v1.33.1"
kubernetes version: "v1.30.0"
Expand Down
2 changes: 1 addition & 1 deletion .husky/lint-staged.branch-ready.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// These checks are slower and more likely to fail than the precommit checks.
// See lint-staged.precommit.config.js for precommit checks.
module.exports = {
"*.{js,jsx,ts,tsx}": ["eslint --cache --fix", "stylelint --fix"],
"*.{js,jsx,ts,tsx}": ["eslint --cache --fix", "stylelint --fix lax"],
"services/example-exercise/src/**/*.{js,jsx,ts,tsx}": () =>
"npx tsc -p services/example-exercise/ --noEmit",
"services/cms/src/**/*.{js,jsx,ts,tsx}": () =>
Expand Down
2 changes: 1 addition & 1 deletion .husky/lint-staged.precommit.config.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Runs on precommit.
// Focuses on tests that are fast to run don't fail often. For other checks, see bin/git-run-branch-ready-checks and lint-staged.branch-ready.config.js
module.exports = {
"*.{js,jsx,ts,tsx}": ["eslint --cache --fix", "stylelint --fix"],
"*.{js,jsx,ts,tsx}": ["eslint --cache --fix", "stylelint --fix lax"],
"*.{md,json,scss,css}": "prettier --write",
"*.rs": () => [
"cargo fmt --manifest-path services/headless-lms/Cargo.toml --all -- --files-with-diff",
Expand Down
2 changes: 2 additions & 0 deletions .stylelintrc
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
"no-duplicate-selectors": null,
"function-no-unknown": null,
"media-query-no-invalid": null,
"declaration-property-value-no-unknown": null,
"at-rule-descriptor-no-unknown": null,
"selector-type-no-unknown": [
true,
{
Expand Down
103 changes: 103 additions & 0 deletions docs/deployment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Deployments

This project is designed to run on Kubernetes, ensuring a consistent and reliable setup across all environments. For local development, we use **Minikube** and **Skaffold** to start a Kubernetes cluster locally and deploy all project services in a production-like manner.

> [!TIP]
>
> ## TL;DR
>
> - **Automated Production Deployment**: Fully automated with GitHub Actions, Skaffold, and Minikube.
> - **Production Environment**: Deployed to Google Cloud Platform (GCP) Kubernetes.
> - **Local Testing**: Use `bin/test` to replicate the production environment locally.
> - **Run System Tests**: Use `bin/system-tests-run-tests` for end-to-end testing. This validates that the system as a whole functions correctly before deployment.
## Production Deployment Workflow

Our CI/CD pipeline ensures efficient and reliable deployments through three main stages:

### 1. Build

- Multi-stage builds are used to create:
- **Cache Images**: Captures dependencies to speed up future builds.
- **Slim Images**: Deployment-ready images with only essential components.
- Docker images and build outputs are saved for reuse during testing and deployment.

### 2. Test

- Images are deployed to a Minikube Kubernetes cluster using Skaffold.
- System tests, via `bin/system-tests-run-tests`, validate application behavior.
- Additional unit and integration tests are executed in the pipeline.
- Deployment is blocked if any test fails.

### 3. Deploy

- If all tests pass on the `master` branch:
- Docker images are pushed to a cloud container registry.
- Skaffold runs database migrations.
- Validated images are deployed to the GCP Kubernetes cluster using Skaffold and Kustomize.

This workflow ensures every change is tested in a production-like environment before reaching live users.

## Kustomize for Environment Management

We use **Kustomize** to manage and customize Kubernetes manifests across environments. A shared base configuration defines most settings, while overlays adjust environment-specific configurations. This ensures consistency while keeping configurations modular.

| Environment | Inherits From | Key Differences |
| --------------- | ------------- | --------------------------------------------------------------------------------------------------- |
| **Base** | None | Contains most configurations; other environments override specifics as needed. |
| **Development** | Base | Used in `bin/dev`; services run in development mode. |
| **Production** | Base | Used for live deployments; services run in production mode. |
| **Test** | Production | Used in `bin/test`; mirrors production but uses local resources like databases for testing locally. |

## Skaffold and Its Integration with Kustomize

**Skaffold** orchestrates the development and deployment workflows for Kubernetes. It is configured to use **Kustomize**, which provides environment-specific Kubernetes manifests.

### How They Work Together:

1. **Kustomize** defines the manifests:
- The base configuration contains shared resources.
- Overlays customize configurations for environments like development, production, and testing.
2. **Skaffold** manages the deployment pipeline:
- For development, Skaffold uses the Kustomize overlay for the `development` environment to deploy locally.
- For production, Skaffold applies the `production` overlay to deploy live services on GCP Kubernetes.

For example:

- **Local Development**: Skaffold applies `kubernetes/dev/kustomization.yaml` to deploy services on Minikube.
- **Production Deployment**: Skaffold applies `kubernetes/production/kustomization.yaml` to deploy services to GCP Kubernetes.

## Ingress Configuration

We use **Ingress** in Kubernetes to handle HTTP routing between multiple services deployed in the same cluster. Powered by the **Ingress-NGINX** controller, it routes HTTP requests based on paths, allowing all services to share a single domain.

Examples of routing:

- `/api`**headless-lms**
- `/cms`**cms**
- `/quizzes`**quizzes**

Routing rules are defined in `kubernetes/base/ingress.yml`. For more details, refer to the [Ingress Documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/) or the [Ingress-NGINX Documentation](https://kubernetes.github.io/ingress-nginx/).

## Local and Production Environments

| Feature | `bin/dev` (Development Mode) | `bin/test` (Local Prod Mode) | Production Deployment |
| ----------------------------- | -------------------------------------------- | ---------------------------------------------- | --------------------------- |
| **Environment Variables** | Development | Development, but production settings turned on | Production-secured values |
| **Database** | Local DB with seed data (`headless_lms_dev`) | Local DB with seed data (`headless_lms_test`) | Cloud SQL for PostgreSQL |
| **Build Process** | Compiled in development mode | Compiled in production mode | Compiled in production mode |
| **Deployment Tool** | Skaffold | Skaffold | Skaffold |
| **Kubernetes Cluster** | Minikube | Minikube | GCP Kubernetes |
| **Domain** | `project-331.local` | `project-331.local` | `courses.mooc.fi` |
| **Deployments Configuration** | `skaffold.yml` | `skaffold.production.yml` | `skaffold.production.yml` |

**`bin/dev`** is optimized for rapid iteration with live reloading. **`bin/test`** closely mirrors production, enabling a production-like system to be tested on any machine. The production deployment process is similar to `bin/test`, but it uses live infrastructure.

## Useful Links

- [Skaffold Documentation](https://skaffold.dev/docs/)
- [Kubernetes Documentation](https://kubernetes.io/docs/)
- [Kustomize Documentation](https://kustomize.io/)
- [Kubernetes Ingress Documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/)
- [Ingress-NGINX Documentation](https://kubernetes.github.io/ingress-nginx/)
- [Google Cloud Kubernetes Engine](https://cloud.google.com/kubernetes-engine)
57 changes: 57 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Secret Project 331 Documentation Index

## 📌 Introduction

**Secret Project 331** is a Learning Management System (LMS) developed by the MOOC Center of the University of Helsinki. It's deployed at [https://courses.mooc.fi](https://courses.mooc.fi). This documentation is intended for developers and provides technical insights, setup instructions, and detailed guides to facilitate development and contribution.

For non-developers:

- **Learn more about the courses**: [https://www.mooc.fi](https://www.mooc.fi)
- **Teacher Documentation**: Refer to the **Wiki** tab in this repository.

## 🚀 Getting Started

- Set up local environment: [Development Environment Setup](./Development.md)
- Configure VDI: [VDI Setup for MOOC Center Employees](./vdi-setup.md)

## 🏛 Architecture

- System architecture overview: [Project Architecture](./architecture.md)

## 💻 Development Guides

- Interact with backend, shared modules: [Frontend Development](./frontend.md)
- Database interactions, migrations: [Backend Development](./headless-lms.md)
- Extend LMS with new exercises: [Plugin System](./plugin-system.md)
- Implement responsive designs: [Mobile-First CSS](./mobile-first-css.md)
- Support multiple languages: [Internationalization](./internationalization.md)
- Develop new blocks for CMS and course material: [Blocks Development](./blocks.md)

## 📦 Deployment

- Deployment process and CI/CD: [Deployment Workflow](./deployment.md)
- Test/manage Dockerfile changes: [Testing Changes to Dockerfiles](./updating-dockerfiles.md)

## 🗄 Database Operations

- Perform database operations: [DataGrip Operations](./datagrip-operations.md)

## 🗂 Version Control

- Enhance Git workflow: [GIT Notes](./git.md)

## 🧪 Testing

- Write and execute tests: [Testing](./tests.md)

## 🔄 Updating Dependencies

- [Updating Dependencies Guide](./updating-dependencies.md)

## 📚 Additional Documentation

- Additional notes: [Miscellaneous Notes](./etc.md)

## 📞 Support and Contributions

- **Project Repository**: [GitHub Repository](https://github.com/rage/secret-project-331)
1 change: 1 addition & 0 deletions docs/plugin-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ Communication between the parent page and the IFrame is restricted to specific m
| `current-state` | IFrame | Parent | Informs the parent that the IFrame's state has changed. Includes data and validity status. |
| `height-changed` | IFrame | Parent | Notifies the parent that the content height has changed, allowing the parent to resize the IFrame. |
| `set-language` | Parent | IFrame | Informs the IFrame of the user's preferred language using IETF BCP 47 language tags. |
| `open-link` | Iframe | Parent | The IFrame requests a link to be opened in the browser's main browsing context. |

### Views

Expand Down
Loading

0 comments on commit 51ebb96

Please sign in to comment.