Update with initial Django/DRF-based Model/API work (#1)

---------

Co-authored-by: Jon Walz <[email protected]>
Co-authored-by: Abram Booth <[email protected]>

.github/dependabot.yml

@@ -0,0 +1,31 @@
+# config for Dependabot updates -- see docs:
+# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
+
+version: 2
+updates:
+  # python dependencies
+  - package-ecosystem: "pip"
+    directory: "/requirements/"
+    schedule:
+      interval: "daily"
+    labels:
+      - "update"
+    target-branch: "develop"
+
+  # Dockerfile dependencies
+  - package-ecosystem: "docker"
+    directory: "/"
+    schedule:
+      interval: "daily"
+    labels:
+      - "update"
+    target-branch: "develop"
+
+  # github actions used in .github/workflows/
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "daily"
+    labels:
+      - "update"
+    target-branch: "develop"

.github/workflows/run_gravyvalet_tests.yml

@@ -0,0 +1,60 @@
+name: run_gravyvalet_tests
+
+on:
+  push:
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  run_gravyvalet_tests:
+    strategy:
+      fail-fast: false
+      matrix:  # use to test upgrades before upgrading
+        python-version: ['3.12']
+        postgres-version: ['15']
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:${{ matrix.postgres-version }}
+        env:
+          POSTGRES_HOST_AUTH_METHOD: trust
+        # Set health checks to wait until postgres has started
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+        ports:
+          - 5432:5432
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: set up python${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+          cache-dependency-path: |
+            requirements/requirements.txt
+            requirements/dev-requirements.txt
+
+      - name: install py dependencies
+        run: pip install -r requirements/dev-requirements.txt
+
+      - name: set up pre-commit cache
+        uses: actions/cache@v3
+        with:
+          path: ~/.cache/pre-commit
+          key: pre-commit|${{ matrix.python-version }}|${{ hashFiles('.pre-commit-config.yaml') }}
+
+      - name: run pre-commit checks
+        run: pre-commit run --all-files --show-diff-on-failure
+
+      - name: run tests
+        run: python3 manage.py test
+        env:
+          DEBUG: 1
+          POSTGRES_HOST: localhost
+          POSTGRES_DB: gravyvalettest
+          POSTGRES_USER: postgres
+          SECRET_KEY: oh-so-secret

.gitignore

@@ -1,3 +1,4 @@
 .python-version
 db.sqlite3
 __pycache__
+.venv

.pre-commit-config.yaml

@@ -0,0 +1,13 @@
+repos:
+  - repo: https://github.com/psf/black
+    rev: 23.11.0
+    hooks:
+      - id: black
+  - repo: https://github.com/pycqa/flake8
+    rev: 6.1.0
+    hooks:
+      - id: flake8
+  - repo: https://github.com/pycqa/isort
+    rev: 5.12.0
+    hooks:
+    - id: isort

Dockerfile

@@ -0,0 +1,18 @@
+# Use the official Python image as the base image
+FROM python:3.12
+
+# System Dependencies:
+RUN apt-get update && apt-get install -y libpq-dev
+
+WORKDIR /code
+COPY requirements/ /code/requirements/
+
+# Python dependencies:
+RUN pip3 install --no-cache-dir -r requirements/requirements.txt
+
+COPY . /code/
+
+EXPOSE 8000
+
+# Start the Django development server
+CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]

README.md

@@ -1,24 +1,42 @@
-# Gravyvalet
+![Center for Open Science Logo](https://mfr.osf.io/export?url=https://osf.io/download/24697/?direct=%26mode=render&format=2400x2400.jpeg)
 
-A thicker, more hands-on counterpart to waterbutler.
+# OSF Addon Service (GravyValet)
 
-# Reason for being
+Welcome to the Open Science Framework's base server for addon integration with our RESTful API (osf.io). This server acts as a gateway between the OSF and external APIs. Authenticated users or machines can access various resources through common file storage and citation management APIs via the OSF. Institutional members can also add their own integrations, tailoring addon usage to their specific communities.
 
-The goal is to split out OSF addons into their own well-encapsulated service. This is the prototype/initial version 
+## Setting up GravyValet Locally
 
-## Approach
+1. Start your PostgreSQL and Django containers with `docker compose up -d`.
+2. Enter the Django container: `docker compose exec addon_service /bin/bash`.
+3. Migrate the existing models: `python3 manage.py migrate`.
+4. Visit [http://0.0.0.0:8004/](http://0.0.0.0:8004/).
 
-Mostly just started off by figuring out the necessary endpoints and putting in stubs.  Then started inlining code from the OSF, chasing down things through base classes, decorators, and utility functions.  Foolishly attempted to inline the actual django model code for addons.  That broke me. Moved that into a side file and just stubbed out the called model code.  Currently trying to fill out the stubs with simple impls & fixtures.
+## Running Tests
 
-Chose box as the first addon to implement, since it is one of the saner, less corner-casey addons.  Not currently worrying to much about making it extensible, figure that's part of the actual dev.
+To run tests, use the following command:
 
-# Quickstart
+```bash
+python3 manage.py test
+```
+
+Development Tips
 
-It's a Django app. `gravyvalet` is the "root" app, but most of the work is being done in the `charon` app.
+Optionally, but recommended: Set up pre-commit hooks that will run formatters and linters on staged files. Install pre-commit using:
 
-For no particular reason, I've chosen `8011` as the default gravyvalet port.
+```bash
 
+pip install pre-commit
 ```
-$ pip install -r requirements.txt
-$ python manage.py runserver 8011
+
+Then, run:
+
+```bash
+
+pre-commit install --allow-missing-config
 ```
+Reporting Issues and Questions
+
+If you encounter a bug, have a technical question, or want to request a feature, please don't hesitate to contact us 
+at [email protected]. While we may respond to questions through other channels, reaching out to us at [email protected] ensures 
+that your feedback goes to the right person promptly. If you're considering posting an issue on our GitHub issues page,
+ we recommend sending it to [email protected] instead.

addon_service/apps.py

@@ -0,0 +1,6 @@
+from django.apps import AppConfig
+
+
+class AddonServiceConfig(AppConfig):
+    default_auto_field = "django.db.models.BigAutoField"
+    name = "addon_service"

addon_service/authorized_storage_account/models.py

@@ -0,0 +1,32 @@
+# from django.contrib.postgres.fields import ArrayField
+from django.db import models
+
+from addon_service.common.base_model import AddonsServiceBaseModel
+
+
+class AuthorizedStorageAccount(AddonsServiceBaseModel):
+    # TODO: capabilities = ArrayField(...)
+    default_root_folder = models.CharField(blank=True)
+
+    external_storage_service = models.ForeignKey(
+        "addon_service.ExternalStorageService",
+        on_delete=models.CASCADE,
+        related_name="authorized_storage_accounts",
+    )
+    external_account = models.ForeignKey(
+        "addon_service.ExternalAccount",
+        on_delete=models.CASCADE,
+        related_name="authorized_storage_accounts",
+    )
+
+    class Meta:
+        verbose_name = "Authorized Storage Account"
+        verbose_name_plural = "Authorized Storage Accounts"
+        app_label = "addon_service"
+
+    class JSONAPIMeta:
+        resource_name = "authorized-storage-accounts"
+
+    @property
+    def account_owner(self):
+        return self.external_account.owner  # TODO: prefetch/select_related

addon_service/authorized_storage_account/serializers.py

@@ -0,0 +1,55 @@
+from rest_framework_json_api import serializers
+from rest_framework_json_api.relations import (
+    HyperlinkedRelatedField,
+    ResourceRelatedField,
+)
+from rest_framework_json_api.utils import get_resource_type_from_model
+
+from addon_service.models import (
+    AuthorizedStorageAccount,
+    ConfiguredStorageAddon,
+    ExternalStorageService,
+    InternalUser,
+)
+
+
+RESOURCE_NAME = get_resource_type_from_model(AuthorizedStorageAccount)
+
+
+class AuthorizedStorageAccountSerializer(serializers.HyperlinkedModelSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name=f"{RESOURCE_NAME}-detail")
+    account_owner = HyperlinkedRelatedField(
+        many=False,
+        queryset=InternalUser.objects.all(),
+        related_link_view_name=f"{RESOURCE_NAME}-related",
+    )
+    external_storage_service = ResourceRelatedField(
+        queryset=ExternalStorageService.objects.all(),
+        many=False,
+        related_link_view_name=f"{RESOURCE_NAME}-related",
+    )
+    configured_storage_addons = HyperlinkedRelatedField(
+        many=True,
+        queryset=ConfiguredStorageAddon.objects.all(),
+        related_link_view_name=f"{RESOURCE_NAME}-related",
+    )
+
+    included_serializers = {
+        "account_owner": "addon_service.serializers.InternalUserSerializer",
+        "external_storage_service": (
+            "addon_service.serializers.ExternalStorageServiceSerializer"
+        ),
+        "configured_storage_addons": (
+            "addon_service.serializers.ConfiguredStorageAddonSerializer"
+        ),
+    }
+
+    class Meta:
+        model = AuthorizedStorageAccount
+        fields = [
+            "url",
+            "account_owner",
+            "configured_storage_addons",
+            "default_root_folder",
+            "external_storage_service",
+        ]

addon_service/authorized_storage_account/views.py

@@ -0,0 +1,10 @@
+from rest_framework_json_api.views import ModelViewSet
+
+from .models import AuthorizedStorageAccount
+from .serializers import AuthorizedStorageAccountSerializer
+
+
+class AuthorizedStorageAccountViewSet(ModelViewSet):
+    queryset = AuthorizedStorageAccount.objects.all()
+    serializer_class = AuthorizedStorageAccountSerializer
+    # TODO: permissions_classes

addon_service/common/base_model.py

@@ -0,0 +1,22 @@
+from django.db import models
+from django.utils import timezone
+
+
+class AddonsServiceBaseModel(models.Model):
+    created = models.DateTimeField(editable=False)
+    modified = models.DateTimeField()
+
+    def save(self, *args, **kwargs):
+        if not self.id:
+            self.created = timezone.now()
+        self.modified = timezone.now()
+        super().save(*args, **kwargs)
+
+    def __str__(self):
+        return f"<{self.__class__.__qualname__}(pk={self.pk})>"
+
+    def __repr__(self):
+        return self.__str__()
+
+    class Meta:
+        abstract = True

addon_service/configured_storage_addon/models.py

@@ -0,0 +1,26 @@
+from django.db import models
+
+from addon_service.common.base_model import AddonsServiceBaseModel
+
+
+class ConfiguredStorageAddon(AddonsServiceBaseModel):
+    root_folder = models.CharField()
+
+    authorized_storage_account = models.ForeignKey(
+        "addon_service.AuthorizedStorageAccount",
+        on_delete=models.CASCADE,
+        related_name="configured_storage_addons",
+    )
+    internal_resource = models.ForeignKey(
+        "addon_service.InternalResource",
+        on_delete=models.CASCADE,
+        related_name="configured_storage_addons",
+    )
+
+    class Meta:
+        verbose_name = "Configured Storage Addon"
+        verbose_name_plural = "Configured Storage Addons"
+        app_label = "addon_service"
+
+    class JSONAPIMeta:
+        resource_name = "configured-storage-addons"

addon_service/configured_storage_addon/serializers.py

@@ -0,0 +1,41 @@
+from rest_framework_json_api import serializers
+from rest_framework_json_api.relations import ResourceRelatedField
+from rest_framework_json_api.utils import get_resource_type_from_model
+
+from addon_service.models import (
+    ConfiguredStorageAddon,
+    InternalResource,
+)
+
+
+RESOURCE_NAME = get_resource_type_from_model(ConfiguredStorageAddon)
+
+
+class ConfiguredStorageAddonSerializer(serializers.HyperlinkedModelSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name=f"{RESOURCE_NAME}-detail")
+    authorized_storage_account = ResourceRelatedField(
+        queryset=ConfiguredStorageAddon.objects.all(),
+        many=False,
+        related_link_view_name=f"{RESOURCE_NAME}-related",
+    )
+    internal_resource = ResourceRelatedField(
+        queryset=InternalResource.objects.all(),
+        many=False,
+        related_link_view_name=f"{RESOURCE_NAME}-related",
+    )
+
+    included_serializers = {
+        "authorized_storage_account": (
+            "addon_service.serializers.AuthorizedStorageAccountSerializer"
+        ),
+        "internal_resource": "addon_service.serializers.InternalResourceSerializer",
+    }
+
+    class Meta:
+        model = ConfiguredStorageAddon
+        fields = [
+            "url",
+            "root_folder",
+            "authorized_storage_account",
+            "internal_resource",
+        ]