Merge branch 'main' into IngestionStatusCallbackUpdate

README.MD

@@ -1,16 +1,144 @@
 # Pyris V2
-## With local environment
+Pyris is an intermediary system that links the [Artemis](https://github.com/ls1intum/Artemis) platform with various Large Language Models (LLMs). It provides a REST API that allows Artemis to interact with different pipelines based on specific tasks.
 
-### Setup
+## Features
+- **Modular Design**: Pyris is built to be modular, allowing for integration of new models and pipelines. This design helps the system adapt to different requirements.
+- **RAG Support**: Pyris implements Retrieval-Augmented Generation (RAG) using [Weaviate](https://weaviate.io/), a vector database. This feature enables the generation of responses based on retrieved context, potentially improving the relevance of outputs.
+- **Flexible Pipelines**: The system supports various pipelines that can be selected depending on the task at hand, providing versatility in handling different types of requests.
+
+Currently, Pyris empowers [Iris](https://artemis.cit.tum.de/about-iris), a virtual AI Tutor that helps students with their programming exercises on Artemis in a didactically meaningful way.
+
+## Setup
+### With local environment
+> **⚠️ Warning:** For local Weaviate vector database setup, please refer to [Weaviate Docs](https://weaviate.io/developers/weaviate/quickstart).
  - Check python version: `python --version` (should be 3.12)
  - Install packages: `pip install -r requirements.txt`
+ - Create an `application.local.yml` file in the root directory. This file includes configurations that can be used by the application.
+    - Example `application.local.yml`:
+     ```yaml
+    api_keys:
+      - token: "secret"
+
+    weaviate:
+      host: "localhost"
+      port: "8001"
+      grpc_port: "50051"
+
+    env_vars:
+      test: "test"
+     ```
+ - Create an `llm-config.local.yml` file in the root directory. This file includes a list of models with their configurations that can be used by the application.
+   - Example `llm-config.local.yml`:
+     ```yaml
+      - id: "<model-id>"
+        name: "<custom-model-name>"
+        description: "<model-description>"
+        type: "<model-type>, e.g. azure-chat, ollama"
+        endpoint: "<your-endpoint>"
+        api_version: "<your-api-version>"
+        azure_deployment: "<your-azure-deployment-name>"
+        model: "<model>, e.g. gpt-3.5-turbo"
+        api_key: "<your-api-key>"
+        tools: []
+        capabilities:
+          input_cost: 0.5
+          output_cost: 1.5
+          gpt_version_equivalent: 3.5
+          context_length: 16385
+          vendor: "<your-vendor>"
+          privacy_compliance: True
+          self_hosted: False
+          image_recognition: False
+          json_mode: True
+     ```
+ - Each model configuration in the `llm-config.local.yml` file also include capabilities that will be used by the application to select the best model for a specific task.
 
-### Run server
+#### Run server
  - Run server: 
    ```[bash]
      APPLICATION_YML_PATH=<path-to-your-application-yml-file> LLM_CONFIG_PATH=<path-to-your-llm-config-yml> uvicorn app.main:app --reload
     ```
  - Access API docs: http://localhost:8000/docs
 
-## With docker
-TBD
+### With docker
+Pyris can be deployed using Docker, which provides an easy way to set up the application in a consistent environment.
+Below are the instructions for setting up Pyris using Docker.
+
+#### Prerequisites
+- Ensure Docker and Docker Compose are installed on your machine.
+- Clone the Pyris repository to your local machine.
+- 
+#### Setup Instructions
+
+1. **Build and Run the Containers**
+
+   You can run Pyris in different environments: development or production. Docker Compose is used to orchestrate the different services, including Pyris, Weaviate, and Nginx.
+
+   - **For Development:**
+
+     Use the following command to start the development environment:
+
+     ```bash
+     docker-compose -f docker-compose/pyris-dev.yml up --build
+     ```
+
+     This command will:
+     - Build the Pyris application from the Dockerfile.
+     - Start the Pyris application along with Weaviate in development mode.
+     - Mount the local configuration files (`application.local.yml` and `llm-config.local.yml`) for easy modification.
+
+     The application will be available at `http://localhost:8000`.
+
+   - **For Production:**
+
+     Use the following command to start the production environment:
+
+     ```bash
+     docker-compose -f docker-compose/pyris-production.yml up -d
+     ```
+
+     This command will:
+     - Pull the latest Pyris image from the GitHub Container Registry.
+     - Start the Pyris application along with Weaviate and Nginx in production mode.
+     - Nginx will serve as a reverse proxy, handling SSL termination if certificates are provided.
+
+     The application will be available at `https://<your-domain>`.
+
+2. **Configuration**
+
+   - **Weaviate**: Weaviate is configured via the `weaviate.yml` file. By default, it runs on port 8001.
+   - **Pyris Application**: The Pyris application configuration is handled through environment variables and mounted YAML configuration files.
+   - **Nginx**: Nginx is used for handling requests in a production environment and is configured via `nginx.yml`.
+
+3. **Accessing the Application**
+
+   - For development, access the API documentation at: `http://localhost:8000/docs`
+   - For production, access the application at your domain (e.g., `https://<your-domain>`).
+
+4. **Stopping the Containers**
+
+   To stop the running containers, use:
+
+   ```bash
+   docker-compose -f docker-compose/pyris-dev.yml down
+   ```
+
+   or
+
+   ```bash
+   docker-compose -f docker-compose/pyris-production.yml down
+   ```
+
+5. **Logs and Debugging**
+
+   - View the logs for a specific service, e.g., Pyris:
+
+     ```bash
+     docker-compose -f docker-compose/pyris-dev.yml logs pyris-app
+     ```
+
+   - For production, ensure that Nginx and Weaviate services are running smoothly and check their respective logs if needed.
+
+---
+
+This setup should help you run the Pyris application in both development and production environments with Docker. Ensure you modify the configuration files as per your specific requirements before deploying.

app/domain/__init__.py

@@ -3,6 +3,9 @@
 from .pipeline_execution_settings_dto import PipelineExecutionSettingsDTO
 from .chat.chat_pipeline_execution_dto import ChatPipelineExecutionDTO
 from .chat.chat_pipeline_execution_base_data_dto import ChatPipelineExecutionBaseDataDTO
+from .competency_extraction_pipeline_execution_dto import (
+    CompetencyExtractionPipelineExecutionDTO,
+)
 from app.domain.chat.exercise_chat.exercise_chat_pipeline_execution_dto import (
     ExerciseChatPipelineExecutionDTO,
 )

app/domain/chat/chat_pipeline_execution_dto.py

@@ -2,16 +2,11 @@
 
 from pydantic import Field
 
-from app.domain import PipelineExecutionDTO, PipelineExecutionSettingsDTO
+from app.domain import PipelineExecutionDTO
 from app.domain.pyris_message import PyrisMessage
 from app.domain.data.user_dto import UserDTO
-from app.domain.status.stage_dto import StageDTO
 
 
 class ChatPipelineExecutionDTO(PipelineExecutionDTO):
     chat_history: List[PyrisMessage] = Field(alias="chatHistory", default=[])
     user: Optional[UserDTO]
-    settings: Optional[PipelineExecutionSettingsDTO]
-    initial_stages: Optional[List[StageDTO]] = Field(
-        default=None, alias="initialStages"
-    )

app/domain/competency_extraction_pipeline_execution_dto.py

@@ -0,0 +1,22 @@
+from typing import List
+
+from pydantic import Field, BaseModel
+
+from . import PipelineExecutionDTO
+from .data.competency_dto import CompetencyTaxonomy, Competency
+
+
+class CompetencyExtractionPipelineExecutionDTO(BaseModel):
+    execution: PipelineExecutionDTO
+    course_description: str = Field(alias="courseDescription")
+    current_competencies: list[Competency] = Field(
+        alias="currentCompetencies", default=[]
+    )
+    taxonomy_options: List[CompetencyTaxonomy] = Field(
+        alias="taxonomyOptions", default=[]
+    )
+    max_n: int = Field(
+        alias="maxN",
+        description="Maximum number of competencies to extract from the course description",
+        default=10,
+    )

app/domain/data/competency_dto.py

@@ -3,6 +3,7 @@
 from typing import Optional
 
 from pydantic import BaseModel, Field
+from pydantic.v1 import validator
 
 
 class CompetencyTaxonomy(str, Enum):
@@ -21,3 +22,29 @@ class CompetencyDTO(BaseModel):
     taxonomy: Optional[CompetencyTaxonomy] = None
     soft_due_date: Optional[datetime] = Field(default=None, alias="softDueDate")
     optional: Optional[bool] = None
+
+
+class Competency(BaseModel):
+    title: str = Field(
+        description="Title of the competency that contains no more than 4 words",
+    )
+    description: str = Field(
+        description="Description of the competency as plain string. DO NOT RETURN A LIST OF STRINGS."
+    )
+    taxonomy: CompetencyTaxonomy = Field(
+        description="Selected taxonomy based on bloom's taxonomy"
+    )
+
+    @validator("title")
+    def validate_title(cls, field):
+        """Validate the subject of the competency."""
+        if len(field.split()) > 4:
+            raise ValueError("Title must contain no more than 4 words")
+        return field
+
+    @validator("taxonomy")
+    def validate_selected_taxonomy(cls, field):
+        """Validate the selected taxonomy."""
+        if field not in CompetencyTaxonomy.__members__:
+            raise ValueError(f"Invalid taxonomy: {field}")
+        return field

app/domain/data/simple_submission_dto.py

@@ -1,8 +1,13 @@
+from typing import Optional
+
 from pydantic import BaseModel, Field
 
 from datetime import datetime
 
 
 class SimpleSubmissionDTO(BaseModel):
-    timestamp: datetime = Field(alias="timestamp")
-    score: float = Field(alias="score")
+    timestamp: Optional[datetime] = Field(alias="timestamp", default=None)
+    score: Optional[float] = Field(alias="score", default=0)
+
+    class Config:
+        require_by_default = False

app/domain/ingestion/ingestion_pipeline_execution_dto.py

@@ -1,15 +1,15 @@
-from typing import List, Optional
+from typing import List
 
 from pydantic import Field
 
-from app.domain import PipelineExecutionDTO, PipelineExecutionSettingsDTO
+from app.domain import PipelineExecutionDTO
 from app.domain.data.lecture_unit_dto import LectureUnitDTO
-from app.domain.status.stage_dto import StageDTO
 
 
 class IngestionPipelineExecutionDto(PipelineExecutionDTO):
     lecture_unit: LectureUnitDTO = Field(..., alias="pyrisLectureUnit")
     settings: Optional[PipelineExecutionSettingsDTO]
     initial_stages: Optional[List[StageDTO]] = Field(
         default=None, alias="initialStages"
+
     )

app/domain/pipeline_execution_dto.py

@@ -1,8 +1,16 @@
-from pydantic import BaseModel
+from typing import Optional
+
+from pydantic import BaseModel, Field
+
+from app.domain.pipeline_execution_settings_dto import PipelineExecutionSettingsDTO
+from app.domain.status.stage_dto import StageDTO
 
 
 class PipelineExecutionDTO(BaseModel):
-    pass
+    settings: Optional[PipelineExecutionSettingsDTO]
+    initial_stages: Optional[list[StageDTO]] = Field(
+        default=None, alias="initialStages"
+    )
 
     class Config:
         populate_by_name = True

app/domain/status/competency_extraction_status_update_dto.py

@@ -0,0 +1,6 @@
+from app.domain.data.competency_dto import Competency
+from app.domain.status.status_update_dto import StatusUpdateDTO
+
+
+class CompetencyExtractionStatusUpdateDTO(StatusUpdateDTO):
+    result: list[Competency] = []

app/llm/external/openai_chat.py

@@ -1,12 +1,13 @@
 import logging
 import time
+import traceback
 from datetime import datetime
 from typing import Literal, Any
 
 from openai import OpenAI
 from openai.lib.azure import AzureOpenAI
 from openai.types.chat import ChatCompletionMessage, ChatCompletionMessageParam
-from openai.types.chat.completion_create_params import ResponseFormat
+from openai.types.shared_params import ResponseFormatJSONObject
 
 from ...common.message_converters import map_str_to_role, map_role_to_str
 from app.domain.data.text_message_content_dto import TextMessageContentDTO
@@ -93,7 +94,7 @@ def chat(
                         messages=convert_to_open_ai_messages(messages),
                         temperature=arguments.temperature,
                         max_tokens=arguments.max_tokens,
-                        response_format=ResponseFormat(type="json_object"),
+                        response_format=ResponseFormatJSONObject(type="json_object"),
                     )
                 else:
                     response = self._client.chat.completions.create(
@@ -106,6 +107,7 @@ def chat(
             except Exception as e:
                 wait_time = initial_delay * (backoff_factor**attempt)
                 logging.warning(f"Exception on attempt {attempt + 1}: {e}")
+                traceback.print_exc()
                 logging.info(f"Retrying in {wait_time} seconds...")
                 time.sleep(wait_time)
         logging.error("Failed to interpret image after several attempts.")

app/pipeline/__init__.py

@@ -1 +1 @@
-from ..pipeline.pipeline import Pipeline
+from app.pipeline.pipeline import Pipeline

app/pipeline/chat/course_chat_pipeline.py

@@ -232,25 +232,24 @@ def get_competency_list() -> list:
             regarding their progress overall or in a specific area.
             A competency has the following attributes: name, description, taxonomy, soft due date, optional,
             and mastery threshold.
-            The response may include metrics for each competency, such as progress and mastery (0%-100%).
+            The response may include metrics for each competency, such as progress and mastery (0% - 100%).
             These are system-generated.
-            The judgment of learning (JOL) values indicate the self-reported confidence by the student (0-5, 5 star).
-            The object describing it also indicates the system-computed confidence at the time when the student
+            The judgment of learning (JOL) values indicate the self-reported mastery by the student (0 - 5, 5 star).
+            The object describing it also indicates the system-computed mastery at the time when the student
             added their JoL assessment.
             """
             self.callback.in_progress("Reading competency list ...")
             if not dto.metrics or not dto.metrics.competency_metrics:
                 return dto.course.competencies
             competency_metrics = dto.metrics.competency_metrics
-            weight = 2.0 / 3.0
             return [
                 {
                     "info": competency_metrics.competency_information.get(comp, None),
                     "exercise_ids": competency_metrics.exercises.get(comp, []),
                     "progress": competency_metrics.progress.get(comp, 0),
-                    "mastery": (
-                        (1 - weight) * competency_metrics.progress.get(comp, 0)
-                        + weight * competency_metrics.confidence.get(comp, 0)
+                    "mastery": get_mastery(
+                        competency_metrics.progress.get(comp, 0),
+                        competency_metrics.confidence.get(comp, 0),
                     ),
                     "judgment_of_learning": (
                         competency_metrics.jol_values.get[comp].json()
@@ -267,7 +266,7 @@ def lecture_content_retrieval() -> str:
             """
             Retrieve content from indexed lecture slides.
             This will run a RAG retrieval based on the chat history on the indexed lecture slides and return the most
-             relevant paragraphs.
+            relevant paragraphs.
             Use this if you think it can be useful to answer the student's question, or if the student explicitly asks
             a question about the lecture content or slides.
             Only use this once.