diff --git a/README.MD b/README.MD index a3bba0c4..2773c331 100644 --- a/README.MD +++ b/README.MD @@ -1,16 +1,344 @@ # Pyris V2 -## With local environment -### Setup - - Check python version: `python --version` (should be 3.12) - - Install packages: `pip install -r requirements.txt` +Pyris is an intermediary system that connects the [Artemis](https://github.com/ls1intum/Artemis) platform with various Large Language Models (LLMs). It provides a REST API that allows Artemis to interact with different pipelines based on specific tasks. -### Run server - - Run server: - ```[bash] - APPLICATION_YML_PATH= LLM_CONFIG_PATH= uvicorn app.main:app --reload - ``` - - Access API docs: http://localhost:8000/docs +Currently, Pyris powers [Iris](https://artemis.cit.tum.de/about-iris), a virtual AI tutor that assists students with their programming exercises on Artemis in a pedagogically meaningful way. -## With docker -TBD \ No newline at end of file +## Table of Contents +- [Features](#features) +- [Setup](#setup) + - [Prerequisites](#prerequisites) + - [Local Development Setup](#local-development-setup) + - [Docker Setup](#docker-setup) + - [Development Environment](#development-environment) + - [Production Environment](#production-environment) +- [Customizing Configuration](#customizing-configuration) +- [Troubleshooting](#troubleshooting) +- [Additional Notes](#additional-notes) + +## Features + +- **Exercise Support**: Empowers Iris to provide feedback on programming exercises, enhancing the learning experience for students. Iris analyzes submitted code, feedback, and build logs generated by Artemis to provide detailed insights. + +- **Course Content Support**: Leverages RAG (Retrieval-Augmented Generation) to enable Iris to provide detailed explanations for course content, making it easier for students to understand complex topics based on instructor-provided learning materials. + +- **Competency Generation**: Automates the generation of competencies for courses, reducing manual effort in creating Artemis competencies. + +## Setup + +### Prerequisites + +- **Python 3.12**: Ensure that Python 3.12 is installed. + + ```bash + python --version + ``` + +- **Docker and Docker Compose**: Required for containerized deployment. + +--- + +### Local Development Setup + +> **Note:** If you need to modify the local Weaviate vector database setup, please refer to the [Weaviate Documentation](https://weaviate.io/developers/weaviate/quickstart). + +#### Steps + +1. **Clone the Pyris Repository** + + Clone the Pyris repository into a directory on your machine: + + ```bash + git clone https://github.com/ls1intum/Pyris.git Pyris + ``` + +2. **Install Dependencies** + + Navigate to the Pyris directory and install the required Python packages: + + ```bash + cd Pyris + pip install -r requirements.txt + ``` + +3. **Create Configuration Files** + + - **Create an Application Configuration File** + + Create an `application.local.yml` file in the root directory. You can use the provided `application.example.yml` as a base. + + ```bash + cp application.example.yml application.local.yml + ``` + + **Example `application.local.yml`:** + + ```yaml + api_keys: + - token: "your-secret-token" + + weaviate: + host: "localhost" + port: "8001" + grpc_port: "50051" + + env_vars: + ``` + + - **Create an LLM Config File** + + Create an `llm-config.local.yml` file in the root directory. You can use the provided `llm-config.example.yml` as a base. + + ```bash + cp llm-config.example.yml llm-config.local.yml + ``` + + **Example OpenAI Configuration:** + + ```yaml + - id: "oai-gpt-35-turbo" + name: "GPT 3.5 Turbo" + description: "GPT 3.5 16k" + type: "openai_chat" + model: "gpt-3.5-turbo" + api_key: "" + tools: [] + capabilities: + input_cost: 0.5 + output_cost: 1.5 + gpt_version_equivalent: 3.5 + context_length: 16385 + vendor: "OpenAI" + privacy_compliance: false + self_hosted: false + image_recognition: false + json_mode: true + ``` + + **Example Azure OpenAI Configuration:** + + ```yaml + - id: "azure-gpt-4-omni" + name: "GPT 4 Omni" + description: "GPT 4 Omni on Azure" + type: "azure_chat" + endpoint: "" + api_version: "2024-02-15-preview" + azure_deployment: "gpt4o" + model: "gpt4o" + api_key: "" + tools: [] + capabilities: + input_cost: 6 + output_cost: 16 + gpt_version_equivalent: 4.5 # Equivalent GPT version of the model + context_length: 128000 + vendor: "OpenAI" + privacy_compliance: true + self_hosted: false + image_recognition: true + json_mode: true + ``` + + **Explanation of Configuration Parameters** + + The configuration parameters are used by Pyris's capability system to select the appropriate model for a task. + + **Parameter Descriptions:** + + - `api_key`: The API key for the model. + - `capabilities`: The capabilities of the model. + + - `context_length`: The maximum number of tokens the model can process in a single request. + - `gpt_version_equivalent`: The equivalent GPT version of the model in terms of overall capabilities. + - `image_recognition`: Whether the model supports image recognition (for multimodal models). + - `input_cost`: The cost of input tokens for the model. + - `output_cost`: The cost of output tokens for the model. + - `json_mode`: Whether the model supports structured JSON output mode. + - `privacy_compliance`: Whether the model complies with privacy regulations. + - `self_hosted`: Whether the model is self-hosted. + - `vendor`: The provider of the model (e.g., OpenAI). + - `speed`: The model's processing speed. + + - `description`: Additional information about the model. + - `id`: Unique identifier for the model across all models. + - `model`: The official name of the model as used by the vendor. + - `name`: A custom, human-readable name for the model. + - `type`: The model type, used to select the appropriate client (e.g., `openai_chat`, `azure_chat`, `ollama`). + - `endpoint`: The URL to connect to the model. + - `api_version`: The API version to use with the model. + - `azure_deployment`: The deployment name of the model on Azure. + - `tools`: The tools supported by the model. + + > **Notes on `gpt_version_equivalent`:** The `gpt_version_equivalent` field is subjective and used to compare capabilities of different models using GPT models as a reference. For example: + >- GPT-4 Omni equivalent: 4.5 + >- GPT-4 Omni Mini equivalent: 4.25 + >- GPT-4 equivalent: 4 + >- GPT-3.5 Turbo equivalent: 3.5 + + > **Warning:** Most existing pipelines in Pyris require a model with a `gpt_version_equivalent` of **4.5 or higher**. It is advised to define models in the `llm-config.local.yml` file with a `gpt_version_equivalent` of 4.5 or higher. + +4. **Run the Server** + + Start the Pyris server: + + ```bash + APPLICATION_YML_PATH=./application.local.yml \ + LLM_CONFIG_PATH=./llm-config.local.yml \ + uvicorn app.main:app --reload + ``` + +5. **Access API Documentation** + + Open your browser and navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to access the interactive API documentation. + +--- + +### Docker Setup + +Deploying Pyris using Docker ensures a consistent environment and simplifies the deployment process. + +#### Prerequisites + +- **Docker**: Install Docker from the [official website](https://www.docker.com/get-started). +- **Docker Compose**: Comes bundled with Docker Desktop or install separately on Linux. +- **Clone the Pyris Repository**: If not already done, clone the repository. +- **Create Configuration Files**: Create the `application.local.yml` and `llm-config.local.yml` files as described in the [Local Development Setup](#local-development-setup) section. + + ```bash + git clone https://github.com/ls1intum/Pyris.git Pyris + cd Pyris + ``` + +#### Docker Compose Files + +- **Development**: `docker-compose/pyris-dev.yml` +- **Production with Nginx**: `docker-compose/pyris-production.yml` +- **Production without Nginx**: `docker-compose/pyris-production-internal.yml` + +#### Running the Containers + +##### **Development Environment** + +1. **Start the Containers** + + ```bash + docker-compose -f docker-compose/pyris-dev.yml up --build + ``` + + - Builds the Pyris application. + - Starts Pyris and Weaviate in development mode. + - Mounts local configuration files for easy modification. + +2. **Access the Application** + + - Application URL: [http://localhost:8000](http://localhost:8000) + - API Docs: [http://localhost:8000/docs](http://localhost:8000/docs) + +##### **Production Environment** + +###### **Option 1: With Nginx** + +1. **Prepare SSL Certificates** + + - Place your SSL certificate (`fullchain.pem`) and private key (`priv_key.pem`) in the specified paths or update the paths in the Docker Compose file. + +2. **Start the Containers** + + ```bash + docker-compose -f docker-compose/pyris-production.yml up -d + ``` + + - Pulls the latest Pyris image. + - Starts Pyris, Weaviate, and Nginx. + - Nginx handles SSL termination and reverse proxying. + +3. **Access the Application** + + - Application URL: `https://your-domain.com` + +###### **Option 2: Without Nginx** + +1. **Start the Containers** + + ```bash + docker-compose -f docker-compose/pyris-production-internal.yml up -d + ``` + + - Pulls the latest Pyris image. + - Starts Pyris and Weaviate. + +2. **Access the Application** + + - Application URL: [http://localhost:8000](http://localhost:8000) + +--- + +#### Managing the Containers + +- **Stop the Containers** + + ```bash + docker-compose -f down + ``` + + Replace `` with the appropriate Docker Compose file. + +- **View Logs** + + ```bash + docker-compose -f logs -f + ``` + + Example: + + ```bash + docker-compose -f docker-compose/pyris-dev.yml logs -f pyris-app + ``` + +- **Rebuild Containers** + + If you've made changes to the code or configurations: + + ```bash + docker-compose -f up --build + ``` + +#### Customizing Configuration + +- **Environment Variables** + + You can customize settings using environment variables: + + - `PYRIS_DOCKER_TAG`: Specifies the Pyris Docker image tag. + - `PYRIS_APPLICATION_YML_FILE`: Path to your `application.yml` file. + - `PYRIS_LLM_CONFIG_YML_FILE`: Path to your `llm-config.yml` file. + - `PYRIS_PORT`: Host port for Pyris application (default is `8000`). + - `WEAVIATE_PORT`: Host port for Weaviate REST API (default is `8001`). + - `WEAVIATE_GRPC_PORT`: Host port for Weaviate gRPC interface (default is `50051`). + +- **Configuration Files** + + Modify configuration files as needed: + + - **Pyris Configuration**: Update `application.yml` and `llm-config.yml`. + - **Weaviate Configuration**: Adjust settings in `weaviate.yml`. + - **Nginx Configuration**: Modify Nginx settings in `nginx.yml` and related config files. + +## Troubleshooting + +- **Port Conflicts** + + If you encounter port conflicts, change the host ports using environment variables: + + ```bash + export PYRIS_PORT=8080 + ``` + +- **Permission Issues** + + Ensure you have the necessary permissions for files and directories, especially for SSL certificates. + +- **Docker Resources** + + If services fail to start, ensure Docker has sufficient resources allocated. \ No newline at end of file diff --git a/app/pipeline/competency_extraction_pipeline.py b/app/pipeline/competency_extraction_pipeline.py index da224ffe..a2288ab5 100644 --- a/app/pipeline/competency_extraction_pipeline.py +++ b/app/pipeline/competency_extraction_pipeline.py @@ -31,7 +31,12 @@ def __init__(self, callback: Optional[CompetencyExtractionCallback] = None): implementation_id="competency_extraction_pipeline_reference_impl" ) self.callback = callback - self.request_handler = CapabilityRequestHandler(requirements=RequirementList()) + self.request_handler = CapabilityRequestHandler( + requirements=RequirementList( + gpt_version_equivalent=4.5, + context_length=16385, + ) + ) self.output_parser = PydanticOutputParser(pydantic_object=Competency) def __call__( diff --git a/application.test.yml b/application.example.yml similarity index 61% rename from application.test.yml rename to application.example.yml index be23a6d7..9e1490ba 100644 --- a/application.test.yml +++ b/application.example.yml @@ -4,7 +4,4 @@ api_keys: weaviate: host: "localhost" port: "8001" - grpc_port: "50051" - -env_vars: - test: "test" \ No newline at end of file + grpc_port: "50051" \ No newline at end of file diff --git a/docker/pyris-production-internal.yml b/docker/pyris-production-internal.yml new file mode 100644 index 00000000..bd4aa04a --- /dev/null +++ b/docker/pyris-production-internal.yml @@ -0,0 +1,36 @@ +# ---------------------------------------------------------------------------------------------------------------------- +# Setup for a Pyris server suitable for internal network requests (without nginx). +# ---------------------------------------------------------------------------------------------------------------------- +# It is designed to take in environment variables for configuration, similar to the production setup. +# ---------------------------------------------------------------------------------------------------------------------- + +services: + pyris-app: + extends: + file: ./pyris.yml + service: pyris-app + image: ghcr.io/ls1intum/pyris:${PYRIS_DOCKER_TAG:-latest} + pull_policy: always + restart: unless-stopped + volumes: + - ${PYRIS_APPLICATION_YML_FILE}:/config/application.yml:ro + - ${PYRIS_LLM_CONFIG_YML_FILE}:/config/llm_config.yml:ro + ports: + - "${PYRIS_PORT:-8000}:8000" + networks: + - pyris + + weaviate: + extends: + file: ./weaviate.yml + service: weaviate + ports: + - "${WEAVIATE_PORT:-8001}:8001" + - "${WEAVIATE_GRPC_PORT:-50051}:50051" + networks: + - pyris + +networks: + pyris: + driver: "bridge" + name: pyris \ No newline at end of file diff --git a/llm_config.example.yml b/llm_config.example.yml new file mode 100644 index 00000000..c7c66714 --- /dev/null +++ b/llm_config.example.yml @@ -0,0 +1,124 @@ +- api_key: + api_version: 2024-05-01-preview + azure_deployment: gpt-35-turbo + capabilities: + context_length: 16385 + gpt_version_equivalent: 3.5 + image_recognition: false + input_cost: 0.5 + json_mode: true + output_cost: 1.5 + privacy_compliance: true + self_hosted: false + vendor: OpenAI + description: GPT 3.5 16k on Azure + endpoint: '' + id: azure-gpt-35-turbo + model: gpt-3.5-turbo + name: GPT 3.5 Turbo + type: azure_chat +- api_key: + capabilities: + input_cost: 0.5 + output_cost: 1.5 + gpt_version_equivalent: 3.5 + context_length: 16385 + vendor: "OpenAI" + privacy_compliance: false + self_hosted: false + image_recognition: false + son_mode: true + description: GPT 3.5 16k + id: oai-gpt-35-turbo + model: gpt-3.5-turbo + name: GPT 3.5 Turbo + type: openai_chat +- api_key: + api_version: 2024-02-15-preview + azure_deployment: gpt-4-turbo + capabilities: + context_length: 128000 + gpt_version_equivalent: 4 + image_recognition: false + input_cost: 10 + json_mode: true + output_cost: 30 + privacy_compliance: true + self_hosted: false + vendor: OpenAI + description: GPT 4 Turbo 128k on Azure + endpoint: '' + id: azure-gpt-4-turbo + model: gpt-4-turbo + name: GPT 4 Turbo + type: azure_chat +- api_key: + api_version: 2024-02-15-preview + azure_deployment: gpt-4o + capabilities: + context_length: 128000 + gpt_version_equivalent: 4.5 + image_recognition: true + input_cost: 5 + json_mode: true + output_cost: 15 + privacy_compliance: true + self_hosted: false + vendor: OpenAI + description: GPT 4 Omni on Azure + endpoint: '' + id: azure-gpt-4-omni + model: gpt-4o + name: GPT 4 Omni + type: azure_chat +- api_key: + api_version: 2023-03-15-preview + azure_deployment: gpt-4o-mini + capabilities: + context_length: 128000 + gpt_version_equivalent: 4.25 + image_recognition: true + input_cost: 0.15 + json_mode: true + output_cost: 0.075 + privacy_compliance: true + self_hosted: false + vendor: OpenAI + description: GPT 4 Omni Mini on Azure + endpoint: '' + id: azure-gpt-4-omni-mini + model: gpt-4o-mini + name: GPT 4 Omni Mini + type: azure_chat +- api_key: + api_version: '2023-05-15T00:00:00.000Z' + azure_deployment: te-3-large + capabilities: + context_length: 8191 + input_cost: 0.13 + output_cost: 0.065 + privacy_compliance: true + self_hosted: false + vendor: OpenAI + description: Embedding Large 8k Azure + endpoint: '' + id: embedding-large + model: text-embedding-3-large + name: Embedding Large + type: azure_embedding +- api_key: + api_version: 2024-02-15-preview + azure_deployment: te-3-small + capabilities: + context_length: 8191 + input_cost: 0.02 + output_cost: 0 + privacy_compliance: true + self_hosted: false + vendor: OpenAI + description: Embedding Small 8k Azure + endpoint: '' + id: embedding-small + model: text-embedding-3-small + name: Embedding Small + type: azure_embedding \ No newline at end of file