Polish LLM benchmark

Polish LLM benchmark is a compact yet powerful platform designed for benchmarking in langue open-source large language models (LLMs) on polish language. Equipped with diverse test datasets, it facilitates rigorous and comprehensive evaluation of LLMs. This tool streamlines the process of assessing model performance, enabling researchers and developers to reliably test and share their findings in the dynamic landscape of AI and natural language processing.

Table of Contents

Getting Started

• For Liunx
• For Windows

Run Benchamrk

• For Liunx

• For Windows

  Below That

• Prerequisites

• Installing

• Running the Tests

• Deployment

• Built With

• Contributing

• Versioning

• Authors

• License

• Acknowledgments

Getting Started

This section will guide you through getting a copy of the project up and running on your local machine for development and testing purposes. Before you start, ensure you have Git installed on your system. If not, you can download and install it from Git's official website.

Prerequisites

• Git installed on your machine.
• Docker installed for running the application in a container (optional but recommended).

Setup Instructions

For Linux Users

Clone the repository to get a local copy on your machine, Change into the project directory, Run the setup script to configure the environment.

git clone https://github.com/knmlprz/Benchmark_LLM.git
cd Benchmark_LLM
./set_venv.sh
source venv/bin/activate

For Windows Users

Clone the repository to get a local copy on your machine, Change into the project directory, Run the setup script to configure the environment

git clone https://github.com/knmlprz/Benchmark_LLM.git
cd Benchmark_LLM
set_venv.bat
venv\Scripts\activate.bat

Running the Benchmark

For Linux

First Time Setup

Build and start the Docker containers. This step is required only once or after any changes in the Docker configuration.

docker compose up --build

Regular Usage

Start the Docker containers without rebuilding.

docker compose up

Shutdown

To stop and remove the containers, networks, and volumes, use:

docker compose down

For Windows

First Time Setup

Build and start the Docker containers. This step is required only once or after any changes in the Docker configuration.

docker compose up --build

Regular Usage

Start the Docker containers without rebuilding.

docker compose up

Shutdown

To stop and remove the containers, networks, and volumes, use:

docker compose down

Below That Good First Issue