Adapt documentation, examples and demo scripts to Quarkus runtime

.gitignore

@@ -20,6 +20,7 @@
 regtests/derby.log
 regtests/metastore_db
 regtests/output/
+regtests/docker-compose.override.yml
 
 # Python stuff
 /poetry.lock

README.md

@@ -39,15 +39,36 @@ for contribution guidelines.
 ## Building and Running 
 
 Apache Polaris is organized into the following modules:
+
 - `polaris-core` - The main Polaris entity definitions and core business logic
-- `polaris-server` - The Polaris REST API server
-- `polaris-eclipselink` - The Eclipselink implementation of the MetaStoreManager interface
+- API modules (implementing the Iceberg REST API and Polaris management API):
+  - `polaris-api-management-model` - The Polaris management model
+  - `polaris-api-management-service` - The Polaris management service
+  - `polaris-api-iceberg-service` - The Iceberg REST service
+- Service modules:
+  - `polaris-service-common` - The main components of the Polaris server
+- Quarkus runtime modules:
+  - `polaris-quarkus-service` - The Quarkus-specific components of the Polaris server
+  - `polaris-quarkus-server` - The Polaris server runtime
+  - `polaris-quarkus-admin-tool` - The Polaris admin & maintenance tool
+- Persistence modules
+  - `polaris-jpa-model` - The JPA entity definitions
+  - `polaris-eclipselink` - The Eclipselink implementation of the MetaStoreManager interface
 
 Apache Polaris is built using Gradle with Java 21+ and Docker 27+.
+
 - `./gradlew build` - To build and run tests. Make sure Docker is running, as the integration tests depend on it.
 - `./gradlew assemble` - To skip tests.
 - `./gradlew test` - To run unit tests and integration tests.
-- `./gradlew runApp` - To run the Polaris server locally on localhost:8181. 
+
+For local development, you can run the following commands:
+
+- `./gradlew --console=plain :polaris-quarkus-service:quarkusRun` - To run the Polaris server
+  locally, with profile `prod`; the server is reachable at localhost:8181.
+- `./gradlew --console=plain :polaris-quarkus-service:quarkusDev` - To run the Polaris server
+  locally, in [Quarkus Dev mode](https://quarkus.io/guides/dev-mode-differences). In dev mode, 
+  Polaris uses the `test` Authenticator and `test` TokenBroker; this configuration is suitable for 
+  running regressions tests, or for connecting with Spark.
 - `./regtests/run_spark_sql.sh` - To connect from Spark SQL. Here are some example commands to run in the Spark SQL shell:
 ```sql
 create database db1;
@@ -57,36 +78,76 @@ insert into db1.table1 values (1, 'a');
 select * from db1.table1;
 ```
 
-Apache Polaris supports the following optional build options:
-- `-PeclipseLink=true` – Enables the EclipseLink extension.
-- `-PeclipseLinkDeps=[groupId]:[artifactId]:[version],...` – Specifies one or more additional dependencies for EclipseLink (e.g., JDBC drivers) separated by commas.
-
 ### More build and run options
-Running in Docker
-- `docker build -t localhost:5001/polaris:latest .` - To build the image.
-  - Optional build options:
-    - `docker build -t localhost:5001/polaris:latest --build-arg ECLIPSELINK=true .` - Enables the EclipseLink extension.
-    - `docker build -t localhost:5001/polaris:latest --build-arg ECLIPSELINK=true --build-arg ECLIPSELINK_DEPS=[groupId]:[artifactId]:[version],... .` – Enables the EclipseLink extension with one or more additional dependencies for EclipseLink (e.g. JDBC drivers) separated by commas.
-- `docker run -p 8181:8181 localhost:5001/polaris:latest` - To run the image in standalone mode.
-
-Running in Kubernetes
-- `./run.sh` - To run Polaris as a mini-deployment locally. This will create one pod that bind itself to ports `8181` and `8182`.
-  - Optional run options:
-    - `./run.sh -b "ECLIPSELINK=true"` - Enables the EclipseLink extension.
-    - `./run.sh -b "ECLIPSELINK=true;ECLIPSELINK_DEPS=[groupId]:[artifactId]:[version],..."` – Enables the EclipseLink extension with one or more additional dependencies for EclipseLink (e.g. JDBC drivers) separated by commas.
-- `kubectl port-forward svc/polaris-service -n polaris 8181:8181 8182:8182` - To create secure connections between a local machine and a pod within the cluster for both service and metrics endpoints.
-  - Currently supported metrics endpoints:
-    - localhost:8182/metrics
-    - localhost:8182/healthcheck
+
+#### Running in Docker
+
+Please note: there are no official Docker images for Apache Polaris yet. For now, you can build the
+Docker images locally.
+
+To build the Polaris server Docker image locally:
+
+```shell
+./gradlew clean :polaris-quarkus-server:assemble -Dquarkus.container-image.build=true
+```
+
+To run the Polaris server Docker image:
+
+```shell
+docker run -p 8181:8181 -p 8182:8182 apache/polaris:latest
+```
+
+#### Running in Kubernetes
+
+- `./run.sh` - To run Polaris as a mini-deployment locally. This will create a Kind cluster, 
+  then deploy one pod and one service. The service is available on ports `8181` and `8182`.
+- `kubectl port-forward svc/polaris-service -n polaris 8181:8181 8182:8182` - To create secure 
+  connections between a local machine and a pod within the cluster for both service and metrics 
+  endpoints.
+  - Currently supported metrics and health endpoints:
+    - http://localhost:8182/q/metrics
+    - http://localhost:8182/q/health
 - `kubectl get pods -n polaris` - To check the status of the pods.
 - `kubectl get deployment -n polaris` - To check the status of the deployment.
 - `kubectl describe deployment polaris-deployment -n polaris` - To troubleshoot if things aren't working as expected.
 
-Running regression tests
-- `./regtests/run.sh` - To run regression tests in another terminal.
-- `docker compose up --build --exit-code-from regtest` - To run regression tests in a Docker environment.
+#### Running regression tests
+
+Regression tests can be run in a local environment or in a Docker environment.
+
+To run regression tests locally, you need to have a Polaris server running locally, with the 
+`test` Authenticator enabled. You can do this by running Polaris in Quarkus Dev mode, as explained
+above:
+
+```shell
+./gradlew --console=plain :polaris-quarkus-service:quarkusDev
+```
+
+Then, you can run the regression tests using the following command:
+
+```shell
+./regtests/run.sh
+```
+
+To run regression tests in a Docker environment, you can use the following command:
+
+```shell
+docker compose -f regtests/docker-compose.yml up --build --exit-code-from regtest
+```
+
+The above command will by default run Polaris with the Docker image `apache/polaris:latest`; if you
+want to use a different image, you can modify the `docker-compose.yaml` file prior to running it;
+alternatively, you can use the following commands to override the image:
+
+```shell
+cat <<EOF > regtests/docker-compose.override.yml
+services: { polaris: { image: localhost:5001/apache/polaris:latest } }
+EOF
+docker compose -f regtests/docker-compose.yml up --build --exit-code-from regtest
+```
+
+#### Building docs
 
-Building docs
 - Docs are generated using [Hugo](https://gohugo.io/) using the [Docsy](https://www.docsy.dev/docs/) theme.
 - To view the site locally, run
   ```bash

k8/deployment.yaml

@@ -39,7 +39,7 @@ spec:
     spec:
       containers:
       - name: polaris
-        image: localhost:5001/polaris:latest
+        image: localhost:5001/apache/polaris:latest
         ports:
         - containerPort: 8181
         - containerPort: 8182

quarkus/admin/src/main/resources/application.properties

@@ -24,4 +24,4 @@ quarkus.container-image.push=false
 quarkus.container-image.registry=docker.io
 quarkus.container-image.group=apache
 quarkus.container-image.name=polaris-admin-tool
-quarkus.container-image.additional-tags=latest
+quarkus.container-image.additional-tags=latest

quarkus/service/src/main/resources/application.properties

@@ -77,7 +77,7 @@ polaris.realm-context.realms=realm1,realm2,realm3
 polaris.realm-context.header-name=Polaris-Realm
 
 polaris.features.defaults."ENFORCE_PRINCIPAL_CREDENTIAL_ROTATION_REQUIRED_CHECKING"=false
-polaris.features.defaults."SUPPORTED_CATALOG_STORAGE_TYPES"=["S3","GCS","AZURE","FILE"]
+polaris.features.defaults."SUPPORTED_CATALOG_STORAGE_TYPES"=["S3","GCS","AZURE"]
 # realm overrides
 # polaris.features.realm-overrides."my-realm"."INITIALIZE_DEFAULT_CATALOG_FILEIO_FOR_TEST"=true
 # polaris.features.realm-overrides."my-realm"."SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION"=true
@@ -148,3 +148,20 @@ polaris.authentication.token-broker.max-token-generation=PT1H
 %test.polaris.storage.aws.secret-key=secretKey
 %test.polaris.storage.gcp.token=token
 %test.polaris.storage.gcp.lifespan=PT1H
+
+%dev.quarkus.log.file.enable=false
+%dev.quarkus.log.category."org.apache.polaris".level=DEBUG
+%dev.quarkus.log.category."org.apache.iceberg.rest".level=DEBUG
+%dev.quarkus.otel.sdk.disabled=true
+%dev.polaris.authentication.authenticator.type=test
+%dev.polaris.authentication.token-service.type=test
+%dev.polaris.authentication.token-broker.type=symmetric-key
+%dev.polaris.authentication.token-broker.symmetric-key.secret=polaris
+%dev.polaris.features.defaults."SUPPORTED_CATALOG_STORAGE_TYPES"=["FILE","S3","GCS","AZURE"]
+%dev.polaris.persistence.type=in-memory
+%dev.polaris.realm-context.realms=default-realm
+%dev.polaris.realm-context.type=test
+%dev.polaris.storage.aws.access-key=accessKey
+%dev.polaris.storage.aws.secret-key=secretKey
+%dev.polaris.storage.gcp.token=token
+%dev.polaris.storage.gcp.lifespan=PT1H