Merge pull request #3552 from magda-io/main

merge main into next branch

CHANGES.md

@@ -1,5 +1,28 @@
 # CHANGELOG
 
+## v4.2.1
+
+- #3551: Pagination Bar sometimes might generate `...` button with invalid page number
+
+## v4.2.0
+
+- #3545: Upgraded OpenSearch to v2.15.0
+
+## v4.1.0
+
+- #3487: Update to use ASGS 2021 and latest LGAs (2023) boundaries
+- Added `indexed` field to region index & Bump region index version to `27`
+  - Migration instruction can be found from [the migration doc](./docs/docs/migration/4.1.0.md)
+- Adjusted & improved region config entry skip logic
+- Improved the region loader performance
+- Fixed: region loader failed to convert number type id property into a string
+
+## v4.0.1
+
+- Add helm chart configuration option allows users to disable gateway auto-gzip response feature
+- #3541 Update default region files url to [Github repo release](https://github.com/magda-io/magda-regions)
+- #3543 Make the region mapping served by search api configurable
+
 ## v4.0.0
 
 - Use OpenSearch (v2.14.0) as main search engine

README.md

@@ -81,7 +81,7 @@ If you just want to install a local testing version, installing Magda using [Hel
 # create a namespace "magda" in your cluster
 kubectl create namespace magda
 
-# install Magda version v2.2.0 to namespace "magda", turn off openfass function and expose the service via loadBalancer
+# install Magda version v4.0.0 to namespace "magda", turn off openfass function and expose the service via loadBalancer
 helm upgrade --namespace magda --install --version 4.0.0 --timeout 9999s --set magda-core.gateway.service.type=LoadBalancer magda oci://ghcr.io/magda-io/charts/magda
 ```
 

deploy/helm/internal-charts/admin-api/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: admin-api
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"

deploy/helm/internal-charts/admin-api/README.md

@@ -1,6 +1,6 @@
 # admin-api
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 

deploy/helm/internal-charts/apidocs-server/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: apidocs-server
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"

deploy/helm/internal-charts/apidocs-server/README.md

@@ -1,6 +1,6 @@
 # apidocs-server
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 

deploy/helm/internal-charts/authorization-api/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: authorization-api
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"

deploy/helm/internal-charts/authorization-api/README.md

@@ -1,6 +1,6 @@
 # authorization-api
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 

deploy/helm/internal-charts/authorization-db/Chart.lock

@@ -1,6 +1,6 @@
 dependencies:
 - name: magda-postgres
   repository: file://../magda-postgres
-  version: 4.0.0
-digest: sha256:a129eca790a32e608d21af2f1dd31d1fb4bcd623eacba28f5d73d5725332c743
-generated: "2024-05-24T06:16:18.532685523Z"
+  version: 4.2.1
+digest: sha256:3396d487955d5e56f0af53a08b754bf2a46a5197b64007e778c9ca3226ae469e
+generated: "2024-07-02T10:25:15.604266444Z"

deploy/helm/internal-charts/authorization-db/Chart.yaml

@@ -1,12 +1,12 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: authorization-db
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"
 dependencies:
   - name: magda-postgres
-    version: "4.0.0"
+    version: "4.2.1"
     repository: "file://../magda-postgres"
     condition: global.useInK8sDbInstance.authorization-db

deploy/helm/internal-charts/authorization-db/README.md

@@ -1,6 +1,6 @@
 # authorization-db
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 
@@ -10,7 +10,7 @@ Kubernetes: `>= 1.14.0-0`
 
 | Repository | Name | Version |
 |------------|------|---------|
-| file://../magda-postgres | magda-postgres | 4.0.0 |
+| file://../magda-postgres | magda-postgres | 4.2.1 |
 
 ## Values
 

deploy/helm/internal-charts/cloud-sql-proxy/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: cloud-sql-proxy
-version: 4.0.0
+version: 4.2.1
 appVersion: "1.11"
 kubeVersion: ">= 1.14.0-0"
 annotations:

deploy/helm/internal-charts/cloud-sql-proxy/README.md

@@ -1,6 +1,6 @@
 # cloud-sql-proxy
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square) ![AppVersion: 1.11](https://img.shields.io/badge/AppVersion-1.11-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square) ![AppVersion: 1.11](https://img.shields.io/badge/AppVersion-1.11-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 

deploy/helm/internal-charts/combined-db/Chart.lock

@@ -1,6 +1,6 @@
 dependencies:
 - name: magda-postgres
   repository: file://../magda-postgres
-  version: 4.0.0
-digest: sha256:56d86a1c8dbeda88ac9c76fea9d43626e737b0b15505949c9fc9450ec274de9d
-generated: "2024-05-24T06:16:18.622218899Z"
+  version: 4.2.1
+digest: sha256:68bb76962afd28d471fe2e2059561d9fe82c271426caa13ef5f690e9f104084a
+generated: "2024-07-02T10:25:15.690760576Z"

deploy/helm/internal-charts/combined-db/Chart.yaml

@@ -4,12 +4,12 @@ description: |
   Unless you attempt to run logical DBs over seperate physical DBs (via `global.useInK8sDbInstance.xxx`), 
   you should always enable this module (even when you use cloud based DB service e.g. AWS RDS or Google Cloud SQL).
 name: combined-db
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"
 dependencies:
   - name: magda-postgres
-    version: "4.0.0"
+    version: "4.2.1"
     repository: "file://../magda-postgres"
     condition: global.useCombinedDb

deploy/helm/internal-charts/combined-db/README.md

@@ -1,6 +1,6 @@
 # combined-db
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 The combined database target module for Magda.
 Unless you attempt to run logical DBs over seperate physical DBs (via `global.useInK8sDbInstance.xxx`),
@@ -12,7 +12,7 @@ Kubernetes: `>= 1.14.0-0`
 
 | Repository | Name | Version |
 |------------|------|---------|
-| file://../magda-postgres | magda-postgres | 4.0.0 |
+| file://../magda-postgres | magda-postgres | 4.2.1 |
 
 ## Values
 

deploy/helm/internal-charts/content-api/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: content-api
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"

deploy/helm/internal-charts/content-api/README.md

@@ -1,6 +1,6 @@
 # content-api
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 

deploy/helm/internal-charts/content-db/Chart.lock

@@ -1,6 +1,6 @@
 dependencies:
 - name: magda-postgres
   repository: file://../magda-postgres
-  version: 4.0.0
-digest: sha256:bf453298de85ee5e6b2e1097280bfd523ac38298cfd70c5ac840d8101a2164b1
-generated: "2024-05-24T06:16:18.709702767Z"
+  version: 4.2.1
+digest: sha256:a8e92bb58c251c44ee19fca501551c2bb662f3dc100a77b8bcb9f3a0e8532570
+generated: "2024-07-02T10:25:15.777009223Z"

deploy/helm/internal-charts/content-db/Chart.yaml

@@ -1,12 +1,12 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: content-db
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"
 dependencies:
   - name: magda-postgres
-    version: "4.0.0"
+    version: "4.2.1"
     repository: "file://../magda-postgres"
     condition: global.useInK8sDbInstance.content-db

deploy/helm/internal-charts/content-db/README.md

@@ -1,6 +1,6 @@
 # content-db
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 
@@ -10,7 +10,7 @@ Kubernetes: `>= 1.14.0-0`
 
 | Repository | Name | Version |
 |------------|------|---------|
-| file://../magda-postgres | magda-postgres | 4.0.0 |
+| file://../magda-postgres | magda-postgres | 4.2.1 |
 
 ## Values
 

deploy/helm/internal-charts/correspondence-api/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: correspondence-api
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"

deploy/helm/internal-charts/correspondence-api/README.md

@@ -1,6 +1,6 @@
 # correspondence-api
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 

deploy/helm/internal-charts/elasticsearch/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: elasticsearch
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"

deploy/helm/internal-charts/elasticsearch/README.md

@@ -1,6 +1,6 @@
 # elasticsearch
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 A Helm chart for Kubernetes
 

deploy/helm/internal-charts/gateway/Chart.yaml

@@ -2,7 +2,7 @@ apiVersion: v2
 description: The Gateway Component of Magda that routes incoming requets to
   other magda components.
 name: gateway
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.14.0-0"
 annotations:
   magdaModuleType: "core"

deploy/helm/internal-charts/gateway/README.md

@@ -1,6 +1,6 @@
 # gateway
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
 The Gateway Component of Magda that routes incoming requets to other magda components.
 
@@ -43,6 +43,7 @@ Kubernetes: `>= 1.14.0-0`
 | defaultWebRouteConfig.methods | list | `["GET"]` | array of string. "all" means all methods will be proxied  |
 | defaultWebRouteConfig.redirectTrailingSlash | bool | `false` | make /xxx auto redirect to /xxxx/ |
 | defaultWebRouteConfig.to | string | `""` | the default web router proxy target. Optional. If set, the default web route set via `web` option will be ignored. |
+| disableGzip | bool | `false` | By default, response will be auto-gizpped depends on MIME type. Set this to true to disable it. |
 | enableAuthEndpoint | bool | `true` | whether or not enable auth endpoint. You can turn it off if you don't need to log into any account. |
 | enableCkanRedirection | bool | `false` | wether or not enable CKAN redirection. when it's on, any incoming ckan alike URL will be redirected to the CKAN instance URL  that is specified by config option `ckanRedirectionDomain` and `ckanRedirectionPath`. |
 | enableHttpsRedirection | bool | `false` | whether or not redirect incoming request using HTTP protocol to HTTPs |

deploy/helm/internal-charts/gateway/templates/deployment.yaml

@@ -59,6 +59,9 @@ spec:
 {{- if .Values.enableCkanRedirection }}
             "--enableCkanRedirection", {{ .Values.enableCkanRedirection | quote }},
 {{- end }}
+{{- if .Values.disableGzip }}
+            "--disableGzip", {{ .Values.disableGzip | quote }},
+{{- end }}
 {{- if not (kindIs "invalid" .Values.registryQueryCacheStdTTL) }}
             "--registryQueryCacheStdTTL", {{ .Values.registryQueryCacheStdTTL | quote }},
 {{- end }}

deploy/helm/internal-charts/gateway/values.yaml

@@ -53,6 +53,9 @@ ckanRedirectionPath:
 # -- whether or not redirect incoming request using HTTP protocol to HTTPs
 enableHttpsRedirection: false
 
+# -- By default, response will be auto-gizpped depends on MIME type. Set this to true to disable it.
+disableGzip: false
+
 # -- a list of authentication plugin config item. 
 # Each authentication plugin config item can contain the following fields:
 # <ul>

deploy/helm/internal-charts/indexer/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
-description: A Helm chart for Kubernetes
+description: A Helm chart for Magda's Indexer service.
 name: indexer
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.21.0"
 annotations:
   magdaModuleType: "core"

deploy/helm/internal-charts/indexer/README.md

@@ -1,8 +1,14 @@
 # indexer
 
-![Version: 4.0.0](https://img.shields.io/badge/Version-4.0.0-informational?style=flat-square)
+![Version: 4.2.1](https://img.shields.io/badge/Version-4.2.1-informational?style=flat-square)
 
-A Helm chart for Kubernetes
+A Helm chart for Magda's Indexer service.
+
+The default config will pull region mapping files from [the `magda-region-mappings` repo release download area](https://github.com/magda-io/magda-regions/releases).
+
+For production deployment, you might want to host those region mapping files yourself in a more reliable way (e.g. put into a storage bucket).
+
+To config region mapping files, please refer to the this repo: https://github.com/magda-io/magda-regions
 
 ## Requirements
 
@@ -31,6 +37,3 @@ Kubernetes: `>= 1.21.0`
 | resources.limits.cpu | string | `"250m"` |  |
 | resources.requests.cpu | string | `"100m"` |  |
 | resources.requests.memory | string | `"250Mi"` |  |
-
-----------------------------------------------
-Autogenerated from chart metadata using [helm-docs v1.13.1](https://github.com/norwoodj/helm-docs/releases/v1.13.1)

deploy/helm/internal-charts/indexer/README.md.gotmpl

@@ -0,0 +1,24 @@
+{{ template "chart.header" . }}
+{{ template "chart.deprecationWarning" . }}
+
+{{ template "chart.versionBadge" . }}{{ template "chart.typeBadge" . }}{{ template "chart.appVersionBadge" . }}
+
+{{ template "chart.description" . }}
+
+The default config will pull region mapping files from [the `magda-region-mappings` repo release download area](https://github.com/magda-io/magda-regions/releases).
+
+For production deployment, you might want to host those region mapping files yourself in a more reliable way (e.g. put into a storage bucket).
+
+To config region mapping files, please refer to the this repo: https://github.com/magda-io/magda-regions
+
+{{ template "chart.homepageLine" . }}
+
+{{ template "chart.maintainersSection" . }}
+
+{{ template "chart.sourcesSection" . }}
+
+{{ template "chart.requirementsSection" . }}
+
+{{ template "chart.valuesHeader" . }}
+
+{{ template "chart.valuesTable" . }}

deploy/helm/internal-charts/ingress/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Kubernetes
 name: ingress
-version: 4.0.0
+version: 4.2.1
 kubeVersion: ">= 1.19.0-0"
 annotations:
   magdaModuleType: "core"