Align to GeoNode master (4.2.0) and improve/fix docker initialization (…

…#457) * - Align to GeoNode master (4.2.0) * - Align to GeoNode master (4.2.0) * - Align to GeoNode master (4.2.0) * - Align to GeoNode master (4.2.0) * - Typo * - Code formatting * - Optimizing and fixing docker stack; removing unneeded/unused variables (starting the process at least) * Update __init__.py * Typo on GEOSERVER_JAVA_OPTS * Update docker-compose-test django healthcheck with retries * drop PUBLIC_PORT and other unused Dockerfile env vars * disable abbreviated params * Removed legacy code * ported fixes to monitoring fixtures * Implement password configuration at init time * Removed Azure vars from .env sample * Add issing headers * - Fix: the oauth2 "logoutUri" should match the public url too * - Fix: update the "geofence-datasource-ovr.properties.j2" in order to use the correct variable for the "geofenceDataSource.username" * set nginx image version * Fixes CodeQL * Improvements to create-envfile documentation * Cumulative improvements * Give humane names to Geoserver variables * Use GEOSERVER_PUBLIC_LOCATION * Remove geoserver_ui variable from .env.sample * moved codeql-config to its folder to avoid trigger false workflow --------- Co-authored-by: Giovanni Allegri <[email protected]>
GeoNode · Aug 22, 2023 · c4430ba · c4430ba
1 parent 4e30faf
commit c4430ba
Show file tree

Hide file tree

Showing 53 changed files with 1,445 additions and 1,491 deletions.
diff --git a/.codeql/codeql-config.yml b/.codeql/codeql-config.yml
@@ -0,0 +1,4 @@
+languages: ${{ matrix.language }}
+
+paths-ignore: 
+  - src/project_name/__init__.py
diff --git a/.env.sample b/.env.sample
@@ -1,6 +1,4 @@
 COMPOSE_PROJECT_NAME={{project_name}}
-DOCKERHOST=
-DOCKER_HOST_IP=
 # See https://github.com/containers/podman/issues/13889
 # DOCKER_BUILDKIT=0
 DOCKER_ENV=production
@@ -13,7 +11,7 @@ C_FORCE_ROOT=1
 FORCE_REINIT=false
 INVOKE_LOG_STDOUT=true
 
-# LANGUAGE_CODE=pt
+# LANGUAGE_CODE=it-it
 # LANGUAGES=(('en-us','English'),('it-it','Italiano'))
 
 DJANGO_SETTINGS_MODULE={{project_name}}.settings
@@ -25,8 +23,10 @@ GEONODE_INSTANCE_NAME=geonode
 POSTGRES_USER=postgres
 POSTGRES_PASSWORD={pgpwd}
 GEONODE_DATABASE={{project_name}}
+GEONODE_DATABASE_USER={{project_name}}
 GEONODE_DATABASE_PASSWORD={dbpwd}
 GEONODE_GEODATABASE={{project_name}}_data
+GEONODE_GEODATABASE_USER={{project_name}}_data
 GEONODE_GEODATABASE_PASSWORD={geodbpwd}
 GEONODE_DATABASE_SCHEMA=public
 GEONODE_GEODATABASE_SCHEMA=public
@@ -43,7 +43,7 @@ ASYNC_SIGNALS=True
 
 SITEURL={siteurl}/
 
-ALLOWED_HOSTS="['django', '*', '{hostname}']"
+ALLOWED_HOSTS="['django', '{hostname}']"
 
 # Data Uploader
 DEFAULT_BACKEND_UPLOADER=geonode.importer
@@ -54,20 +54,13 @@ HAYSTACK_ENGINE_URL=http://elasticsearch:9200/
 HAYSTACK_ENGINE_INDEX_NAME=haystack
 HAYSTACK_SEARCH_RESULTS_PER_PAGE=200
 
-# #################
-# Jenkins
-# CI/CD Server
-# #################
-JENKINS_HTTP_PORT=9080
-JENKINS_HTTPS_PORT=9443
-
 # #################
 # nginx
 # HTTPD Server
 # #################
-GEONODE_LB_HOST_IP={hostname}
-GEONODE_LB_PORT=80
-PUBLIC_PORT={public_port}
+GEONODE_LB_HOST_IP=django
+GEONODE_LB_PORT=8000
+NGINX_BASE_URL={siteurl}
 
 # IP or domain name and port where the server can be reached on HTTPS (leave HOST empty if you want to use HTTP only)
 # port where the server can be reached on HTTPS
@@ -91,9 +84,11 @@ RESOLVER=127.0.0.11
 # #################
 # geoserver
 # #################
-GEOSERVER_WEB_UI_LOCATION={geoserver_ui}/geoserver/
-GEOSERVER_PUBLIC_LOCATION={geoserver_ui}/geoserver/
-GEOSERVER_LOCATION=http://geoserver:8080/geoserver/
+GEOSERVER_LB_HOST_IP=geoserver
+GEOSERVER_LB_PORT=8080
+GEOSERVER_WEB_UI_LOCATION={siteurl}/geoserver/
+GEOSERVER_PUBLIC_LOCATION={siteurl}/geoserver/
+GEOSERVER_LOCATION=http://${GEOSERVER_LB_HOST_IP}:${GEOSERVER_LB_PORT}/geoserver/
 GEOSERVER_ADMIN_USER=admin
 GEOSERVER_ADMIN_PASSWORD={geoserverpwd}
 
@@ -106,7 +101,7 @@ OGC_REQUEST_POOL_CONNECTIONS=10
 # Java Options & Memory
 ENABLE_JSONP=true
 outFormat=text/javascript
-GEOSERVER_JAVA_OPTS=-Djava.awt.headless=true -Xms2G -Xmx4G -Dgwc.context.suffix=gwc -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=/var/log/jvm.log -XX:PerfDataSamplingInterval=500 -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:-UseGCOverheadLimit -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -Duser.timezone=GMT -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL={geoserver_ui}/geoserver/pdf -DALLOW_ENV_PARAMETRIZATION=true -Xbootclasspath/a:/usr/local/tomcat/webapps/geoserver/WEB-INF/lib/marlin-0.9.3-Unsafe.jar -Dsun.java2d.renderer=org.marlin.pisces.MarlinRenderingEngine
+GEOSERVER_JAVA_OPTS='-Djava.awt.headless=true -Xms4G -Xmx4G -Dgwc.context.suffix=gwc -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=/var/log/jvm.log -XX:PerfDataSamplingInterval=500 -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:-UseGCOverheadLimit -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -Duser.timezone=GMT -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL={geoserver_ui}/geoserver/pdf -DALLOW_ENV_PARAMETRIZATION=true -Xbootclasspath/a:/usr/local/tomcat/webapps/geoserver/WEB-INF/lib/marlin-0.9.3-Unsafe.jar -Dsun.java2d.renderer=org.marlin.pisces.MarlinRenderingEngine'
 
 # #################
 # Security
@@ -152,6 +147,8 @@ ACCOUNT_EMAIL_REQUIRED=True
 ACCOUNT_APPROVAL_REQUIRED=False
 ACCOUNT_CONFIRM_EMAIL_ON_GET=False
 ACCOUNT_EMAIL_VERIFICATION=none
+ACCOUNT_EMAIL_CONFIRMATION_EMAIL=False
+ACCOUNT_EMAIL_CONFIRMATION_REQUIRED=False
 ACCOUNT_AUTHENTICATION_METHOD=username_email
 AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME=True
 
@@ -170,7 +167,7 @@ TASTYPIE_APIKEY=
 # #################
 DEBUG={debug}
 
-SECRET_KEY="{secret_key}"
+SECRET_KEY='{secret_key}'
 
 STATIC_ROOT=/mnt/volumes/statics/static/
 MEDIA_ROOT=/mnt/volumes/statics/uploaded/
@@ -184,7 +181,7 @@ MEMCACHED_LOCATION=127.0.0.1:11211
 MEMCACHED_LOCK_EXPIRE=3600
 MEMCACHED_LOCK_TIMEOUT=10
 
-MAX_DOCUMENT_SIZE=2
+MAX_DOCUMENT_SIZE=200
 CLIENT_RESULTS_LIMIT=5
 API_LIMIT_PER_PAGE=1000
 

diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
@@ -2,11 +2,11 @@ name: "CodeQL"
 
 on:
   push:
-    branches: [ "master", "3.3.x", "4.x", "4.0.x" ]
+    branches: [ "master", 4.1.x ]
   pull_request:
-    branches: [ "master" ]
+    branches: [ "master", 4.1.x ]
   schedule:
-    - cron: "53 23 * * 3"
+    - cron: '38 4 * * 5'
 
 jobs:
   analyze:
@@ -20,23 +20,16 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        language: [ javascript, python ]
+        language: [ 'javascript', 'python' ]
 
     steps:
-      - name: Checkout
-        uses: actions/checkout@v3
+    - name: Checkout repository
+      uses: actions/checkout@v3
 
-      - name: Initialize CodeQL
-        uses: github/codeql-action/init@v2
-        with:
-          languages: ${{ matrix.language }}
-          queries: +security-and-quality
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v2
+      with:
+        config-file: .codeql/codeql-config.yml
 
-      - name: Autobuild
-        uses: github/codeql-action/autobuild@v2
-        if: ${{ matrix.language == 'javascript' || matrix.language == 'python' }}
-
-      - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@v2
-        with:
-          category: "/language:${{ matrix.language }}"
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v2
diff --git a/.override_dev_env.sample b/.override_dev_env.sample
@@ -62,6 +62,8 @@ export ACCOUNT_EMAIL_REQUIRED=True
 export ACCOUNT_APPROVAL_REQUIRED=False
 export ACCOUNT_CONFIRM_EMAIL_ON_GET=False
 export ACCOUNT_EMAIL_VERIFICATION=none
+export ACCOUNT_EMAIL_CONFIRMATION_EMAIL=False
+export ACCOUNT_EMAIL_CONFIRMATION_REQUIRED=False
 export ACCOUNT_AUTHENTICATION_METHOD=username_email
 
 export OAUTH2_API_KEY=

diff --git a/Dockerfile b/Dockerfile
@@ -1,54 +1,17 @@
-FROM ubuntu:22.04
+FROM geonode/geonode-base:latest-ubuntu-22.10
 LABEL GeoNode development team
 
 RUN mkdir -p /usr/src/{{project_name}}
 
-## Enable postgresql-client-13
-RUN apt-get update -y && apt-get install curl wget unzip gnupg2 -y
-RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
-# will install python3.10 
-RUN apt-get install lsb-core -y
-RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" |tee  /etc/apt/sources.list.d/pgdg.list
-# This section is borrowed from the official Django image but adds GDAL and others
-RUN apt-get update -y && apt-get upgrade -y
-
-# Prepraing dependencies
-RUN apt-get install -y \
-    libgdal-dev libpq-dev libxml2-dev \
-    libxml2 libxslt1-dev zlib1g-dev libjpeg-dev \
-    libmemcached-dev libldap2-dev libsasl2-dev libffi-dev
-
-RUN apt-get update -y && apt-get install -y --no-install-recommends \
-    gcc zip gettext geoip-bin cron \
-    postgresql-client-13 \
-    python3-all-dev python3-dev \
-    python3-gdal python3-psycopg2 python3-ldap \
-    python3-pip python3-pil python3-lxml \
-    uwsgi uwsgi-plugin-python3 python3-gdbm python-is-python3 gdal-bin
-
-RUN apt-get install -y devscripts build-essential debhelper pkg-kde-tools sharutils
-# RUN git clone https://salsa.debian.org/debian-gis-team/proj.git /tmp/proj
-# RUN cd /tmp/proj && debuild -i -us -uc -b && dpkg -i ../*.deb
-
-# Install pip packages
-RUN pip install pip --upgrade \
-    && pip install pygdal==$(gdal-config --version).* \
-        flower==0.9.4
-
-# Activate "memcached"
-RUN apt install -y memcached
-RUN pip install pylibmc \
-    && pip install sherlock
-
 # add bower and grunt command
 COPY src /usr/src/{{project_name}}/
 WORKDIR /usr/src/{{project_name}}
 
-COPY src/monitoring-cron /etc/cron.d/monitoring-cron
-RUN chmod 0644 /etc/cron.d/monitoring-cron
-RUN crontab /etc/cron.d/monitoring-cron
-RUN touch /var/log/cron.log
-RUN service cron start
+#COPY src/monitoring-cron /etc/cron.d/monitoring-cron
+#RUN chmod 0644 /etc/cron.d/monitoring-cron
+#RUN crontab /etc/cron.d/monitoring-cron
+#RUN touch /var/log/cron.log
+#RUN service cron start
 
 COPY src/wait-for-databases.sh /usr/bin/wait-for-databases
 RUN chmod +x /usr/bin/wait-for-databases
@@ -67,11 +30,13 @@ RUN chmod +x /usr/bin/celery-cmd
 # RUN cd /usr/src/geonode-contribs/geonode-logstash; pip install --upgrade  -e . \
 #     cd /usr/src/geonode-contribs/ldap; pip install --upgrade  -e .
 
-RUN pip install --upgrade --no-cache-dir  --src /usr/src -r requirements.txt
-RUN pip install --upgrade  -e .
+RUN yes w | pip install --src /usr/src -Ur requirements.txt &&\
+    yes w | pip install --upgrade -e .
 
 # Cleanup apt update lists
-RUN rm -rf /var/lib/apt/lists/*
+RUN apt-get autoremove --purge &&\
+    apt-get clean &&\
+    rm -rf /var/lib/apt/lists/*
 
 # Export ports
 EXPOSE 8000

diff --git a/README.md b/README.md
@@ -4,7 +4,8 @@ GeoNode template project. Generates a django project with GeoNode support.
 
 ## Table of Contents
 
--  [Developer Workshop](#developer-Workshop)
+-  [Quick Docker Start](#quick-docker-start)
+-  [Developer Workshop](#developer-workshop)
 -  [Create a custom project](#create-a-custom-project)
 -  [Start your server using Docker](#start-your-server-using-docker)
 -  [Run the instance in development mode](#run-the-instance-in-development-mode)
@@ -14,6 +15,48 @@ GeoNode template project. Generates a django project with GeoNode support.
 -  [Recommended: Track your changes](#recommended-track-your-changes)
 -  [Hints: Configuring `requirements.txt`](#hints-configuring-requirementstxt)
 
+## Quick Docker Start
+
+  ```bash
+    python3.10 -m venv ~/.venvs/project_name
+    source ~/.venvs/{{ project_name }}/bin/activate
+
+    pip install Django==3.2.*
+
+    mkdir ~/project_name
+  ```
+
+  ```bash
+    GN_VERSION=master
+
+    django-admin startproject --template=https://github.com/GeoNode/geonode-project/archive/refs/tags/$GN_VERSION.zip -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile project_name ~/project_name
+  ```
+
+  ```bash
+    cd ~/project_name
+    python create-envfile.py 
+  ```
+`create-envfile.py` accepts the following arguments:
+
+- `--https`: Enable SSL. It's disabled by default
+- `--env_type`: 
+   - When set to `prod` `DEBUG` is disabled and the creation of a valid `SSL` is requested to Letsencrypt's ACME server
+   - When set to `test`  `DEBUG` is disabled and a test `SSL` certificate is generated for local testing
+   - When set to `dev`  `DEBUG` is enabled and no `SSL` certificate is generated
+- `--hostname`: The URL that whill serve GeoNode (`localhost` by default)
+- `--email`: The administrator's email. Notice that a real email and a valid SMPT configurations are required if  `--env_type` is seto to `prod`. Letsencrypt uses to email for issuing the SSL certificate 
+- `--geonodepwd`: GeoNode's administrator password. A random value is set if left empty
+- `--geoserverpwd`: GeoNode's administrator password. A random value is set if left empty
+- `--pgpwd`: PostgreSQL's administrator password. A random value is set if left empty
+- `--dbpwd`: GeoNode DB user role's password. A random value is set if left empty
+- `--geodbpwd`: GeoNode data DB user role's password. A random value is set if left empty
+- `--clientid`: Client id of Geoserver's GeoNode Oauth2 client. A random value is set if left empty
+- `--clientsecret`: Client secret of Geoserver's GeoNode Oauth2 client. A random value is set if left empty
+```bash
+  docker compose build
+  docker compose up -d
+```
+
 ## Developer Workshop
 
 Available at