Geoserver: update to latest version 2.22.2 to get vulnerability fix (#307)

tlvu · web-flow · commit 60116e2c90c2 · 2023-03-31T23:46:46.000-04:00
Geoserver: update to latest version 2.22.2 to get vulnerability fix For vulnerability in `jt-jiffle` < 1.1.22, see https://nvd.nist.gov/vuln/detail/CVE-2022-24816, and GHSA-v92f-jx6p-73rx. Changed to use the CORS (Cross-Origin Resource Sharing) default config from the image instead of our own. Both are quite similar so if we can use the default config, future upgrade will be simpler. New Geoserver version will have `jt-jiffle` 1.1.24. The old one had version 1.1.20. ``` $ docker run -it --rm --entrypoint bash pavics/geoserver:2.22.2-kartoza-build20230226-r5-allow-change-context-root-and-fix-missing-stable-plugins _ __ _ ____ _ ____ ____ | |/ /__ _ _ __| |_ ___ ______ _ | _ \ ___ ___| | _____ _ __ / ___| ___ ___/ ___| ___ _ ____ _____ _ __ | ' // _` | '__| __/ _ \_ / _` | | | | |/ _ \ / __| |/ / _ \ '__| | | _ / _ \/ _ \___ \ / _ \ '__\ \ / / _ \ '__| | . \ (_| | | | || (_) / / (_| | | |_| | (_) | (__| < __/ | | |_| | __/ (_) |__) | __/ | \ V / __/ | |_|\_\__,_|_| \__\___/___\__,_| |____/ \___/ \___|_|\_\___|_| \____|\___|\___/____/ \___|_| \_/ \___|_| root@c3787dccea2d:/geoserver# find / -iname '**jt-jiffle**' /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/jt-jiffle-language-1.1.24.jar /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/jt-jiffle-op-1.1.24.jar root@c3787dccea2d:/geoserver# ``` Used our own custom build image because the original kartoza image is missing 2 plugins that we use, see kartoza/docker-geoserver#508 and to avoid excessively slow startup due to kartoza/docker-geoserver#515. CORS config difference: ```diff --- web.xml.old 2023-03-22 16:10:20.000000000 -0400 +++ web.xml.new 2023-03-22 16:10:06.000000000 -0400 <filter> <filter-name>CorsFilter</filter-name> <filter-class>org.apache.catalina.filters.CorsFilter</filter-class> <init-param> - <param-name>cors.allowed.methods</param-name> - <param-value>GET,POST,HEAD,OPTIONS,PUT</param-value> - </init-param> - <init-param> <param-name>cors.allowed.origins</param-name> <param-value>*</param-value> </init-param> <init-param> <param-name>cors.allowed.headers</param-name> - <param-value>Content-Type,X-Requested-With,accept,Origin,Access-Control-Request-Method,Access-Control-Request-Headers,Authorization,Authentication</param-value> + <param-value>Content-Type,X-Requested-With,accept,Access-Control-Request-Method,Access-Control-Request-Headers,If-Modified-Since,Range,Origin,Authorization</param-value> + </init-param> + <init-param> + <param-name>cors.exposed.headers</param-name> + <param-value>Access-Control-Allow-Origin,Access-Control-Allow-Credentials</param-value> </init-param> </filter> ``` Missing `cors.allowed.methods`, new `cors.exposed.headers`. For `cors.allowed.headers`, missing `Authentication`, new `If-Modified-Since,Range`. Hopefully everything still works with the new CORS config and future upgrade will be simpler. The new version is able to start now. Syncing production data to vm VM to test for upgrade problem. Last upgrade there was some problem with the existing data. I highly suggest all organizations to test with their existing data before going live. Tested with the following notebooks, hopefully CORS changes are effectively tested there: * https://github.com/Ouranosinc/pavics-sdi/blob/f4aecf64889f0c8503ea67b59b6558ae18407cf6/docs/source/notebooks/WFS_example.ipynb * https://github.com/Ouranosinc/pavics-sdi/blob/f4aecf64889f0c8503ea67b59b6558ae18407cf6/docs/source/notebooks/regridding.ipynb * https://github.com/bird-house/finch/blob/877312d325d4de5c3efcb4f1f75fbe5cd22660d6/docs/source/notebooks/subset.ipynb * https://github.com/Ouranosinc/raven/blob/0be6d77d71bcaf4546de97b13bafc6724068a73d/docs/source/notebooks/01_Getting_watershed_boundaries.ipynb with `RAVEN_GEO_URL` pointing to another Geoserver (also from this PR) to test CORS (Cross-Origin Resource Sharing) ## Other changes - Raven: allow to customize the Geoserver it will use Useful to test the local Geoserver or to have your own Geoserver with your own data. Default to PAVICS Geoserver. Set `RAVEN_GEO_URL` in `env.local` to something like `https://host/geoserver/`. - env.local.example: change default Geoserver admin user from 'admin' to 'admingeo' This only impacts new deployment when `env.local.example` is instanciated to `env.local`. This is to avoid confusion with the admin user of Magpie, which is also 'admin'.
diff --git a/.bumpversion.cfg b/.bumpversion.cfg
@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.24.1
+current_version = 1.25.0
 commit = True
 tag = False
 tag_name = {new_version}
@@ -30,11 +30,11 @@ search = {current_version}
 replace = {new_version}
 
 [bumpversion:file:RELEASE.txt]
-search = {current_version} 2023-03-27T16:12:50Z
+search = {current_version} 2023-04-01T03:45:16Z
 replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ}
 
 [bumpversion:part:releaseTime]
-values = 2023-03-27T16:12:50Z
+values = 2023-04-01T03:45:16Z
 
 [bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template]
 search = 'version': '{current_version}'
diff --git a/CHANGES.md b/CHANGES.md
@@ -16,6 +16,94 @@
 
 [//]: # (list changes here, using '-' for each new entry, remove this when items are added)
 
+[1.25.0](https://github.com/bird-house/birdhouse-deploy/tree/1.25.0) (2023-04-01)
+------------------------------------------------------------------------------------------------------------------
+
+## Fixes
+- Geoserver: update to latest version 2.22.2 to get vulnerability fix
+
+  For vulnerability in `jt-jiffle` < 1.1.22, see
+  https://nvd.nist.gov/vuln/detail/CVE-2022-24816, and
+  https://github.com/geosolutions-it/jai-ext/security/advisories/GHSA-v92f-jx6p-73rx.
+
+  Changed to use the CORS (Cross-Origin Resource Sharing) default config from
+  the image instead of our own.  Both are quite similar so if we can use the
+  default config, future upgrade will be simpler.
+
+  New Geoserver version will have `jt-jiffle` 1.1.24.  The old one had version 1.1.20.
+  ```
+  $ docker run -it --rm --entrypoint bash pavics/geoserver:2.22.2-kartoza-build20230226-r5-allow-change-context-root-and-fix-missing-stable-plugins
+
+  | |/ /__ _ _ __| |_ ___ ______ _  |  _ \  ___   ___| | _____ _ __   / ___| ___  ___/ ___|  ___ _ ____   _____ _ __
+  | ' // _` | '__| __/ _ \_  / _` | | | | |/ _ \ / __| |/ / _ \ '__| | |  _ / _ \/ _ \___ \ / _ \ '__\ \ / / _ \ '__|
+  | . \ (_| | |  | || (_) / / (_| | | |_| | (_) | (__|   <  __/ |    | |_| |  __/ (_) |__) |  __/ |   \ V /  __/ |
+  |_|\_\__,_|_|   \__\___/___\__,_| |____/ \___/ \___|_|\_\___|_|     \____|\___|\___/____/ \___|_|    \_/ \___|_|
+
+  root@c3787dccea2d:/geoserver# find / -iname '**jt-jiffle**'
+  /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/jt-jiffle-language-1.1.24.jar
+  /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/jt-jiffle-op-1.1.24.jar
+  root@c3787dccea2d:/geoserver#
+  ```
+
+  Used our own custom build image because the original kartoza image is missing 2 plugins that we use, see https://github.com/kartoza/docker-geoserver/issues/508 and to avoid excessively slow startup due to https://github.com/kartoza/docker-geoserver/issues/515.
+
+  CORS config difference:
+  ```diff
+  --- web.xml.old 2023-03-22 16:10:20.000000000 -0400
+  +++ web.xml.new 2023-03-22 16:10:06.000000000 -0400
+
+       <filter>
+           <filter-name>CorsFilter</filter-name>
+           <filter-class>org.apache.catalina.filters.CorsFilter</filter-class>
+           <init-param>
+  -            <param-name>cors.allowed.methods</param-name>
+  -            <param-value>GET,POST,HEAD,OPTIONS,PUT</param-value>
+  -        </init-param>
+  -        <init-param>
+               <param-name>cors.allowed.origins</param-name>
+               <param-value>*</param-value>
+           </init-param>
+           <init-param>
+               <param-name>cors.allowed.headers</param-name>
+  -            <param-value>Content-Type,X-Requested-With,accept,Origin,Access-Control-Request-Method,Access-Control-Request-Headers,Authorization,Authentication</param-value>
+  +            <param-value>Content-Type,X-Requested-With,accept,Access-Control-Request-Method,Access-Control-Request-Headers,If-Modified-Since,Range,Origin,Authorization</param-value>
+  +        </init-param>
+  +        <init-param>
+  +            <param-name>cors.exposed.headers</param-name>
+  +            <param-value>Access-Control-Allow-Origin,Access-Control-Allow-Credentials</param-value>
+           </init-param>
+       </filter>
+  ```
+  Missing `cors.allowed.methods`, new `cors.exposed.headers`.
+
+  For `cors.allowed.headers`, missing `Authentication`, new `If-Modified-Since,Range`.
+
+  Hopefully everything still works with the new CORS config and future upgrade will be simpler.
+
+  Tested with the following notebooks, hopefully CORS changes are effectively tested there:
+  * https://github.com/Ouranosinc/pavics-sdi/blob/f4aecf64889f0c8503ea67b59b6558ae18407cf6/docs/source/notebooks/WFS_example.ipynb
+  * https://github.com/Ouranosinc/pavics-sdi/blob/f4aecf64889f0c8503ea67b59b6558ae18407cf6/docs/source/notebooks/regridding.ipynb
+  * https://github.com/bird-house/finch/blob/877312d325d4de5c3efcb4f1f75fbe5cd22660d6/docs/source/notebooks/subset.ipynb
+  * https://github.com/Ouranosinc/raven/blob/0be6d77d71bcaf4546de97b13bafc6724068a73d/docs/source/notebooks/01_Getting_watershed_boundaries.ipynb
+    with `RAVEN_GEO_URL` pointing to another Geoserver (also from this PR) to
+    test CORS (Cross-Origin Resource Sharing)
+
+## Changes
+- Raven: allow to customize the Geoserver it will use
+
+  Useful to test the local Geoserver or to have your own Geoserver with your
+  own data.  Default to PAVICS Geoserver.
+
+  Set `RAVEN_GEO_URL` in `env.local` to something like `https://host/geoserver/`.
+
+- env.local.example: change default Geoserver admin user from 'admin' to 'admingeo'
+
+  This only impacts new deployment when `env.local.example` is instanciated
+  to `env.local`.
+
+  This is to avoid confusion with the admin user of Magpie, which is also 'admin'.
+
+
 [1.24.1](https://github.com/bird-house/birdhouse-deploy/tree/1.24.1) (2023-03-27)
 ------------------------------------------------------------------------------------------------------------------
 
@@ -32,7 +120,9 @@
 [1.24.0](https://github.com/bird-house/birdhouse-deploy/tree/1.24.0) (2023-03-22)
 ------------------------------------------------------------------------------------------------------------------
 ## Fixes
-- The default stack was not configurable. This meant that if someone wanted to deploy a 
+- Make all components pluggable
+
+  The default stack was not configurable. This meant that if someone wanted to deploy a
   subset of the default stack there was no good way of configuring birdhouse-deploy to run
   this subset only. 
 
diff --git a/Makefile b/Makefile
@@ -1,7 +1,7 @@
 # Generic variables
 override SHELL       := bash
 override APP_NAME    := birdhouse-deploy
-override APP_VERSION := 1.24.1
+override APP_VERSION := 1.25.0
 
 # utility to remove comments after value of an option variable
 override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")
diff --git a/README.rst b/README.rst
@@ -14,13 +14,13 @@ for a full-fledged production platform.
     * - releases
       - | |latest-version| |commits-since|
 
-.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.24.1.svg
+.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.25.0.svg
     :alt: Commits since latest release
-    :target: https://github.com/bird-house/birdhouse-deploy/compare/1.24.1...master
+    :target: https://github.com/bird-house/birdhouse-deploy/compare/1.25.0...master
 
-.. |latest-version| image:: https://img.shields.io/badge/tag-1.24.1-blue.svg?style=flat
+.. |latest-version| image:: https://img.shields.io/badge/tag-1.25.0-blue.svg?style=flat
     :alt: Latest Tag
-    :target: https://github.com/bird-house/birdhouse-deploy/tree/1.24.1
+    :target: https://github.com/bird-house/birdhouse-deploy/tree/1.25.0
 
 .. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
     :alt: ReadTheDocs Build Status (latest version)
diff --git a/RELEASE.txt b/RELEASE.txt
@@ -1 +1 @@
-1.24.1 2023-03-27T16:12:50Z
+1.25.0 2023-04-01T03:45:16Z
diff --git a/birdhouse/config/canarie-api/docker_configuration.py.template b/birdhouse/config/canarie-api/docker_configuration.py.template
@@ -17,8 +17,8 @@ SERVICES = {
         'info': {
             'name': 'Node',
             'synopsis': 'Nodes are data, compute and index endpoints accessed through the PAVICS platform or external clients. The Node service is the backend that allows: data storage, harvesting, indexation and discovery of local and federated data; authentication and authorization; server registration and management. Node service is therefore composed of several other services.',
-            'version': '1.24.1',
-            'releaseTime': '2023-03-27T16:12:50Z',
+            'version': '1.25.0',
+            'releaseTime': '2023-04-01T03:45:16Z',
             'institution': 'Ouranos',
             'researchSubject': 'Climatology',
             'supportEmail': '${SUPPORT_EMAIL}',
@@ -47,8 +47,8 @@ PLATFORMS = {
         'info': {
             'name': 'PAVICS',
             'synopsis': 'The PAVICS (Power Analytics for Visualization of Climate Science) platform is a collection of climate analysis services served through Open Geospatial Consortium (OGC) protocols. These services include data access, processing and visualization. Both data and algorithms can be accessed either programmatically, through OGC-compliant clients such as QGIS or ArcGIS, or a custom web interface.',
-            'version': '1.24.1',
-            'releaseTime': '2023-03-27T16:12:50Z',
+            'version': '1.25.0',
+            'releaseTime': '2023-04-01T03:45:16Z',
             'institution': 'Ouranos',
             'researchSubject': 'Climatology',
             'supportEmail': '${SUPPORT_EMAIL}',
diff --git a/birdhouse/config/geoserver/default.env b/birdhouse/config/geoserver/default.env
@@ -7,17 +7,17 @@
 # Cache kartoza/geoserver docker build on pavics org since their tags are
 # "moving" tags, meaning not reproducible behavior !
 # See https://github.com/kartoza/docker-geoserver/issues/232#issuecomment-808754831
-export GEOSERVER_IMAGE="pavics/geoserver:2.19.0-kartoza-build20210329-r2-with-snakeyaml"
+export GEOSERVER_IMAGE="pavics/geoserver:2.22.2-kartoza-build20230226-r7-allow-change-context-root-and-fix-missing-stable-plugins-and-avoid-chown-datadir"
 
 export GEOSERVER_ADMIN_USER="admin"
 
 # # Install the stable plugin specified in
 # https://github.com/kartoza/docker-geoserver/blob/master/build_data/stable_plugins.txt
-export GEOSERVER_STABLE_EXTENSIONS="grib-plugin,netcdf-plugin,netcdf-out-plugin"
+export GEOSERVER_STABLE_EXTENSIONS="grib-plugin,netcdf-plugin,netcdf-out-plugin,csw-iso-plugin,metadata-plugin"
 
 # Install the community edition plugins specified in
 # https://github.com/kartoza/docker-geoserver/blob/master/build_data/community_plugins.txt
-export GEOSERVER_COMMUNITY_EXTENSIONS="csw-iso-plugin,geopkg-plugin,metadata-plugin"
+export GEOSERVER_COMMUNITY_EXTENSIONS="geopkg-plugin"
 
 # Must use single-quote for delayed eval.
 export GEOSERVER_DATA_DIR='${DATA_PERSIST_ROOT}/geoserver'
diff --git a/birdhouse/config/geoserver/docker-compose-extra.yml b/birdhouse/config/geoserver/docker-compose-extra.yml
@@ -14,23 +14,22 @@ services:
     ports:
       - "8087:8080"
     environment:
-      # for custom entrypoint
-      PAVICS_FQDN_PUBLIC: ${PAVICS_FQDN_PUBLIC}
       STABLE_EXTENSIONS: ${GEOSERVER_STABLE_EXTENSIONS}
       COMMUNITY_EXTENSIONS: ${GEOSERVER_COMMUNITY_EXTENSIONS}
       GEOSERVER_ADMIN_USER: ${GEOSERVER_ADMIN_USER}
       GEOSERVER_ADMIN_PASSWORD: ${GEOSERVER_ADMIN_PASSWORD}
       # fix "WARNING: Illegal reflective access by org.geotools.image.ImageWorker" in `docker logs geoserver`
       JAVA_OPTS: --add-exports=java.desktop/com.sun.imageio.plugins.jpeg=ALL-UNNAMED
       MAXIMUM_MEMORY: 8G
+      # https://github.com/kartoza/docker-geoserver#proxy-base-url
+      HTTP_PROXY_NAME: ${PAVICS_FQDN_PUBLIC}
+      HTTP_SCHEME: https
     volumes:
       # run deployment/fix-geoserver-data-dir-perm on existing
       # GEOSERVER_DATA_DIR to match user geoserveruser inside docker image
       - ${GEOSERVER_DATA_DIR}:/opt/geoserver/data_dir
-      - ./config/geoserver/entrypointwrapper:/entrypointwrapper:ro
     links:
       - postgis
-    entrypoint: /entrypointwrapper
     restart: always
     logging: *default-logging
     healthcheck:
diff --git a/birdhouse/config/geoserver/entrypointwrapper b/birdhouse/config/geoserver/entrypointwrapper
diff --git a/birdhouse/config/raven/default.env b/birdhouse/config/raven/default.env
@@ -1,3 +1,11 @@
+# The Geoserver that Raven will connect to.
+# Same default value as
+# https://github.com/CSHS-CWRA/RavenPy/blob/2e56041b605e83ab28ffdc5d817e645481dcc5fc/ravenpy/utilities/geoserver.py#L51
+# This is the production Geoserver that is always available with appropriate data.
+# For site that want to run your own Geoserver with your own data, please
+# override this variable with your own Geoserver instance.
+# Ex: RAVEN_GEO_URL="https://${PAVICS_FQDN}/geoserver/"
+export RAVEN_GEO_URL="https://pavics.ouranos.ca/geoserver/"
 
 # When canarie-api is monitoring twitcher, at least one WPS service protected by twitcher
 # needs to be available. This sets this component as the service used to monitor twitcher.
diff --git a/birdhouse/config/raven/docker-compose-extra.yml b/birdhouse/config/raven/docker-compose-extra.yml
@@ -15,6 +15,7 @@ services:
       - "8096:9099"
     environment:
       PYWPS_CFG: /wps.cfg
+      GEO_URL: "${RAVEN_GEO_URL}"
     volumes:
       - ./config/raven/wps.cfg:/wps.cfg
       - /tmp
diff --git a/birdhouse/env.local.example b/birdhouse/env.local.example
@@ -30,7 +30,7 @@ export POSTGRES_PAVICS_USERNAME=postgres-pavics
 export POSTGRES_PAVICS_PASSWORD=postgres-qwerty
 export POSTGRES_MAGPIE_USERNAME=postgres-magpie
 export POSTGRES_MAGPIE_PASSWORD=postgres-qwerty
-export GEOSERVER_ADMIN_USER=admin
+export GEOSERVER_ADMIN_USER=admingeo
 export GEOSERVER_ADMIN_PASSWORD=geoserverpass
 
 #############################################################################
@@ -442,6 +442,11 @@ export GEOSERVER_ADMIN_PASSWORD=geoserverpass
 #export JUPYTER_LOGIN_BANNER_TOP_SECTION=""
 #export JUPYTER_LOGIN_BANNER_BOTTOM_SECTION=""
 
+# Raven to use the local Geoserver instead of the default production.
+# See raven/default.env for more info.
+#export RAVEN_GEO_URL="https://${PAVICS_FQDN}/geoserver/"
+
+
 #############################################################################
 # Monitoring components configs
 #############################################################################
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -69,9 +69,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.24.1'
+version = '1.25.0'
 # The full version, including alpha/beta/rc tags.
-release = '1.24.1'
+release = '1.25.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

Original file line number	Diff line number	Diff line change
`@@ -1 +1 @@`
`1`		`-1.24.1 2023-03-27T16:12:50Z`
	`1`	`+1.25.0 2023-04-01T03:45:16Z`