Incompatibility with Shapely >2 Due to NumPy 2.0 Update #330
Description
We have identified a compatibility issue with ckanext-spatial when using Shapely versions higher than 2 (see Error logs), due to the recent release of NumPy 2.0. This issue arises because NumPy 2.0 introduces changes to the C ABI that require rebuilding packages that use the NumPy C API, such as Shapely, to import it, so it is recommended to add this upper pin now for releases.
Refs
- NumPy v2.0.0 has just been released.
- Compatibility with NumPy 2.0 shapely/shapely#1972
- Ecosystem compatibility with numpy 2.0 numpy/numpy#26191
Todo
To address the compatibility issues between ckanext-spatial and the recent release of NumPy 2.0, we propose the following modifications to the requirements file:
- We added a pin for NumPy to ensure compatibility:
numpy>=1.26,<2: This ensures that the version of NumPy used is compatible with both current and upcoming changes, but avoids the breaking changes introduced in NumPy 2.0.
- Shapely version is set to
2.0.4, which is confirmed to be compatible with the specified NumPy range.
ckantoolkit
lxml>=2.3
argparse
pyparsing>=2.1.10
requests>=1.1.0
cython==0.29.36; python_version < '3.9'
pyproj==2.6.1; python_version < '3.9'
pyproj==3.6.1; python_version >= '3.9'
# Compatibility with NumPy 2.0: https://github.com/shapely/shapely/issues/1972
numpy>=1.26,<2
Shapely==2.0.4
OWSLib==0.28.1
geojson==3.0.1Error logs
2024-06-17 07:49:32,637 INFO [ckan.cli] Using configuration file /srv/app/ckan.ini
2024-06-17 07:49:32,638 INFO [ckan.config.environment] Loading static files from public
2024-06-17 07:49:32,808 WARNI [ckan.config.environment] Setting CKAN_DB as an env var is deprecated and will be removed in a future release. Use CKAN_SQLALCHEMY_URL instead.
2024-06-17 07:49:32,843 INFO [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.
Traceback (most recent call last): File "/usr/bin/ckan", line 8, in <module>
sys.exit(ckan())
File "/usr/lib/python3.9/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.9/site-packages/click/core.py", line 781, in main
with self.make_context(prog_name, args, **extra) as ctx:
File "/usr/lib/python3.9/site-packages/click/core.py", line 700, in make_context
self.parse_args(ctx, args)
File "/srv/app/src/ckan/ckan/cli/cli.py", line 116, in parse_args
result = super(ExtendableGroup, self).parse_args(ctx, args)
File "/usr/lib/python3.9/site-packages/click/core.py", line 1212, in parse_args
rest = Command.parse_args(self, ctx, args)
File "/usr/lib/python3.9/site-packages/click/core.py", line 1048, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
File "/usr/lib/python3.9/site-packages/click/core.py", line 1630, in handle_parse_result
value = invoke_param_callback(self.callback, ctx, self, value)
File "/usr/lib/python3.9/site-packages/click/core.py", line 123, in invoke_param_callback
return callback(ctx, param, value)
File "/srv/app/src/ckan/ckan/cli/cli.py", line 126, in _init_ckan_config
_add_ctx_object(ctx, value)
File "/srv/app/src/ckan/ckan/cli/cli.py", line 135, in _add_ctx_object
ctx.obj = CtxObject(path)
File "/srv/app/src/ckan/ckan/cli/cli.py", line 57, in __init__
self.app = make_app(self.config)
File "/srv/app/src/ckan/ckan/config/middleware/__init__.py", line 56, in make_app
load_environment(conf)
File "/srv/app/src/ckan/ckan/config/environment.py", line 123, in load_environment
p.load_all()
File "/srv/app/src/ckan/ckan/plugins/core.py", line 165, in load_all
load(*plugins)
File "/srv/app/src/ckan/ckan/plugins/core.py", line 179, in load
service = _get_service(plugin)
File "/srv/app/src/ckan/ckan/plugins/core.py", line 281, in _get_service
return plugin.load()(name=plugin_name)
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2450, in load
return self.resolve()
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/srv/app/src/ckanext-spatial/ckanext/spatial/plugin/__init__.py", line 7, in <module>
import shapely.geometry
File "/usr/lib/python3.9/site-packages/shapely/__init__.py", line 1, in <module>
from .lib import GEOSException # NOQA
AttributeError: _ARRAY_API not found
Traceback (most recent call last):
File "/usr/bin/ckan", line 8, in <module>
sys.exit(ckan())
File "/usr/lib/python3.9/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.9/site-packages/click/core.py", line 781, in main
with self.make_context(prog_name, args, **extra) as ctx:
File "/usr/lib/python3.9/site-packages/click/core.py", line 700, in make_context
self.parse_args(ctx, args)
File "/srv/app/src/ckan/ckan/cli/cli.py", line 116, in parse_args
result = super(ExtendableGroup, self).parse_args(ctx, args)
File "/usr/lib/python3.9/site-packages/click/core.py", line 1212, in parse_args
rest = Command.parse_args(self, ctx, args)
File "/usr/lib/python3.9/site-packages/click/core.py", line 1048, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
File "/usr/lib/python3.9/site-packages/click/core.py", line 1630, in handle_parse_result
value = invoke_param_callback(self.callback, ctx, self, value)
File "/usr/lib/python3.9/site-packages/click/core.py", line 123, in invoke_param_callback
return callback(ctx, param, value)
File "/srv/app/src/ckan/ckan/cli/cli.py", line 126, in _init_ckan_config
_add_ctx_object(ctx, value)
File "/srv/app/src/ckan/ckan/cli/cli.py", line 135, in _add_ctx_object
ctx.obj = CtxObject(path)
File "/srv/app/src/ckan/ckan/cli/cli.py", line 57, in __init__
self.app = make_app(self.config)
File "/srv/app/src/ckan/ckan/config/middleware/__init__.py", line 56, in make_app
load_environment(conf)
File "/srv/app/src/ckan/ckan/config/environment.py", line 123, in load_environment
p.load_all()
File "/srv/app/src/ckan/ckan/plugins/core.py", line 165, in load_all
load(*plugins)
File "/srv/app/src/ckan/ckan/plugins/core.py", line 179, in load
service = _get_service(plugin)
File "/srv/app/src/ckan/ckan/plugins/core.py", line 281, in _get_service
return plugin.load()(name=plugin_name)
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2450, in load
return self.resolve()
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/srv/app/src/ckanext-spatial/ckanext/spatial/plugin/__init__.py", line 7, in <module>
import shapely.geometry
File "/usr/lib/python3.9/site-packages/shapely/__init__.py", line 1, in <module>
from .lib import GEOSException # NOQA
ImportError: numpy.core.multiarray failed to import