Skip to content

make terracotta compatible with python 3.13 #364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dionhaefner merged 7 commits into from
Apr 30, 2025
Merged

make terracotta compatible with python 3.13 #364

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
647b4a7
make terracotta compatible with python 3.13
jkittner Apr 16, 2025
5808695
pin marshmallow to <4
dionhaefner Apr 30, 2025
9cbb4dd
make terracotta compatible with python 3.13
jkittner Apr 16, 2025
360204f
Merge branch 'python3-13' of github.com:jkittner/terracotta into pyth…
dionhaefner Apr 30, 2025
f637162
ignore sqlite3 resource warning
dionhaefner Apr 30, 2025
2503aa4
roll back marshmallow pin
dionhaefner Apr 30, 2025
622aeeb
also upgrade conda tests
dionhaefner Apr 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:

matrix:
os: [ubuntu-latest]
python-version: ["3.9", "3.12"]
python-version: ["3.9", "3.13"]

defaults:
run:
Expand Down
4 changes: 2 additions & 2 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.12.2-slim-bullseye AS build_wheel
FROM python:3.13.3-slim-bullseye AS build_wheel

RUN apt-get update \
&& apt-get install \
Expand All @@ -15,7 +15,7 @@ RUN python -m pip install --upgrade pip \
&& python setup.py bdist_wheel


FROM python:3.12.2-slim-bullseye
FROM python:3.13.3-slim-bullseye

COPY --from=build_wheel /terracotta/dist/*.whl /terracotta/

Expand Down
3 changes: 2 additions & 1 deletion setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ filterwarnings =
ignore:numpy.dtype size changed:RuntimeWarning
ignore:numpy.ufunc size changed:RuntimeWarning
# sometimes, dependencies leak resources
ignore:.*socket\.socket.*:pytest.PytestUnraisableExceptionWarning
ignore::pytest.PytestUnraisableExceptionWarning
ignore:.*socket\.socket.*
Copy link
Contributor Author

@jkittner jkittner Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help me out on this? It passed with 3.12 the way it was, but for 3.13 this change was needed. But I believe this now ignores more than before, which is not a good thing...

Copy link
Collaborator

@dionhaefner dionhaefner Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was warning being raised?

Copy link
Contributor Author

@jkittner jkittner Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what we are getting, which should be ignored by this (or probably cleaned up) but so far everything looks good - all connect seem to use a context manager...

=================================================================================== FAILURES ===================================================================================
______________________________________________________________________________ test_where[mysql] _______________________________________________________________________________

cls = <class '_pytest.runner.CallInfo'>, func = <function call_and_report.<locals>.<lambda> at 0x78ee0b2351c0>, when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: Callable[[], TResult],
        when: Literal["collect", "setup", "call", "teardown"],
        reraise: type[BaseException] | tuple[type[BaseException], ...] | None = None,
    ) -> CallInfo[TResult]:
        """Call func, wrapping the result in a CallInfo.
    
        :param func:
            The function to call. Called without arguments.
        :type func: Callable[[], _pytest.runner.TResult]
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: TResult | None = func()

venv/lib/python3.13/site-packages/_pytest/runner.py:341: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
venv/lib/python3.13/site-packages/_pytest/runner.py:242: in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
venv/lib/python3.13/site-packages/pluggy/_hooks.py:513: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
venv/lib/python3.13/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
venv/lib/python3.13/site-packages/_pytest/threadexception.py:92: in pytest_runtest_call
    yield from thread_exception_runtest_hook()
venv/lib/python3.13/site-packages/_pytest/threadexception.py:68: in thread_exception_runtest_hook
    yield
venv/lib/python3.13/site-packages/_pytest/unraisableexception.py:95: in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def unraisable_exception_runtest_hook() -> Generator[None]:
        with catch_unraisable_exception() as cm:
            try:
                yield
            finally:
                if cm.unraisable:
                    if cm.unraisable.err_msg is not None:
                        err_msg = cm.unraisable.err_msg
                    else:
                        err_msg = "Exception ignored in"
                    msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
                    msg += "".join(
                        traceback.format_exception(
                            cm.unraisable.exc_type,
                            cm.unraisable.exc_value,
                            cm.unraisable.exc_traceback,
                        )
                    )
>                   warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E                   pytest.PytestUnraisableExceptionWarning: Exception ignored in: <sqlite3.Connection object at 0x78ee0bde5e40>
E                   
E                   Traceback (most recent call last):
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
E                       self.dialect.do_execute(
E                       ~~~~~~~~~~~~~~~~~~~~~~~^
E                           cursor, str_statement, effective_parameters, context
E                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E                       )
E                       ^
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/sqlalchemy/engine/default.py", line 945, in do_execute
E                       cursor.execute(statement, parameters)
E                       ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/pymysql/cursors.py", line 153, in execute
E                       result = self._query(query)
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/pymysql/cursors.py", line 322, in _query
E                       conn.query(q)
E                       ~~~~~~~~~~^^^
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/pymysql/connections.py", line 563, in query
E                       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
E                                             ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/pymysql/connections.py", line 825, in _read_query_result
E                       result.read()
E                       ~~~~~~~~~~~^^
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/pymysql/connections.py", line 1199, in read
E                       first_packet = self.connection._read_packet()
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/pymysql/connections.py", line 775, in _read_packet
E                       packet.raise_for_error()
E                       ~~~~~~~~~~~~~~~~~~~~~~^^
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/pymysql/protocol.py", line 219, in raise_for_error
E                       err.raise_mysql_exception(self._data)
E                       ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
E                       raise errorclass(errno, errval)
E                   pymysql.err.ProgrammingError: (1146, "Table 'flprauxwowkvktrdhwtiidkx.key_names' doesn't exist")
E                   
E                   During handling of the above exception, another exception occurred:
E                   
E                   Traceback (most recent call last):
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 2222, in _handle_dbapi_exception
E                       def _handle_dbapi_exception(
E                       
E                   ResourceWarning: unclosed database in <sqlite3.Connection object at 0x78ee0bde5e40>

venv/lib/python3.13/site-packages/_pytest/unraisableexception.py:85: PytestUnraisableExceptionWarning

Copy link
Collaborator

@dionhaefner dionhaefner Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, that's really weird.

Table 'flprauxwowkvktrdhwtiidkx.key_names' doesn't exist

Why would an error like this be unraisable...?

Copy link
Collaborator

@dionhaefner dionhaefner Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless we have a good idea as to why this happens, let's ignore it selectively – that is, something like this:

ignore:pymysql.err.ProgrammingError:pytest.PytestUnraisableExceptionWarning

Copy link
Contributor Author

@jkittner jkittner Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue lies deeper, I believe. I also get this for sqlite, but only sometimes. So this could be many things. The connect method not closing properly or the engine not being disposed properly? I tried defining __del__ for the class disposing the engine, that reduced the number of warnings, but still did not fix all of them.

that's why we unfortunately would have to keep the more generic ignore...

Not sure how to solve that - maybe using sessions might fix that?

Copy link
Collaborator

@dionhaefner dionhaefner Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the warnings you're seeing all the same or are some different to what you posted above?

(I can't reproduce this locally rn since I don't have a working MySQL install)

Copy link
Contributor Author

@jkittner jkittner Apr 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just ran the dbs in docker compose:

services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: root
    ports:
      - 127.0.0.1:3306:3306
  
  postgres:
    image: postgres
    environment:
      POSTGRES_USER: root
      POSTGRES_PASSWORD: root
    ports:
      - 127.0.0.1:5432:5432

The full output I get is this:

errors 
======================================================================================================================= ERRORS ========================================================================================================================
_______________________________________________________________________________________________ ERROR at setup of test_non_writable_lazy_loading[mysql] _______________________________________________________________________________________________

cls = <class '_pytest.runner.CallInfo'>, func = <function call_and_report.<locals>.<lambda> at 0x7dbb3ba3fb00>, when = 'setup', reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: Callable[[], TResult],
        when: Literal["collect", "setup", "call", "teardown"],
        reraise: type[BaseException] | tuple[type[BaseException], ...] | None = None,
    ) -> CallInfo[TResult]:
        """Call func, wrapping the result in a CallInfo.
    
        :param func:
            The function to call. Called without arguments.
        :type func: Callable[[], _pytest.runner.TResult]
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: TResult | None = func()

venv/lib/python3.13/site-packages/_pytest/runner.py:341: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv/lib/python3.13/site-packages/_pytest/runner.py:242: in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
venv/lib/python3.13/site-packages/pluggy/_hooks.py:513: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
venv/lib/python3.13/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
venv/lib/python3.13/site-packages/_pytest/unraisableexception.py:90: in pytest_runtest_setup
    yield from unraisable_exception_runtest_hook()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def unraisable_exception_runtest_hook() -> Generator[None]:
        with catch_unraisable_exception() as cm:
            try:
                yield
            finally:
                if cm.unraisable:
                    if cm.unraisable.err_msg is not None:
                        err_msg = cm.unraisable.err_msg
                    else:
                        err_msg = "Exception ignored in"
                    msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
                    msg += "".join(
                        traceback.format_exception(
                            cm.unraisable.exc_type,
                            cm.unraisable.exc_value,
                            cm.unraisable.exc_traceback,
                        )
                    )
>                   warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E                   pytest.PytestUnraisableExceptionWarning: Exception ignored in: <sqlite3.Connection object at 0x7dbb32596b60>
E                   
E                   Traceback (most recent call last):
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 1318, in oneshot
E                       def oneshot(*args, **kw):
E                       
E                   ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32596b60>

venv/lib/python3.13/site-packages/_pytest/unraisableexception.py:85: PytestUnraisableExceptionWarning
====================================================================================================================== FAILURES =======================================================================================================================
_____________________________________________________________________________________________________________ test_path_override[sqlite] ______________________________________________________________________________________________________________

cls = <class '_pytest.runner.CallInfo'>, func = <function call_and_report.<locals>.<lambda> at 0x7dbb325d60c0>, when = 'call', reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: Callable[[], TResult],
        when: Literal["collect", "setup", "call", "teardown"],
        reraise: type[BaseException] | tuple[type[BaseException], ...] | None = None,
    ) -> CallInfo[TResult]:
        """Call func, wrapping the result in a CallInfo.
    
        :param func:
            The function to call. Called without arguments.
        :type func: Callable[[], _pytest.runner.TResult]
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: TResult | None = func()

venv/lib/python3.13/site-packages/_pytest/runner.py:341: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv/lib/python3.13/site-packages/_pytest/runner.py:242: in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
venv/lib/python3.13/site-packages/pluggy/_hooks.py:513: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
venv/lib/python3.13/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
venv/lib/python3.13/site-packages/_pytest/threadexception.py:92: in pytest_runtest_call
    yield from thread_exception_runtest_hook()
venv/lib/python3.13/site-packages/_pytest/threadexception.py:68: in thread_exception_runtest_hook
    yield
venv/lib/python3.13/site-packages/_pytest/unraisableexception.py:95: in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def unraisable_exception_runtest_hook() -> Generator[None]:
        with catch_unraisable_exception() as cm:
            try:
                yield
            finally:
                if cm.unraisable:
                    if cm.unraisable.err_msg is not None:
                        err_msg = cm.unraisable.err_msg
                    else:
                        err_msg = "Exception ignored in"
                    msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
                    msg += "".join(
                        traceback.format_exception(
                            cm.unraisable.exc_type,
                            cm.unraisable.exc_value,
                            cm.unraisable.exc_traceback,
                        )
                    )
>                   warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E                   pytest.PytestUnraisableExceptionWarning: Exception ignored in: <sqlite3.Connection object at 0x7dbb323d45e0>
E                   
E                   Traceback (most recent call last):
E                     File "/home/kittnjdr/workspace/terracotta/venv/lib/python3.13/site-packages/sqlalchemy/sql/schema.py", line 5060, in columns_autoinc_first
E                       @property
E                       
E                   ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb323d45e0>

venv/lib/python3.13/site-packages/_pytest/unraisableexception.py:85: PytestUnraisableExceptionWarning
=============================================================================================================== short test summary info ===============================================================================================================
FAILED tests/drivers/test_raster_drivers.py::test_path_override[sqlite] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <sqlite3.Connection object at 0x7dbb323d45e0>
ERROR tests/drivers/test_raster_drivers.py::test_non_writable_lazy_loading[mysql] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <sqlite3.Connection object at 0x7dbb32596b60>
======================================================================================================= 1 failed, 122 passed, 1 error in 37.28s =======================================================================================================
Exception ignored in: <sqlite3.Connection object at 0x7dbb32595f30>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32595f30>
Exception ignored in: <sqlite3.Connection object at 0x7dbb32449a80>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32449a80>
Exception ignored in: <sqlite3.Connection object at 0x7dbb323d4400>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb323d4400>
Exception ignored in: <sqlite3.Connection object at 0x7dbb323d6020>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb323d6020>
Exception ignored in: <sqlite3.Connection object at 0x7dbb323d4d60>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb323d4d60>
Exception ignored in: <sqlite3.Connection object at 0x7dbb323d5300>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb323d5300>
Exception ignored in: <sqlite3.Connection object at 0x7dbb32448040>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32448040>
Exception ignored in: <sqlite3.Connection object at 0x7dbb324498a0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb324498a0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb324486d0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb324486d0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb32be7e20>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32be7e20>
Exception ignored in: <sqlite3.Connection object at 0x7dbb32be73d0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32be73d0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3382e6b0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3382e6b0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb325956c0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb325956c0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb32595210>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32595210>
Exception ignored in: <sqlite3.Connection object at 0x7dbb32595990>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32595990>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3244bb50>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3244bb50>
Exception ignored in: <sqlite3.Connection object at 0x7dbb32597880>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb32597880>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3247c5e0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3247c5e0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3247e980>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3247e980>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3247d210>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3247d210>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3247d4e0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3247d4e0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb322a4b80>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb322a4b80>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3247ee30>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3247ee30>
Exception ignored in: <sqlite3.Connection object at 0x7dbb322a65c0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb322a65c0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3b1a06d0>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3b1a06d0>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3b1a0d60>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3b1a0d60>
Exception ignored in: <sqlite3.Connection object at 0x7dbb323d6c50>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb323d6c50>
Exception ignored in: <sqlite3.Connection object at 0x7dbb322a5030>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb322a5030>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3a0d9b70>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3a0d9b70>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3a0d9c60>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3a0d9c60>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3b1a1a80>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3b1a1a80>
Exception ignored in: <sqlite3.Connection object at 0x7dbb3b1a1c60>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb3b1a1c60>
Exception ignored in: <sqlite3.Connection object at 0x7dbb31618d60>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb31618d60>
Exception ignored in: <sqlite3.Connection object at 0x7dbb31618c70>
ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7dbb31618c70

This is very inconsistent and it's not always the same (number of) tests that fail or throw the warning. If you only run parts of the test suite, the errors/warnings change or become more frequent. If you add test randomization e.g. using randomly everything breaks so in addition, there is some test pollution going on as well...

Could resource warnings also be some multi-processing/race condition issue?

so IMO the only way forward to at least make it work for 3.13 is ignoring these warnings for now and in the long term try to find the test pollution that are likely caused by the database state not being reset for each test.

Copy link
Collaborator

@dionhaefner dionhaefner Apr 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I can reproduce it now. No luck with isolating this issue though, so I opted to ignore that specific warning, which also seems to be what other projects do. I suspect this is an issue somewhere within the bowels of SQLAlchemy, since we at no point interact directly with sqlite3.

Not closing a SQLite database should be pretty benign to begin with so I think we're okay for now.

Copy link
Contributor Author

@jkittner jkittner Apr 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that's good to see that others have the same issue. I am fine with ignoring this for now and will subscribe to the other issue - maybe it can be resolved one way or another 👍


[mypy]
show_error_codes = True
Expand Down
3 changes: 2 additions & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Framework :: Flask",
"Operating System :: Microsoft :: Windows :: Windows 10",
"Operating System :: MacOS :: MacOS X",
Expand Down Expand Up @@ -67,7 +68,7 @@
"color-operations",
"flask",
"flask_cors",
"marshmallow>=3.0.0",
"marshmallow>=3.0.0,<4",
"mercantile",
"numpy%s" % numpy_version,
"pillow",
Expand Down
2 changes: 1 addition & 1 deletion terracotta/drivers/mysql_meta_store.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ class MySQLMetaStore(RelationalMetaStore):
SQL_TIMEOUT_KEY = "connect_timeout"

_CHARSET = "utf8mb4"
SQLA_STRING = functools.partial(VARCHAR, charset=_CHARSET)
SQLA_STRING = staticmethod(functools.partial(VARCHAR, charset=_CHARSET))

MAX_PRIMARY_KEY_SIZE = 767 // 4 # Max key length for MySQL is at least 767B
DEFAULT_PORT = 3306
Expand Down
2 changes: 1 addition & 1 deletion terracotta/handlers/compute.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"""

from typing import Sequence, Tuple, Mapping, Optional, TypeVar
from typing.io import BinaryIO
from typing import BinaryIO
from concurrent.futures import Future

from terracotta import get_settings, get_driver, image, xyz, exceptions
Expand Down
2 changes: 1 addition & 1 deletion terracotta/handlers/rgb.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"""

from typing import Sequence, Tuple, Optional, TypeVar
from typing.io import BinaryIO
from typing import BinaryIO
from concurrent.futures import Future

from terracotta import get_settings, get_driver, image, xyz, exceptions
Expand Down
2 changes: 1 addition & 1 deletion terracotta/handlers/singleband.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"""

from typing import Sequence, Mapping, Union, Tuple, Optional, TypeVar, cast
from typing.io import BinaryIO
from typing import BinaryIO

import collections

Expand Down
2 changes: 1 addition & 1 deletion terracotta/image.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"""

from typing import List, Sequence, Tuple, TypeVar, Union
from typing.io import BinaryIO
from typing import BinaryIO

from io import BytesIO

Expand Down
Loading