Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,10 @@ PYTHON_TEST_VERSION ?= $(python_version)
PG_TEST_VERSION ?= $(MAJORVERSION)
UNSUPPORTS_SQLALCHEMY=$(shell python -c "import sqlalchemy;import psycopg2" 1> /dev/null 2>&1; echo $$?)

# Check if Python version is 3.12 or greater
PYTHON_VERSION_MAJOR_MINOR=$(shell python -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
PYTHON_GE_312=$(shell python -c "import sys; print(1 if (sys.version_info.major, sys.version_info.minor) >= (3, 12) else 0)")

TESTS = test-$(PYTHON_TEST_VERSION)/sql/multicorn_cache_invalidation.sql \
test-$(PYTHON_TEST_VERSION)/sql/multicorn_column_options_test.sql \
test-$(PYTHON_TEST_VERSION)/sql/multicorn_error_test.sql \
Expand All @@ -121,11 +125,15 @@ TESTS = test-$(PYTHON_TEST_VERSION)/sql/multicorn_cache_invalidation.sql
test-$(PYTHON_TEST_VERSION)/sql/multicorn_test_dict.sql \
test-$(PYTHON_TEST_VERSION)/sql/multicorn_test_list.sql \
test-$(PYTHON_TEST_VERSION)/sql/multicorn_test_sort.sql \
test-$(PYTHON_TEST_VERSION)/sql/write_filesystem.sql \
test-$(PYTHON_TEST_VERSION)/sql/write_savepoints.sql \
test-$(PYTHON_TEST_VERSION)/sql/write_test.sql \
test-$(PYTHON_TEST_VERSION)/sql/import_test.sql

# Only include write_filesystem.sql for Python versions < 3.12
ifneq ($(PYTHON_GE_312), 1)
TESTS += test-$(PYTHON_TEST_VERSION)/sql/write_filesystem.sql
endif

ifeq (${UNSUPPORTS_SQLALCHEMY}, 0)
TESTS += test-$(PYTHON_TEST_VERSION)/sql/multicorn_alchemy_test.sql \
test-$(PYTHON_TEST_VERSION)/sql/write_sqlalchemy.sql \
Expand Down
14 changes: 10 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Multicorn2
==========

Multicorn Python3 Foreign Data Wrapper (FDW) for Postgresql. Tested on Linux w/ Python 3.9-3.12 & Postgres 13-17.
Multicorn Python3 Foreign Data Wrapper (FDW) for Postgresql. Tested on Linux w/ Python 3.9-3.13 & Postgres 13-17.

Testing is underway for supporting Python 3.13 and is expected in v3.1. Newest versions of major linux distro's (Debian 12, Ubuntu 24.04 & EL10) are all still using Python 3.12 so sticking with using 3.12 is advised in the short run.

Expand Down Expand Up @@ -132,6 +132,15 @@ In your running instance of Postgres from the PSQL command line
CREATE EXTENSION multicorn;
```

## Known Issues

### PL/Python

multicorn2 and PL/Python are incompatible with each other as-of Python 3.12. Due to internal [technical limitations](https://github.com/pgsql-io/multicorn2/issues/60), both systems cannot be used simultaneously within the same PostgreSQL database.

However, both can be installed on the same system without conflict. Since PL/Python is commonly installed by default in packaged PostgreSQL distributions, multicorn2 can still be installed and used when PL/Python is not actively being used.


## Integration tests

multicorn2 has an extensive suite of integration tests which run against live PostgreSQL servers. In order to manage the matrix of supported versions of Python and PostgreSQL, the Nix package manager can be used to provide all the dependencies and run all the tests. The Nix package manager is supported on Linux, MacOS, and Windows Subsystem for Linux. To install Nix, follow the instructions at https://nixos.org/download/.
Expand All @@ -150,9 +159,6 @@ To run a faster test suite against a specific version of Python and PostgreSQL,
nix build .#testSuites.test_pg13_py39
```

**Known issues:**
- The tests cover the supported range of Python & PostgreSQL combinations;

### Adding new Python or PostgreSQL versions to the test suite

1. Perform a `nix flake update` in order to provide access to the latest packages available from the Nix package manager.
Expand Down
20 changes: 12 additions & 8 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@
# python39 # end of security support is scheduled for 2025-10-31; therefore nixpkgs support was dropped before nixos 25.05 was released
# python310 # error: sphinx-8.2.3 not supported for interpreter python3.10
python311
# python312 # tests are currently broken where plpython3u is used -- https://github.com/pgsql-io/multicorn2/issues/60
# python313 # tests are currently broken where plpython3u is used -- https://github.com/pgsql-io/multicorn2/issues/60
python312
python313
];
testPostgresVersions = with pkgs; [
postgresql_13
Expand Down Expand Up @@ -126,15 +126,15 @@
# "# -> Build order", so to speak... structed to build up a PostgreSQL with a compatible Python interpreter that
# is already configured to load the multicorn module.
#
# 1. Multicorn python package first, using the "raw" Python & "raw" PostgreSQL
# 1. PostgreSQL w/ plpython3, using "raw" Python
pythonEnabledPostgres = (makePostgresWithPlPython test_python test_postgresql);

# 2. Multicorn python package first, using the "raw" Python & "raw" PostgreSQL
multicornPython = (makeMulticornPythonPackage test_python test_postgresql);

# 2. Python enhanced w/ the multicorn package
# 3. Python enhanced w/ the multicorn package
enhancedPython = (test_python.withPackages (ps: [multicornPython] ++ (requiredPythonPackages ps) ));

# 3. PostgreSQL w/ plpython3, using "enhanced" Python
pythonEnabledPostgres = (makePostgresWithPlPython enhancedPython test_postgresql);

# 4. Multicorn postgresql extension, using the "enhanced" Python & plpython3 PostgreSQL
multicornPostgresExtension = (makeMulticornPostgresExtension enhancedPython pythonEnabledPostgres);

Expand All @@ -145,7 +145,11 @@
]);

pgMajorVersion = pkgs.lib.versions.major test_postgresql.version;
expectedTestCount = if pkgs.lib.versionOlder pgMajorVersion "14" then "18" else "19";
pythonVersion = pkgs.lib.versions.majorMinor test_python.version;
isPython312OrHigher = pkgs.lib.versionAtLeast pythonVersion "3.12";

baseTestCount = if pkgs.lib.versionOlder pgMajorVersion "14" then 18 else 19;
expectedTestCount = toString (baseTestCount - (if isPython312OrHigher then 1 else 0));
in pkgs.stdenv.mkDerivation {
name = "multicorn2-python-test-pg${test_postgresql.version}-py${test_python.version}";

Expand Down