diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index c635fd8c1b..8b8fc0e791 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -18,7 +18,7 @@ good bug reports. They will not be repeated in detail here, but in general, the Pull Requests ------------- If you're able to fix a bug yourself, you can `fork the repository `_ and submit a `Pull Request `_ with the fix. -Please include tests demonstrating the issue and fix. For examples of how to run the tests, consult the `dev README `_. +Please include tests demonstrating the issue and fix. For examples of how to run the tests, consult the further parts of this document. Design and Implementation Guidelines ------------------------------------ @@ -26,3 +26,97 @@ Design and Implementation Guidelines - This project follows `semantic versioning `_, so breaking API changes will only be introduced in major versions. - Legacy ``cqlengine`` has varying degrees of overreaching client-side validation. Going forward, we will avoid client validation where server feedback is adequate and not overly expensive. - When writing tests, try to achieve maximal coverage in unit tests (where it is faster to run across many runtimes). Integration tests are good for things where we need to test server interaction, or where it is important to test across different server versions (emulating in unit tests would not be effective). + +Dev setup +========= + +We recommend using `uv` tool for running tests, linters and basically everything else, +since it makes Python tooling ecosystem mostly usable. +To install it, see instructions at https://docs.astral.sh/uv/getting-started/installation/ +The rest of this document assumes you have `uv` installed. + +It is also strongly recommended to use C/C++-caching tool like ccache or sccache. +When modifying driver files, rebuilding Cython modules is often necessary. +Without caching, each such rebuild may take over a minute. Caching usually brings it +down to about 2-3 seconds. + +Building the Docs +================= + +To build and preview the documentation for the ScyllaDB Python driver locally, you must first manually install `python-driver`. +This is necessary for autogenerating the reference documentation of the driver. +You can find detailed instructions on how to install the driver in the `Installation guide `_. + +After installing the driver, you can build the documentation: +- Make sure you have Python version compatible with docs. You can see supported version in ``docs/pyproject.toml`` - look for ``python`` in ``tool.poetry.dependencies`` section. +- Install poetry: ``pip install poetry`` +- To preview docs in your browser: ``make -C docs preview`` + +Tests +===== + +Running Unit Tests +------------------ +Unit tests can be run like so:: + + uv run pytest tests/unit + EVENT_LOOP_MANAGER=gevent uv run pytest tests/unit/io/test_geventreactor.py + EVENT_LOOP_MANAGER=eventlet uv run pytest tests/unit/io/test_eventletreactor.py + +You can run a specific test method like so:: + + uv run pytest tests/unit/test_connection.py::ConnectionTest::test_bad_protocol_version + +Running Integration Tests +------------------------- +In order to run integration tests, you must specify a version to run using either of: +* ``SCYLLA_VERSION`` e.g. ``release:2025.2`` +* ``CASSANDRA_VERSION`` +environment variable:: + + SCYLLA_VERSION="release:2025.2" uv run pytest tests/integration/standard tests/integration/cqlengine/ + +Or you can specify a scylla/cassandra directory (to test unreleased versions):: + + SCYLLA_VERSION=/path/to/scylla uv run pytest tests/integration/standard/ + +Specifying the usage of an already running Scylla cluster +------------------------------------------------------------ +The test will start the appropriate Scylla clusters when necessary but if you don't want this to happen because a Scylla cluster is already running the flag ``USE_CASS_EXTERNAL`` can be used, for example:: + + USE_CASS_EXTERNAL=1 SCYLLA_VERSION='release:5.1' uv run pytest tests/integration/standard + +Specify a Protocol Version for Tests +------------------------------------ +The protocol version defaults to: +- 4 for Scylla >= 3.0 and Scylla Enterprise > 2019. +- 3 for older versions of Scylla +- 5 for Cassandra >= 4.0, 4 for Cassandra >= 2.2, 3 for Cassandra >= 2.1, 2 for Cassandra >= 2.0 +You can overwrite it with the ``PROTOCOL_VERSION`` environment variable:: + + PROTOCOL_VERSION=3 SCYLLA_VERSION="release:5.1" uv run pytest tests/integration/standard tests/integration/cqlengine/ + +Seeing Test Logs in Real Time +----------------------------- +Sometimes it's useful to output logs for the tests as they run:: + + uv run pytest -s tests/unit/ + +Use tee to capture logs and see them on your terminal:: + + uv run pytest -s tests/unit/ 2>&1 | tee test.log + + +Running the Benchmarks +====================== +There needs to be a version of Scyll running locally so before running the benchmarks, if ccm is installed: + + uv run ccm create benchmark_cluster --scylla -v release:2025.2 -n 1 -s + +To run the benchmarks, pick one of the files under the ``benchmarks/`` dir and run it:: + + uv run benchmarks/future_batches.py + +There are a few options. Use ``--help`` to see them all:: + + uv run benchmarks/future_batches.py --help diff --git a/MAINTENANCE.md b/MAINTENANCE.md new file mode 100644 index 0000000000..8fc860ac4b --- /dev/null +++ b/MAINTENANCE.md @@ -0,0 +1,13 @@ +Releasing +========= +* Run the tests and ensure they all pass +* Update the version in ``cassandra/__init__.py`` +* Add the new version in ``docs/conf.py`` (variables: ``TAGS``, ``LATEST_VERSION``, ``DEPRECATED_VERSIONS``). + * For patch version releases (like ``3.26.8-scylla -> 3.26.9-scylla``) replace the old version with new one in ``TAGS`` and update ``LATEST_VERSION``. + * For minor version releases (like ``3.26.9-scylla -> 3.27.0-scylla``) add new version to ``TAGS``, update ``LATEST_VERSION`` and add previous minor version to ``DEPRECATED_VERSIONS``. +* Commit the version changes, e.g. ``git commit -m 'Release 3.26.9'`` +* Tag the release. For example: ``git tag -a 3.26.9-scylla -m 'Release 3.26.9'`` +* Push the tag and new ``master`` SIMULTANEOUSLY: ``git push --atomic origin master v6.0.21-scylla`` +* Now new version and its docs should be automatically published. Check `PyPI `_ and `docs `_ to make sure its there. +* If you didn't push branch and tag simultaneously (or doc publishing failed for other reason) then restart the relevant job from GitHub Actions UI. +* Publish a GitHub Release and a post on community forum. diff --git a/README-dev.rst b/README-dev.rst deleted file mode 100644 index f158226de0..0000000000 --- a/README-dev.rst +++ /dev/null @@ -1,102 +0,0 @@ -Releasing -========= -* Run the tests and ensure they all pass -* Update the version in ``cassandra/__init__.py`` -* Add the new version in ``docs/conf.py`` (variables: ``TAGS``, ``LATEST_VERSION``, ``DEPRECATED_VERSIONS``). - * For patch version releases (like ``3.26.8-scylla -> 3.26.9-scylla``) replace the old version with new one in ``TAGS`` and update ``LATEST_VERSION``. - * For minor version releases (like ``3.26.9-scylla -> 3.27.0-scylla``) add new version to ``TAGS``, update ``LATEST_VERSION`` and add previous minor version to ``DEPRECATED_VERSIONS``. -* Commit the version changes, e.g. ``git commit -m 'Release 3.26.9'`` -* Tag the release. For example: ``git tag -a 3.26.9-scylla -m 'Release 3.26.9'`` -* Push the tag and new ``master`` SIMULTANEOUSLY: ``git push --atomic origin master v6.0.21-scylla`` -* Now new version and its docs should be automatically published. Check `PyPI `_ and `docs `_ to make sure its there. -* If you didn't push branch and tag simultaneously (or doc publishing failed for other reason) then restart the relevant job from GitHub Actions UI. -* Publish a GitHub Release and a post on community forum. - -Building the Docs -================= - -To build and preview the documentation for the ScyllaDB Python driver locally, you must first manually install `python-driver`. -This is necessary for autogenerating the reference documentation of the driver. -You can find detailed instructions on how to install the driver in the `Installation guide `_. - -After installing the driver, you can build the documentation: -- Make sure you have Python version compatible with docs. You can see supported version in ``docs/pyproject.toml`` - look for ``python`` in ``tool.poetry.dependencies`` section. -- Install poetry: ``pip install poetry`` -- To preview docs in your browser: ``make -C docs preview`` - -Tooling -======= - -We recommend using `uv` tool for running tests, linters and basically everything else, -since it makes Python tooling ecosystem mostly usable. -To install it, see instructions at https://docs.astral.sh/uv/getting-started/installation/ -The rest of this document assumes you have `uv` installed. - -Tests -===== - -Running Unit Tests ------------------- -Unit tests can be run like so:: - - uv run pytest tests/unit - EVENT_LOOP_MANAGER=gevent uv run pytest tests/unit/io/test_geventreactor.py - EVENT_LOOP_MANAGER=eventlet uv run pytest tests/unit/io/test_eventletreactor.py - -You can run a specific test method like so:: - - uv run pytest tests/unit/test_connection.py::ConnectionTest::test_bad_protocol_version - -Running Integration Tests -------------------------- -In order to run integration tests, you must specify a version to run using either of: -* ``SCYLLA_VERSION`` e.g. ``release:5.1`` -* ``CASSANDRA_VERSION`` -environment variable:: - - SCYLLA_VERSION="release:5.1" uv run pytest tests/integration/standard tests/integration/cqlengine/ - -Or you can specify a scylla/cassandra directory (to test unreleased versions):: - - SCYLLA_VERSION=/path/to/scylla uv run pytest tests/integration/standard/ - -Specifying the usage of an already running Scylla cluster ------------------------------------------------------------- -The test will start the appropriate Scylla clusters when necessary but if you don't want this to happen because a Scylla cluster is already running the flag ``USE_CASS_EXTERNAL`` can be used, for example:: - - USE_CASS_EXTERNAL=1 SCYLLA_VERSION='release:5.1' uv run pytest tests/integration/standard - -Specify a Protocol Version for Tests ------------------------------------- -The protocol version defaults to: -- 4 for Scylla >= 3.0 and Scylla Enterprise > 2019. -- 3 for older versions of Scylla -- 5 for Cassandra >= 4.0, 4 for Cassandra >= 2.2, 3 for Cassandra >= 2.1, 2 for Cassandra >= 2.0 -You can overwrite it with the ``PROTOCOL_VERSION`` environment variable:: - - PROTOCOL_VERSION=3 SCYLLA_VERSION="release:5.1" uv run pytest tests/integration/standard tests/integration/cqlengine/ - -Seeing Test Logs in Real Time ------------------------------ -Sometimes it's useful to output logs for the tests as they run:: - - uv run pytest -s tests/unit/ - -Use tee to capture logs and see them on your terminal:: - - uv run pytest -s tests/unit/ 2>&1 | tee test.log - - -Running the Benchmarks -====================== -There needs to be a version of cassandra running locally so before running the benchmarks, if ccm is installed: - - uv run ccm create benchmark_cluster -v 3.0.1 -n 1 -s - -To run the benchmarks, pick one of the files under the ``benchmarks/`` dir and run it:: - - uv run benchmarks/future_batches.py - -There are a few options. Use ``--help`` to see them all:: - - uv run benchmarks/future_batches.py --help