Merge pull request scylladb#5 from riptano/oss-next

Unify DSE and OSS drivers

Author: aboudreault

CHANGELOG.rst

@@ -7,12 +7,89 @@ Features
 * Allow passing ssl context for Twisted (PYTHON-1161)
 * ssl context and cloud support for Eventlet (PYTHON-1162)
 * Cloud Twisted support (PYTHON-1163)
+* Add additional_write_policy and read_repair to system schema parsing (PYTHON-1048)
+* Handle prepared id mismatch when repreparing on the fly (PYTHON-1124)
+* Remove *read_repair_chance table options (PYTHON-1140)
+* Flexible version parsing (PYTHON-1174)
+* Support NULL in collection deserializer (PYTHON-1123)
+* Avoid warnings about unspecified load balancing policy when connecting to a cloud cluster (PYTHON-1177)
+* Add Python 3.8 support (PYTHON-1189)
+* Add new DSE CQL keywords (PYTHON-1122)
+* [GRAPH] Ability to execute Fluent Graph queries asynchronously (PYTHON-1129)
 
 Bug Fixes
 ---------
 * re-raising the CQLEngineException will fail on Python 3 (PYTHON-1166)
+* asyncio message chunks can be processed discontinuously (PYTHON-1185)
+* Reconnect attempts persist after downed node removed from peers (PYTHON-1181)
 * Connection fails to validate ssl certificate hostname when SSLContext.check_hostname is set (PYTHON-1186)
 * ResponseFuture._set_result crashes on connection error when used with PrepareMessage (PYTHON-1187)
+* Insights fail to serialize the startup message when the SSL Context is from PyOpenSSL (PYTHON-1192)
+
+Others
+------
+* The driver has a new dependency: geomet. It comes from the dse-driver unification and
+  is used to support DSE geo types.
+
+Deprecations
+------------
+
+* DSELoadBalancingPolicy will be removed in the next major, consider using
+  the DefaultLoadBalancingPolicy.
+
+Merged from dse-driver:
+
+Features
+--------
+
+* Insights integration (PYTHON-1047)
+* Graph execution profiles should preserve their graph_source when graph_options is overridden (PYTHON-1021)
+* Add NodeSync metadata (PYTHON-799)
+* Add new NodeSync failure values (PYTHON-934)
+* DETERMINISTIC and MONOTONIC Clauses for Functions and Aggregates (PYTHON-955)
+* GraphOptions should show a warning for unknown parameters (PYTHON-819)
+* DSE protocol version 2 and continous paging backpressure (PYTHON-798)
+* GraphSON2 Serialization/Deserialization Support (PYTHON-775)
+* Add graph-results payload option for GraphSON format (PYTHON-773)
+* Create an AuthProvider for the DSE transitional mode (PYTHON-831)
+* Implement serializers for the Graph String API (PYTHON-778)
+* Provide deserializers for GraphSON types (PYTHON-782)
+* Add Graph DurationType support (PYTHON-607)
+* Support DSE DateRange type (PYTHON-668)
+* RLAC CQL output for materialized views (PYTHON-682)
+* Add Geom Types wkt deserializer
+* DSE Graph Client timeouts in custom payload (PYTHON-589)
+* Make DSEGSSAPIAuthProvider accept principal name (PYTHON-574)
+* Add config profiles to DSE graph execution (PYTHON-570)
+* DSE Driver version checking (PYTHON-568)
+* Distinct default timeout for graph queries (PYTHON-477)
+* Graph result parsing for known types (PYTHON-479,487)
+* Distinct read/write CL for graph execution (PYTHON-509)
+* Target graph analytics query to spark master when available (PYTHON-510)
+
+Bug Fixes
+---------
+
+* Continuous paging sessions raise RuntimeError when results are not entirely consumed (PYTHON-1054)
+* GraphSON Property deserializer should return a dict instead of a set (PYTHON-1033)
+* ResponseFuture.has_more_pages may hold the wrong value (PYTHON-946)
+* DETERMINISTIC clause in AGGREGATE misplaced in CQL generation (PYTHON-963)
+* graph module import cause a DLL issue on Windows due to its cythonizing failure (PYTHON-900)
+* Update date serialization to isoformat in graph (PYTHON-805)
+* DateRange Parse Error (PYTHON-729)
+* MontonicTimestampGenerator.__init__ ignores class defaults (PYTHON-728)
+* metadata.get_host returning None unexpectedly (PYTHON-709)
+* Sockets associated with sessions not getting cleaned up on session.shutdown() (PYTHON-673)
+* Resolve FQDN from ip address and use that as host passed to SASLClient (PYTHON-566)
+* Geospatial type implementations don't handle 'EMPTY' values. (PYTHON-481)
+* Correctly handle other types in geo type equality (PYTHON-508)
+
+Other
+-----
+* Add tests around cqlengine and continuous paging (PYTHON-872)
+* Add an abstract GraphStatement to handle different graph statements  (PYTHON-789)
+* Write documentation examples for DSE 2.0 features (PYTHON-732)
+* DSE_V1 protocol should not include all of protocol v5 (PYTHON-694)
 
 3.20.2
 ======
@@ -64,6 +141,7 @@ Bug Fixes
 * Call ConnectionException with correct kwargs (PYTHON-1117)
 * Can't connect to clusters built from source because version parsing doesn't handle 'x.y-SNAPSHOT' (PYTHON-1118)
 * Discovered node doesn´t honor the configured Cluster port on connection (PYTHON-1127)
+* Exception when use pk__token__gt filter In python 3.7 (PYTHON-1121)
 
 Other
 -----

README-dev.rst

@@ -2,23 +2,33 @@ Releasing
 =========
 * Run the tests and ensure they all pass
 * Update CHANGELOG.rst
-
   * Check for any missing entries
   * Add today's date to the release section
 * Update the version in ``cassandra/__init__.py``
-
   * For beta releases, use a version like ``(2, 1, '0b1')``
   * For release candidates, use a version like ``(2, 1, '0rc1')``
   * When in doubt, follow PEP 440 versioning
 * Add the new version in ``docs.yaml``
-
 * Commit the changelog and version changes, e.g. ``git commit -m'version 1.0.0'``
 * Tag the release.  For example: ``git tag -a 1.0.0 -m 'version 1.0.0'``
 * Push the tag and new ``master``: ``git push origin 1.0.0 ; git push origin master``
-* Upload the package to pypi::
+* Update the `python-driver` submodule of `python-driver-wheels`,
+  commit then push. This will trigger TravisCI and the wheels building.
+* For a GA release, upload the package to pypi::
+
+    # Clean the working directory
+    python setup.py clean
+    rm dist/*
 
-    python setup.py register
-    python setup.py sdist upload
+    # Build the source distribution
+    python setup.py sdist
+
+    # Download all wheels from the jfrog repository and copy them in
+    # the dist/ directory
+    cp /path/to/wheels/*.whl dist/
+
+    # Upload all files
+    twine upload dist/*
 
 * On pypi, make the latest GA the only visible version
 * Update the docs (see below)
@@ -27,6 +37,12 @@ Releasing
 
   * After a beta or rc release, this should look like ``(2, 1, '0b1', 'post0')``
 
+* After the release has been tagged, add a section to docs.yaml with the new tag ref::
+
+    versions:
+      - name: <version name>
+        ref: <release tag>
+
 * Commit and push
 * Update 'cassandra-test' branch to reflect new release
 
@@ -80,45 +96,66 @@ directory and build from scratch::
 
     rm -rf docs/_build/*
 
-Running the Tests
-=================
-In order for the extensions to be built and used in the test, run::
+Documentor
+==========
+We now also use another tool called Documentor with Sphinx source to build docs.
+This gives us versioned docs with nice integrated search. This is a private tool
+of DataStax.
 
-    nosetests
+Dependencies
+------------
+Sphinx
+~~~~~~
+Installed as described above
 
-You can run a specific test module or package like so::
+Documentor
+~~~~~~~~~~
+Clone and setup Documentor as specified in `the project <https://github.com/riptano/documentor#installation-and-quick-start>`_.
+This tool assumes Ruby, bundler, and npm are present.
 
-    nosetests -w tests/unit/
+Building
+--------
+The setup script expects documentor to be in the system path. You can either add it permanently or run with something
+like this::
 
-You can run a specific test method like so::
+    PATH=$PATH:<documentor repo>/bin python setup.py doc
 
-    nosetests -w tests/unit/test_connection.py:ConnectionTest.test_bad_protocol_version
+The docs will not display properly just browsing the filesystem in a browser. To view the docs as they would be in most
+web servers, use the SimpleHTTPServer module::
 
-Seeing Test Logs in Real Time
------------------------------
-Sometimes it's useful to output logs for the tests as they run::
+    cd docs/_build/
+    python -m SimpleHTTPServer
 
-    nosetests -w tests/unit/ --nocapture --nologcapture
+Then, browse to `localhost:8000 <http://localhost:8000>`_.
 
-Use tee to capture logs and see them on your terminal::
+Tests
+=====
 
-    nosetests -w tests/unit/ --nocapture --nologcapture 2>&1 | tee test.log
+Running Unit Tests
+------------------
+Unit tests can be run like so::
 
-Specifying a Cassandra Version for Integration Tests
-----------------------------------------------------
-You can specify a cassandra version with the ``CASSANDRA_VERSION`` environment variable::
+    nosetests -w tests/unit/
+
+You can run a specific test method like so::
+
+    nosetests -w tests/unit/test_connection.py:ConnectionTest.test_bad_protocol_version
+
+Running Integration Tests
+-------------------------
+In order to run integration tests, you must specify a version to run using the ``CASSANDRA_VERSION`` or ``DSE_VERSION`` environment variable::
 
     CASSANDRA_VERSION=2.0.9 nosetests -w tests/integration/standard
 
-You can also specify a cassandra directory (to test unreleased versions)::
+Or you can specify a cassandra directory (to test unreleased versions)::
 
-    CASSANDRA_DIR=/home/thobbs/cassandra nosetests -w tests/integration/standard
+    CASSANDRA_DIR=/home/thobbs/cassandra nosetests -w tests/integration/standard/
 
 Specifying the usage of an already running Cassandra cluster
-----------------------------------------------------
-The test will start the appropriate Cassandra clusters when necessary  but if you don't want this to happen because a Cassandra cluster is already running the flag ``USE_CASS_EXTERNAL`` can be used, for example: 
+------------------------------------------------------------
+The test will start the appropriate Cassandra clusters when necessary  but if you don't want this to happen because a Cassandra cluster is already running the flag ``USE_CASS_EXTERNAL`` can be used, for example::
 
-	USE_CASS_EXTERNAL=1 python setup.py nosetests -w tests/integration/standard
+    USE_CASS_EXTERNAL=1 CASSANDRA_VERSION=2.0.9 nosetests -w tests/integration/standard
 
 Specify a Protocol Version for Tests
 ------------------------------------
@@ -127,20 +164,24 @@ it with the ``PROTOCOL_VERSION`` environment variable::
 
     PROTOCOL_VERSION=3 nosetests -w tests/integration/standard
 
+Seeing Test Logs in Real Time
+-----------------------------
+Sometimes it's useful to output logs for the tests as they run::
+
+    nosetests -w tests/unit/ --nocapture --nologcapture
+
+Use tee to capture logs and see them on your terminal::
+
+    nosetests -w tests/unit/ --nocapture --nologcapture 2>&1 | tee test.log
+
 Testing Multiple Python Versions
 --------------------------------
-If you want to test all of python 2.7, 3.4, 3.5, 3.6 and pypy, use tox (this is what
+If you want to test all of python 2.7, 3.4, 3.5, 3.6, 3.7, and pypy, use tox (this is what
 TravisCI runs)::
 
     tox
 
-By default, tox only runs the unit tests because I haven't put in the effort
-to get the integration tests to run on TravicCI.  However, the integration
-tests should work locally.  To run them, edit the following line in tox.ini::
-
-    commands = {envpython} setup.py build_ext --inplace nosetests --verbosity=2 tests/unit/
-
-and change ``tests/unit/`` to ``tests/``.
+By default, tox only runs the unit tests.
 
 Running the Benchmarks
 ======================
@@ -169,3 +210,42 @@ name to specify the built version::
     python setup.py egg_info -b-`git rev-parse --short HEAD` sdist --formats=zip
 
 The file (``dist/cassandra-driver-<version spec>.zip``) is packaged with Cassandra in ``cassandra/lib/cassandra-driver-internal-only*zip``.
+
+Releasing an EAP
+================
+
+An EAP release is only uploaded on a private server and it is not published on pypi.
+
+* Clean the environment::
+
+    python setup.py clean
+
+* Package the source distribution::
+
+    python setup.py sdist
+
+* Test the source distribution::
+
+    pip install dist/cassandra-driver-<version>.tar.gz
+
+* Upload the package on the EAP download server.
+* Build the documentation::
+
+    python setup.py doc
+
+* Upload the docs on the EAP download server.
+
+Adding a New Python Runtime Support
+===================================
+
+* Add the new python version to our jenkins image:
+  https://github.com/riptano/openstack-jenkins-drivers/
+
+* Add the new python version in job-creator:
+  https://github.com/riptano/job-creator/
+
+* Run the tests and ensure they all pass
+  * also test all event loops
+
+* Update the wheels building repo to support that version:
+  https://github.com/riptano/python-dse-driver-wheels

README.rst

@@ -1,14 +1,13 @@
-DataStax Python Driver for Apache Cassandra
-===========================================
+DataStax Driver for Apache Cassandra
+====================================
 
 .. image:: https://travis-ci.org/datastax/python-driver.png?branch=master
    :target: https://travis-ci.org/datastax/python-driver
 
-A modern, `feature-rich <https://github.com/datastax/python-driver#features>`_ and highly-tunable Python client library for Apache Cassandra (2.1+) using exclusively Cassandra's binary protocol and Cassandra Query Language v3.
+A modern, `feature-rich <https://github.com/datastax/python-driver#features>`_ and highly-tunable Python client library for Apache Cassandra (2.1+) and
+DataStax Enterprise (4.7+) using exclusively Cassandra's binary protocol and Cassandra Query Language v3.
 
-The driver supports Python 2.7, 3.4, 3.5, 3.6 and 3.7.
-
-If you require compatibility with DataStax Enterprise, use the `DataStax Enterprise Python Driver <http://docs.datastax.com/en/developer/python-dse-driver/>`_.
+The driver supports Python 2.7, 3.4, 3.5, 3.6, 3.7 and 3.8.
 
 **Note:** DataStax products do not support big-endian systems.
 
@@ -28,6 +27,9 @@ Features
 * `Concurrent execution utilities <http://datastax.github.io/python-driver/api/cassandra/concurrent.html>`_
 * `Object mapper <http://datastax.github.io/python-driver/object_mapper.html>`_
 * `Connecting to DataStax Apollo database (cloud) <https://docs.datastax.com/en/developer/python-driver/latest/cloud/>`_
+* DSE Graph execution API
+* DSE Geometric type serialization
+* DSE PlainText and GSSAPI authentication
 
 Installation
 ------------
@@ -70,7 +72,7 @@ Getting Help
 ------------
 Your best options for getting help with the driver are the
 `mailing list <https://groups.google.com/a/lists.datastax.com/forum/#!forum/python-driver-user>`_
-and the ``#datastax-drivers`` channel in the `DataStax Academy Slack <https://academy.datastax.com/slack>`_.
+and the `DataStax Community <https://community.datastax.com>`_.
 
 License
 -------