|
| 1 | +.. role:: javascript(code) |
| 2 | + :language: javascript |
| 3 | + |
| 4 | +============== |
| 5 | +Change Streams |
| 6 | +============== |
| 7 | + |
| 8 | +.. contents:: |
| 9 | + |
| 10 | +-------- |
| 11 | + |
| 12 | +Introduction |
| 13 | +============ |
| 14 | + |
| 15 | +The YAML and JSON files in this directory are platform-independent tests that |
| 16 | +drivers can use to prove their conformance to the Change Streams Spec. |
| 17 | + |
| 18 | +Several prose tests, which are not easily expressed in YAML, are also presented |
| 19 | +in this file. Those tests will need to be manually implemented by each driver. |
| 20 | + |
| 21 | +Spec Test Format |
| 22 | +================ |
| 23 | + |
| 24 | +Each YAML file has the following keys: |
| 25 | + |
| 26 | +- ``database_name``: The default database |
| 27 | +- ``collection_name``: The default collection |
| 28 | +- ``database2_name``: Another database |
| 29 | +- ``collection2_name``: Another collection |
| 30 | +- ``tests``: An array of tests that are to be run independently of each other. |
| 31 | + Each test will have some of the following fields: |
| 32 | + |
| 33 | + - ``description``: The name of the test. |
| 34 | + - ``minServerVersion``: The minimum server version to run this test against. If not present, assume there is no minimum server version. |
| 35 | + - ``maxServerVersion``: Reserved for later use |
| 36 | + - ``failPoint``: Reserved for later use |
| 37 | + - ``target``: The entity on which to run the change stream. Valid values are: |
| 38 | + |
| 39 | + - ``collection``: Watch changes on collection ``database_name.collection_name`` |
| 40 | + - ``database``: Watch changes on database ``database_name`` |
| 41 | + - ``client``: Watch changes on entire clusters |
| 42 | + - ``topology``: An array of server topologies against which to run the test. |
| 43 | + Valid topologies are ``single``, ``replicaset``, and ``sharded``. |
| 44 | + - ``changeStreamPipeline``: An array of additional aggregation pipeline stages to add to the change stream |
| 45 | + - ``changeStreamOptions``: Additional options to add to the changeStream |
| 46 | + - ``operations``: Array of documents, each describing an operation. Each document has the following fields: |
| 47 | + |
| 48 | + - ``database``: Database against which to run the operation |
| 49 | + - ``collection``: Collection against which to run the operation |
| 50 | + - ``name``: Name of the command to run |
| 51 | + - ``arguments`` (optional): Object of arguments for the command (ex: document to insert) |
| 52 | + |
| 53 | + - ``expectations``: Optional list of command-started events in Extended JSON format |
| 54 | + - ``result``: Document with ONE of the following fields: |
| 55 | + |
| 56 | + - ``error``: Describes an error received during the test |
| 57 | + - ``success``: An Extended JSON array of documents expected to be received from the changeStream |
| 58 | + |
| 59 | +Spec Test Match Function |
| 60 | +======================== |
| 61 | + |
| 62 | +The definition of MATCH or MATCHES in the Spec Test Runner is as follows: |
| 63 | + |
| 64 | +- MATCH takes two values, ``expected`` and ``actual`` |
| 65 | +- Notation is "Assert [actual] MATCHES [expected] |
| 66 | +- Assertion passes if ``expected`` is a subset of ``actual``, with the values ``42`` and ``"42"`` acting as placeholders for "any value" |
| 67 | + |
| 68 | +Pseudocode implementation of ``actual`` MATCHES ``expected``: |
| 69 | + |
| 70 | +:: |
| 71 | + |
| 72 | + If expected is "42" or 42: |
| 73 | + Assert that actual exists (is not null or undefined) |
| 74 | + Else: |
| 75 | + Assert that actual is of the same JSON type as expected |
| 76 | + If expected is a JSON array: |
| 77 | + For every idx/value in expected: |
| 78 | + Assert that actual[idx] MATCHES value |
| 79 | + Else if expected is a JSON object: |
| 80 | + For every key/value in expected |
| 81 | + Assert that actual[key] MATCHES value |
| 82 | + Else: |
| 83 | + Assert that expected equals actual |
| 84 | + |
| 85 | +The expected values for ``result.success`` and ``expectations`` are written in Extended JSON. Drivers may adopt any of the following approaches to comparisons, as long as they are consistent: |
| 86 | + |
| 87 | +- Convert ``actual`` to Extended JSON and compare to ``expected`` |
| 88 | +- Convert ``expected`` and ``actual`` to BSON, and compare them |
| 89 | +- Convert ``expected`` and ``actual`` to native equivalents of JSON, and compare them |
| 90 | + |
| 91 | +Spec Test Runner |
| 92 | +================ |
| 93 | + |
| 94 | +Before running the tests |
| 95 | + |
| 96 | +- Create a MongoClient ``globalClient``, and connect to the server |
| 97 | + |
| 98 | +For each YAML file, for each element in ``tests``: |
| 99 | + |
| 100 | +- If ``topology`` does not include the topology of the server instance(s), skip this test. |
| 101 | +- Use ``globalClient`` to |
| 102 | + |
| 103 | + - Drop the database ``database_name`` |
| 104 | + - Drop the database ``database2_name`` |
| 105 | + - Create the database ``database_name`` and the collection ``database_name.collection_name`` |
| 106 | + - Create the database ``database2_name`` and the collection ``database2_name.collection2_name`` |
| 107 | + |
| 108 | +- Create a new MongoClient ``client`` |
| 109 | +- Begin monitoring all APM events for ``client``. (If the driver uses global listeners, filter out all events that do not originate with ``client``). Filter out any "internal" commands (e.g. ``isMaster``) |
| 110 | +- Using ``client``, create a changeStream ``changeStream`` against the specified ``target``. Use ``changeStreamPipeline`` and ``changeStreamOptions`` if they are non-empty |
| 111 | +- Using ``globalClient``, run every operation in ``operations`` in serial against the server |
| 112 | +- Wait until either: |
| 113 | + |
| 114 | + - An error occurs |
| 115 | + - All operations have been successful AND the changeStream has received as many changes as there are in ``result.success`` |
| 116 | + |
| 117 | +- Close ``changeStream`` |
| 118 | +- If there was an error: |
| 119 | + |
| 120 | + - Assert that an error was expected for the test. |
| 121 | + - Assert that the error MATCHES ``results.error`` |
| 122 | + |
| 123 | +- Else: |
| 124 | + |
| 125 | + - Assert that no error was expected for the test |
| 126 | + - Assert that the changes received from ``changeStream`` MATCH the results in ``results.success`` |
| 127 | + |
| 128 | +- If there are any ``expectations`` |
| 129 | + |
| 130 | + - For each (``expected``, ``idx``) in ``expectations`` |
| 131 | + |
| 132 | + - Assert that ``actual[idx]`` MATCHES ``expected`` |
| 133 | + |
| 134 | +- Close the MongoClient ``client`` |
| 135 | + |
| 136 | +After running all tests |
| 137 | + |
| 138 | +- Close the MongoClient ``globalClient`` |
| 139 | +- Drop database ``database_name`` |
| 140 | +- Drop database ``database2_name`` |
| 141 | + |
| 142 | + |
| 143 | +Prose Tests |
| 144 | +=========== |
| 145 | + |
| 146 | +The following tests have not yet been automated, but MUST still be tested |
| 147 | + |
| 148 | +#. ``ChangeStream`` must continuously track the last seen ``resumeToken`` |
| 149 | +#. ``ChangeStream`` will throw an exception if the server response is missing the resume token |
| 150 | +#. ``ChangeStream`` will automatically resume one time on a resumable error (including `not master`) with the initial pipeline and options, except for the addition/update of a ``resumeToken``. |
| 151 | +#. ``ChangeStream`` will not attempt to resume on any error encountered while executing an ``aggregate`` command. |
| 152 | +#. ``ChangeStream`` will not attempt to resume after encountering error code 11601 (Interrupted), 136 (CappedPositionLost), or 237 (CursorKilled) while executing a ``getMore`` command. |
| 153 | +#. ``ChangeStream`` will perform server selection before attempting to resume, using initial ``readPreference`` |
| 154 | +#. Ensure that a cursor returned from an aggregate command with a cursor id and an initial empty batch is not closed on the driver side. |
| 155 | +#. The ``killCursors`` command sent during the "Resume Process" must not be allowed to throw an exception. |
| 156 | +#. ``$changeStream`` stage for ``ChangeStream`` against a server ``>=4.0`` that has not received any results yet MUST include a ``startAtOperationTime`` option when resuming a changestream. |
| 157 | +#. ``ChangeStream`` will resume after a ``killCursors`` command is issued for its child cursor. |
0 commit comments