minor: sync new crud tests

Author: saghm

src/test/spec/json/crud/README.rst

@@ -19,21 +19,22 @@ version requirements as noted by the ``runOn`` section, if provided.
 Subdirectories for Test Formats
 -------------------------------
 
-This document describes a current test format, which should be used for any new
-CRUD tests. Additionally, it refers to a "legacy" format, which dates back to
-the initial version of the CRUD specification. Until such time that all original
-tests have been ported to the current format, tests in each format will be
-grouped in their own subdirectory:
+This document describes two legacy formats for CRUD tests: legacy-v1, which dates back
+to the first version of the CRUD specification, and legacy-v2, which was an update to
+the initial format. New CRUD tests should be written in the `unified test format <../../../../unified-test-format/unified-test-format.rst>`_
+and placed under ``unified/``. Until such time that all original tests have been ported
+to the unified test format, tests in each format will be grouped in their own subdirectory:
 
-- ``v1/``: Legacy format tests
-- ``v2/``: Current format tests
+- ``v1/``: Legacy-v1 format tests
+- ``v2/``: Legacy-v2 format tests
+- ``unified/``: Tests using the `unified test format <../../../../unified-test-format/unified-test-format.rst>`_
 
 Since some drivers may not have a unified test runner capable of executing tests
-in both formats, segregating tests in this manner will make it easier for
+in all three formats, segregating tests in this manner will make it easier for
 drivers to sync and feed test files to different test runners.
 
-Test Format
-===========
+Legacy-v2 Test Format
+=====================
 
 *Note: this section pertains to test files in the "v2" directory.*
 
@@ -128,17 +129,17 @@ Each YAML file has the following keys:
         present then use the collection under test.
 
       - ``data``: The data that should exist in the collection after the
-        operation has been run.
+        operation has been run, sorted by "_id".
 
-Legacy Test Format for Single Operations
-----------------------------------------
+Legacy-v1 Test Format for Single Operations
+-------------------------------------------
 
 *Note: this section pertains to test files in the "v1" directory.*
 
 The test format above supports both multiple operations and APM expectations,
 and is consistent with the formats used by other specifications. Previously, the
 CRUD spec tests used a simplified format that only allowed for executing a
-single operation. Notable differences from the current format are as follows:
+single operation. Notable differences from the legacy-v2 format are as follows:
 
 - Instead of a ``tests[i].operations`` array, a single operation was defined as
   a document in ``tests[i].operation``. That document consisted of only the
@@ -156,9 +157,9 @@ single operation. Notable differences from the current format are as follows:
   field is not present, it should be assumed that there is no corresponding bound
   on the required server version.
 
-The legacy format should not conflict with the newer, multi-operation format
+The legacy-v1 format should not conflict with the newer, multi-operation format
 used by other specs (e.g. Transactions). It is possible to create a unified test
-runner capable of executing both formats (as some drivers do).
+runner capable of executing both legacy formats (as some drivers do).
 
 Error Assertions for Bulk Write Operations
 ==========================================
@@ -172,7 +173,9 @@ messages into the bulk write exception's top-level message.
 Test Runner Implementation
 ==========================
 
-This section provides guidance for implementing a test runner.
+This section provides guidance for implementing a test runner for legacy-v1 and
+legacy-v2 tests. See the `unified test format spec <../../../../unified-test-format/unified-test-format.rst>`_ for how to run tests under
+``unified/``.
 
 Before running the tests:
 
@@ -246,6 +249,8 @@ For each test file:
 
   - If the ``outcome`` field is present, assert the contents of the specified
     collection using ``globalMongoClient``.
+    Note the server does not guarantee that documents returned by a find
+    command will be in inserted order. This find MUST sort by ``{_id:1}``.
 
 Evaluating Matches
 ------------------
@@ -286,6 +291,14 @@ Note that in the case of result objects for some CRUD operations, ``expected``
 may condition additional, optional fields (see:
 `Optional Fields in Expected Result Objects`_).
 
+Fields that must NOT be present in Actual Documents
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Some command-started events in ``expectations`` include ``null`` values for
+optional fields such as ``allowDiskUse``.
+Tests MUST assert that the actual command **omits** any field that has a
+``null`` value in the expected command.
+
 Optional Fields in Expected Result Objects
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -294,3 +307,35 @@ specification, such as ``insertedId`` (for InsertOneResult), ``insertedIds``
 (for InsertManyResult), and ``upsertedCount`` (for UpdateResult). Drivers that
 do not implement these fields should ignore them when comparing ``actual`` with
 ``expected``.
+
+Prose Tests
+===========
+
+The following tests have not yet been automated, but MUST still be tested.
+
+"errInfo" is propagated
+-----------------------
+Test that a writeConcernError "errInfo" is propagated to the user in whatever way is idiomatic to the driver (exception, error object, etc.). Using a 4.0+ server, set the following failpoint:
+
+.. code:: javascript
+
+   {
+     "configureFailPoint": "failCommand",
+     "data": {
+       "failCommands": ["insert"],
+       "writeConcernError": {
+         "code": 100,
+         "codeName": "UnsatisfiableWriteConcern",
+         "errmsg": "Not enough data-bearing nodes",
+         "errInfo": {
+           "writeConcern": {
+             "w": 2,
+             "wtimeout": 0,
+             "provenance": "clientSupplied"
+           }
+         }
+       }
+     },
+     "mode": { "times": 1 }
+   }
+Then, perform an insert on the same database. Assert that an error occurs and that the "errInfo" is accessible and matches the one set in the failpoint.