--strict-equality for None (#19718)

Fixes #18386 (at least partly)

(edited)

In a first test run, in which I included checks against `None` in
`--strict-equality`, the Mypy primer gave hundreds of new
`comparison-overlap` reports. Many of them seem really helpful
(including those for the Mypy source code itself), because it is often
hard to tell if non-overlapping `None` checks are just remnants of
incomplete refactorings or can handle cases with corrupted data or
similar issues. As it was only a little effort, I decided to add the
option `--strict-equality-for-none` to Mypy, which is disabled even in
`--strict` mode. Other libraries could adjust to this new behaviour if
and when they want. If many of them do so, we could eventually enable
`--strict-equality-for-none` in `--strict` mode or even merge it with
`--strict-equality` later.

The remaining new true positives revealed by the Mypy primer are the
result of no longer excluding types with custom `__eq__` methods for
identity checks (which, in my opinion, makes sense even in case
`--strict-equality-for-none` would be rejected).

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Author: tyralla

docs/source/command_line.rst

@@ -728,9 +728,22 @@ of the above sections.
        if text != b'other bytes':  # Error: non-overlapping equality check!
            ...
 
-       assert text is not None  # OK, check against None is allowed as a special case.
+       assert text is not None  # OK, check against None is allowed
 
 
+.. option:: --strict-equality-for-none
+
+    This flag extends :option:`--strict-equality <mypy --strict-equality>` for checks
+    against ``None``:
+
+    .. code-block:: python
+
+       text: str
+       assert text is not None  # Error: non-overlapping identity check!
+
+    Note that :option:`--strict-equality-for-none <mypy --strict-equality-for-none>`
+    only works in combination with :option:`--strict-equality <mypy --strict-equality>`.
+
 .. option:: --strict-bytes
 
     By default, mypy treats ``bytearray`` and ``memoryview`` as subtypes of ``bytes`` which

docs/source/config_file.rst

@@ -834,7 +834,15 @@ section of the command line docs.
    :default: False
 
    Prohibit equality checks, identity checks, and container checks between
-   non-overlapping types.
+   non-overlapping types (except ``None``).
+
+.. confval:: strict_equality_for_none
+
+   :type: boolean
+   :default: False
+
+   Include ``None`` in strict equality checks (requires :confval:`strict_equality`
+   to be activated).
 
 .. confval:: strict_bytes
 

docs/source/error_code_list2.rst

@@ -145,6 +145,29 @@ literal:
     def is_magic(x: bytes) -> bool:
         return x == b'magic'  # OK
 
+:option:`--strict-equality <mypy --strict-equality>` does not include comparisons with
+``None``:
+
+.. code-block:: python
+
+    # mypy: strict-equality
+
+    def is_none(x: str) -> bool:
+        return x is None  # OK
+
+If you want such checks, you must also activate
+:option:`--strict-equality-for-none <mypy --strict-equality-for-none>` (we might merge
+these two options later).
+
+.. code-block:: python
+
+    # mypy: strict-equality strict-equality-for-none
+
+    def is_none(x: str) -> bool:
+        # Error: Non-overlapping identity check
+        #        (left operand type: "str", right operand type: "None")
+        return x is None
+
 .. _code-no-untyped-call:
 
 Check that no untyped functions are called [no-untyped-call]

mypy/checkexpr.py

@@ -3721,7 +3721,7 @@ def visit_comparison_expr(self, e: ComparisonExpr) -> Type:
             elif operator == "is" or operator == "is not":
                 right_type = self.accept(right)  # validate the right operand
                 sub_result = self.bool_type()
-                if self.dangerous_comparison(left_type, right_type):
+                if self.dangerous_comparison(left_type, right_type, identity_check=True):
                     # Show the most specific literal types possible
                     left_type = try_getting_literal(left_type)
                     right_type = try_getting_literal(right_type)
@@ -3763,6 +3763,7 @@ def dangerous_comparison(
         original_container: Type | None = None,
         seen_types: set[tuple[Type, Type]] | None = None,
         prefer_literal: bool = True,
+        identity_check: bool = False,
     ) -> bool:
         """Check for dangerous non-overlapping comparisons like 42 == 'no'.
 
@@ -3790,10 +3791,12 @@ def dangerous_comparison(
 
         left, right = get_proper_types((left, right))
 
-        # We suppress the error if there is a custom __eq__() method on either
-        # side. User defined (or even standard library) classes can define this
+        # We suppress the error for equality and container checks if there is a custom __eq__()
+        # method on either side. User defined (or even standard library) classes can define this
         # to return True for comparisons between non-overlapping types.
-        if custom_special_method(left, "__eq__") or custom_special_method(right, "__eq__"):
+        if (
+            custom_special_method(left, "__eq__") or custom_special_method(right, "__eq__")
+        ) and not identity_check:
             return False
 
         if prefer_literal:
@@ -3817,7 +3820,10 @@ def dangerous_comparison(
             #
             # TODO: find a way of disabling the check only for types resulted from the expansion.
             return False
-        if isinstance(left, NoneType) or isinstance(right, NoneType):
+        if self.chk.options.strict_equality_for_none:
+            if isinstance(left, NoneType) and isinstance(right, NoneType):
+                return False
+        elif isinstance(left, NoneType) or isinstance(right, NoneType):
             return False
         if isinstance(left, UnionType) and isinstance(right, UnionType):
             left = remove_optional(left)

mypy/main.py

@@ -903,7 +903,16 @@ def add_invertible_flag(
         "--strict-equality",
         default=False,
         strict_flag=True,
-        help="Prohibit equality, identity, and container checks for non-overlapping types",
+        help="Prohibit equality, identity, and container checks for non-overlapping types "
+        "(except `None`)",
+        group=strictness_group,
+    )
+
+    add_invertible_flag(
+        "--strict-equality-for-none",
+        default=False,
+        strict_flag=False,
+        help="Extend `--strict-equality` for `None` checks",
         group=strictness_group,
     )
 

mypy/options.py

@@ -55,6 +55,7 @@ class BuildType:
     "mypyc",
     "strict_concatenate",
     "strict_equality",
+    "strict_equality_for_none",
     "strict_optional",
     "warn_no_return",
     "warn_return_any",
@@ -230,6 +231,9 @@ def __init__(self) -> None:
         # This makes 1 == '1', 1 in ['1'], and 1 is '1' errors.
         self.strict_equality = False
 
+        # Extend the logic of `scrict_equality` for comparisons with `None`.
+        self.strict_equality_for_none = False
+
         # Disable treating bytearray and memoryview as subtypes of bytes
         self.strict_bytes = False
 

test-data/unit/check-expressions.test

@@ -2419,6 +2419,35 @@ assert a == c
 [builtins fixtures/list.pyi]
 [typing fixtures/typing-full.pyi]
 
+[case testStrictEqualityForNone]
+# flags: --strict-equality --strict-equality-for-none
+
+class A: ...
+
+def a1(x: A) -> None:
+    assert x is None  # E: Non-overlapping identity check (left operand type: "A", right operand type: "None")
+def a2(x: A) -> None:
+    x is not None  # E: Non-overlapping identity check (left operand type: "A", right operand type: "None")
+def a3(x: A) -> None:
+    None == x  # E: Non-overlapping equality check (left operand type: "None", right operand type: "A")
+def a4(x: list[A]) -> None:
+    None in x  # E: Non-overlapping container check (element type: "None", container item type: "A")
+
+class B:
+    def __eq__(self, x: object) -> bool: ...
+
+def b1(x: B) -> None:
+    assert x is None  # E: Non-overlapping identity check (left operand type: "B", right operand type: "None")
+def b2(x: B) -> None:
+    x is not None  # E: Non-overlapping identity check (left operand type: "B", right operand type: "None")
+def b3(x: B) -> None:
+    x == None
+def b4(x: list[B]) -> None:
+    None in x
+
+[builtins fixtures/list.pyi]
+[typing fixtures/typing-full.pyi]
+
 [case testUnimportedHintAny]
 def f(x: Any) -> None:  # E: Name "Any" is not defined \
                         # N: Did you forget to import it from "typing"? (Suggestion: "from typing import Any")