Merge branch 'master' into bugfix/st-property-access-rewired

Author: sterliakov

CHANGELOG.md

@@ -2,11 +2,70 @@
 
 ## Next Release
 
-### Remove Support for targeting Python 3.8
+## Mypy 1.17
 
-Mypy now requires `--python-version 3.9` or greater. Support for only Python 3.8 is
-fully removed now. Given an unsupported version, mypy will default to the oldest
-supported one, currently 3.9.
+We’ve just uploaded mypy 1.17 to the Python Package Index ([PyPI](https://pypi.org/project/mypy/)).
+Mypy is a static type checker for Python. This release includes new features and bug fixes.
+You can install it as follows:
+
+    python3 -m pip install -U mypy
+
+You can read the full documentation for this release on [Read the Docs](http://mypy.readthedocs.io).
+
+### Optionally Check That Match Is Exhaustive
+
+Mypy can now optionally generate an error if a match statement does not
+match exhaustively, without having to use `assert_never(...)`. Enable
+this by using `--enable-error-code exhaustive-match`.
+
+Example:
+
+```python
+# mypy: enable-error-code=exhaustive-match
+
+import enum
+
+class Color(enum.Enum):
+    RED = 1
+    BLUE = 2
+
+def show_color(val: Color) -> None:
+    # error: Unhandled case for values of type "Literal[Color.BLUE]"
+    match val:
+        case Color.RED:
+            print("red")
+```
+
+This feature was contributed by Donal Burns (PR [19144](https://github.com/python/mypy/pull/19144)).
+
+### Further Improvements to Attribute Resolution
+
+This release includes additional improvements to how attribute types
+and kinds are resolved. These fix many bugs and overall improve consistency.
+
+* Handle corner case: protocol/class variable/descriptor (Ivan Levkivskyi, PR [19277](https://github.com/python/mypy/pull/19277))
+* Fix a few inconsistencies in protocol/type object interactions (Ivan Levkivskyi, PR [19267](https://github.com/python/mypy/pull/19267))
+* Refactor/unify access to static attributes (Ivan Levkivskyi, PR [19254](https://github.com/python/mypy/pull/19254))
+* Remove inconsistencies in operator handling (Ivan Levkivskyi, PR [19250](https://github.com/python/mypy/pull/19250))
+* Make protocol subtyping more consistent (Ivan Levkivskyi, PR [18943](https://github.com/python/mypy/pull/18943))
+
+### Fixes to Nondeterministic Type Checking
+
+Previous mypy versions could infer different types for certain expressions
+across different runs (typically depending on which order certain types
+were processed, and this order was nondeterministic). This release includes
+fixes to several such issues.
+
+* Fix nondeterministic type checking by making join with explicit Protocol and type promotion commute (Shantanu, PR [18402](https://github.com/python/mypy/pull/18402))
+* Fix nondeterministic type checking caused by nonassociative of None joins (Shantanu, PR [19158](https://github.com/python/mypy/pull/19158))
+* Fix nondeterministic type checking caused by nonassociativity of joins (Shantanu, PR [19147](https://github.com/python/mypy/pull/19147))
+* Fix nondeterministic type checking by making join between `type` and TypeVar commute (Shantanu, PR [19149](https://github.com/python/mypy/pull/19149))
+
+### Remove Support for Targeting Python 3.8
+
+Mypy now requires `--python-version 3.9` or greater. Support for targeting Python 3.8 is
+fully removed now. Since 3.8 is an unsupported version, mypy will default to the oldest
+supported version (currently 3.9) if you still try to target 3.8.
 
 This change is necessary because typeshed stopped supporting Python 3.8 after it
 reached its End of Life in October 2024.
@@ -17,18 +76,128 @@ Contributed by Marc Mueller
 ### Initial Support for Python 3.14
 
 Mypy is now tested on 3.14 and mypyc works with 3.14.0b3 and later.
-Mypyc compiled wheels of mypy itself will be available for new versions after 3.14.0rc1 is released.
+Binary wheels compiled with mypyc for mypy itself will be available for 3.14
+some time after 3.14.0rc1 has been released.
 
-Note that not all new features might be supported just yet.
+Note that not all features are supported just yet.
 
 Contributed by Marc Mueller (PR [19164](https://github.com/python/mypy/pull/19164))
 
-### Deprecated Flag: \--force-uppercase-builtins
+### Deprecated Flag: `--force-uppercase-builtins`
 
-Mypy only supports Python 3.9+. The \--force-uppercase-builtins flag is now deprecated and a no-op. It will be removed in a future version.
+Mypy only supports Python 3.9+. The `--force-uppercase-builtins` flag is now
+deprecated as unnecessary, and a no-op. It will be removed in a future version.
 
 Contributed by Marc Mueller (PR [19176](https://github.com/python/mypy/pull/19176))
 
+### Mypyc: Improvements to Generators and Async Functions
+
+This release includes both performance improvements and bug fixes related
+to generators and async functions (these share many implementation details).
+
+* Fix exception swallowing in async try/finally blocks with await (Chainfire, PR [19353](https://github.com/python/mypy/pull/19353))
+* Fix AttributeError in async try/finally with mixed return paths (Chainfire, PR [19361](https://github.com/python/mypy/pull/19361))
+* Make generated generator helper method internal (Jukka Lehtosalo, PR [19268](https://github.com/python/mypy/pull/19268))
+* Free coroutine after await encounters StopIteration (Jukka Lehtosalo, PR [19231](https://github.com/python/mypy/pull/19231))
+* Use non-tagged integer for generator label (Jukka Lehtosalo, PR [19218](https://github.com/python/mypy/pull/19218))
+* Merge generator and environment classes in simple cases (Jukka Lehtosalo, PR [19207](https://github.com/python/mypy/pull/19207))
+
+### Mypyc: Partial, Unsafe Support for Free Threading
+
+Mypyc has minimal, quite memory-unsafe support for the free threaded
+builds of 3.14. It is also only lightly tested. Bug reports and experience
+reports are welcome!
+
+Here are some of the major limitations:
+* Free threading only works when compiling a single module at a time.
+* If there is concurrent access to an object while another thread is mutating the same
+  object, it's possible to encounter segfaults and memory corruption.
+* There are no efficient native primitives for thread synthronization, though the
+  regular `threading` module can be used.
+* Some workloads don't scale well to multiple threads for no clear reason.
+
+Related PRs:
+
+* Enable partial, unsafe support for free-threading (Jukka Lehtosalo, PR [19167](https://github.com/python/mypy/pull/19167))
+* Fix incref/decref on free-threaded builds (Jukka Lehtosalo, PR [19127](https://github.com/python/mypy/pull/19127))
+
+### Other Mypyc Fixes and Improvements
+
+* Derive .c file name from full module name if using multi_file (Jukka Lehtosalo, PR [19278](https://github.com/python/mypy/pull/19278))
+* Support overriding the group name used in output files (Jukka Lehtosalo, PR [19272](https://github.com/python/mypy/pull/19272))
+* Add note about using non-native class to subclass built-in types (Jukka Lehtosalo, PR [19236](https://github.com/python/mypy/pull/19236))
+* Make some generated classes implicitly final (Jukka Lehtosalo, PR [19235](https://github.com/python/mypy/pull/19235))
+* Don't simplify module prefixes if using separate compilation (Jukka Lehtosalo, PR [19206](https://github.com/python/mypy/pull/19206))
+
+### Stubgen Improvements
+
+* Add import for `types` in `__exit__` method signature (Alexey Makridenko, PR [19120](https://github.com/python/mypy/pull/19120))
+* Add support for including class and property docstrings (Chad Dombrova, PR [17964](https://github.com/python/mypy/pull/17964))
+* Don't generate `Incomplete | None = None` argument annotation (Sebastian Rittau, PR [19097](https://github.com/python/mypy/pull/19097))
+* Support several more constructs in stubgen's alias printer (Stanislav Terliakov, PR [18888](https://github.com/python/mypy/pull/18888))
+
+### Miscellaneous Fixes and Improvements
+
+* Combine the revealed types of multiple iteration steps in a more robust manner (Christoph Tyralla, PR [19324](https://github.com/python/mypy/pull/19324))
+* Improve the handling of "iteration dependent" errors and notes in finally clauses (Christoph Tyralla, PR [19270](https://github.com/python/mypy/pull/19270))
+* Lessen dmypy suggest path limitations for Windows machines (CoolCat467, PR [19337](https://github.com/python/mypy/pull/19337))
+* Fix type ignore comments erroneously marked as unused by dmypy (Charlie Denton, PR [15043](https://github.com/python/mypy/pull/15043))
+* Fix misspelled `exhaustive-match` error code (johnthagen, PR [19276](https://github.com/python/mypy/pull/19276))
+* Fix missing error context for unpacking assignment involving star expression (Brian Schubert, PR [19258](https://github.com/python/mypy/pull/19258))
+* Fix and simplify error de-duplication (Ivan Levkivskyi, PR [19247](https://github.com/python/mypy/pull/19247))
+* Disallow `ClassVar` in type aliases (Brian Schubert, PR [19263](https://github.com/python/mypy/pull/19263))
+* Add script that prints list of compiled files when compiling mypy (Jukka Lehtosalo, PR [19260](https://github.com/python/mypy/pull/19260))
+* Fix help message url for "None and Optional handling" section (Guy Wilson, PR [19252](https://github.com/python/mypy/pull/19252))
+* Display fully qualified name of imported base classes in errors about incompatible overrides (Mikhail Golubev, PR [19115](https://github.com/python/mypy/pull/19115))
+* Avoid false `unreachable`, `redundant-expr`, and `redundant-casts` warnings in loops more robustly and efficiently, and avoid multiple `revealed type` notes for the same line (Christoph Tyralla, PR [19118](https://github.com/python/mypy/pull/19118))
+* Fix type extraction from `isinstance` checks (Stanislav Terliakov, PR [19223](https://github.com/python/mypy/pull/19223))
+* Erase stray type variables in `functools.partial` (Stanislav Terliakov, PR [18954](https://github.com/python/mypy/pull/18954))
+* Make inferring condition value recognize the whole truth table (Stanislav Terliakov, PR [18944](https://github.com/python/mypy/pull/18944))
+* Support type aliases, `NamedTuple` and `TypedDict` in constrained TypeVar defaults (Stanislav Terliakov, PR [18884](https://github.com/python/mypy/pull/18884))
+* Move dataclass `kw_only` fields to the end of the signature (Stanislav Terliakov, PR [19018](https://github.com/python/mypy/pull/19018))
+* Provide a better fallback value for the `python_version` option (Marc Mueller, PR [19162](https://github.com/python/mypy/pull/19162))
+* Avoid spurious non-overlapping equality error with metaclass with `__eq__` (Michael J. Sullivan, PR [19220](https://github.com/python/mypy/pull/19220))
+* Narrow type variable bounds (Ivan Levkivskyi, PR [19183](https://github.com/python/mypy/pull/19183))
+* Add classifier for Python 3.14 (Marc Mueller, PR [19199](https://github.com/python/mypy/pull/19199))
+* Capitalize syntax error messages (Charulata, PR [19114](https://github.com/python/mypy/pull/19114))
+* Infer constraints eagerly if actual is Any (Ivan Levkivskyi, PR [19190](https://github.com/python/mypy/pull/19190))
+* Include walrus assignments in conditional inference (Stanislav Terliakov, PR [19038](https://github.com/python/mypy/pull/19038))
+* Use PEP 604 syntax when converting types to strings (Marc Mueller, PR [19179](https://github.com/python/mypy/pull/19179))
+* Use more lower-case builtin types in error messages (Marc Mueller, PR [19177](https://github.com/python/mypy/pull/19177))
+* Fix example to use correct method of Stack (Łukasz Kwieciński, PR [19123](https://github.com/python/mypy/pull/19123))
+* Forbid `.pop` of `Readonly` `NotRequired` TypedDict items (Stanislav Terliakov, PR [19133](https://github.com/python/mypy/pull/19133))
+* Emit a friendlier warning on invalid exclude regex, instead of a stacktrace (wyattscarpenter, PR [19102](https://github.com/python/mypy/pull/19102))
+* Enable ANSI color codes for dmypy client in Windows (wyattscarpenter, PR [19088](https://github.com/python/mypy/pull/19088))
+* Extend special case for context-based type variable inference to unions in return position (Stanislav Terliakov, PR [18976](https://github.com/python/mypy/pull/18976))
+
+### Acknowledgements
+
+Thanks to all mypy contributors who contributed to this release:
+
+* Alexey Makridenko
+* Brian Schubert
+* Chad Dombrova
+* Chainfire
+* Charlie Denton
+* Charulata
+* Christoph Tyralla
+* CoolCat467
+* Donal Burns
+* Guy Wilson
+* Ivan Levkivskyi
+* johnthagen
+* Jukka Lehtosalo
+* Łukasz Kwieciński
+* Marc Mueller
+* Michael J. Sullivan
+* Mikhail Golubev
+* Sebastian Rittau
+* Shantanu
+* Stanislav Terliakov
+* wyattscarpenter
+
+I’d also like to thank my employer, Dropbox, for supporting mypy development.
+
 ## Mypy 1.16
 
 We’ve just uploaded mypy 1.16 to the Python Package Index ([PyPI](https://pypi.org/project/mypy/)).

docs/source/conf.py

@@ -278,9 +278,9 @@
 intersphinx_mapping = {
     "python": ("https://docs.python.org/3", None),
     "attrs": ("https://www.attrs.org/en/stable/", None),
-    "cython": ("https://docs.cython.org/en/latest", None),
+    "cython": ("https://cython.readthedocs.io/en/stable", None),
     "monkeytype": ("https://monkeytype.readthedocs.io/en/latest", None),
-    "setuptools": ("https://setuptools.readthedocs.io/en/latest", None),
+    "setuptools": ("https://setuptools.pypa.io/en/latest", None),
 }
 
 

misc/log_trace_check.py

@@ -0,0 +1,85 @@
+"""Compile mypy using mypyc with trace logging enabled, and collect a trace.
+
+The trace log can be used to analyze low-level performance bottlenecks.
+
+By default does a self check as the workload.
+
+This works on all supported platforms, unlike some of our other performance tools.
+"""
+
+from __future__ import annotations
+
+import argparse
+import glob
+import os
+import shutil
+import subprocess
+import sys
+import time
+
+from perf_compare import build_mypy, clone
+
+# Generated files, including binaries, go under this directory to avoid overwriting user state.
+TARGET_DIR = "mypy.log_trace.tmpdir"
+
+
+def perform_type_check(target_dir: str, code: str | None) -> None:
+    cache_dir = os.path.join(target_dir, ".mypy_cache")
+    if os.path.exists(cache_dir):
+        shutil.rmtree(cache_dir)
+    args = []
+    if code is None:
+        args.extend(["--config-file", "mypy_self_check.ini"])
+        for pat in "mypy/*.py", "mypy/*/*.py", "mypyc/*.py", "mypyc/test/*.py":
+            args.extend(glob.glob(pat))
+    else:
+        args.extend(["-c", code])
+    check_cmd = ["python", "-m", "mypy"] + args
+    t0 = time.time()
+    subprocess.run(check_cmd, cwd=target_dir, check=True)
+    elapsed = time.time() - t0
+    print(f"{elapsed:.2f}s elapsed")
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Compile mypy and collect a trace log while type checking (by default, self check)."
+    )
+    parser.add_argument(
+        "--multi-file",
+        action="store_true",
+        help="compile mypy into one C file per module (to reduce RAM use during compilation)",
+    )
+    parser.add_argument(
+        "--skip-compile", action="store_true", help="use compiled mypy from previous run"
+    )
+    parser.add_argument(
+        "-c",
+        metavar="CODE",
+        default=None,
+        type=str,
+        help="type check Python code fragment instead of mypy self-check",
+    )
+    args = parser.parse_args()
+    multi_file: bool = args.multi_file
+    skip_compile: bool = args.skip_compile
+    code: str | None = args.c
+
+    target_dir = TARGET_DIR
+
+    if not skip_compile:
+        clone(target_dir, "HEAD")
+
+        print(f"Building mypy in {target_dir} with trace logging enabled...")
+        build_mypy(target_dir, multi_file, log_trace=True, opt_level="0")
+    elif not os.path.isdir(target_dir):
+        sys.exit("error: Can't find compile mypy from previous run -- can't use --skip-compile")
+
+    perform_type_check(target_dir, code)
+
+    trace_fnam = os.path.join(target_dir, "mypyc_trace.txt")
+    print(f"Generated event trace log in {trace_fnam}")
+
+
+if __name__ == "__main__":
+    main()

misc/perf_compare.py

@@ -1,3 +1,5 @@
+#! /usr/bin/env python
+
 """Compare performance of mypyc-compiled mypy between one or more commits/branches.
 
 Simple usage:
@@ -35,13 +37,22 @@ def heading(s: str) -> None:
     print()
 
 
-def build_mypy(target_dir: str, multi_file: bool, *, cflags: str | None = None) -> None:
+def build_mypy(
+    target_dir: str,
+    multi_file: bool,
+    *,
+    cflags: str | None = None,
+    log_trace: bool = False,
+    opt_level: str = "2",
+) -> None:
     env = os.environ.copy()
     env["CC"] = "clang"
-    env["MYPYC_OPT_LEVEL"] = "2"
+    env["MYPYC_OPT_LEVEL"] = opt_level
     env["PYTHONHASHSEED"] = "1"
     if multi_file:
         env["MYPYC_MULTI_FILE"] = "1"
+    if log_trace:
+        env["MYPYC_LOG_TRACE"] = "1"
     if cflags is not None:
         env["CFLAGS"] = cflags
     cmd = [sys.executable, "setup.py", "--use-mypyc", "build_ext", "--inplace"]

mypy/binder.py

@@ -138,6 +138,10 @@ def __init__(self, options: Options) -> None:
         # flexible inference of variable types (--allow-redefinition-new).
         self.bind_all = options.allow_redefinition_new
 
+        # This tracks any externally visible changes in binder to invalidate
+        # expression caches when needed.
+        self.version = 0
+
     def _get_id(self) -> int:
         self.next_id += 1
         return self.next_id
@@ -158,6 +162,7 @@ def push_frame(self, conditional_frame: bool = False) -> Frame:
         return f
 
     def _put(self, key: Key, type: Type, from_assignment: bool, index: int = -1) -> None:
+        self.version += 1
         self.frames[index].types[key] = CurrentType(type, from_assignment)
 
     def _get(self, key: Key, index: int = -1) -> CurrentType | None:
@@ -185,6 +190,7 @@ def put(self, expr: Expression, typ: Type, *, from_assignment: bool = True) -> N
         self._put(key, typ, from_assignment)
 
     def unreachable(self) -> None:
+        self.version += 1
         self.frames[-1].unreachable = True
 
     def suppress_unreachable_warnings(self) -> None: