bpo-40360: Deprecate lib2to3 module in light of PEP 617 (GH-19663)

Deprecate lib2to3 module in light of PEP 617.

We anticipate removal in the 3.12 timeframe.

Author: carljm

Doc/library/2to3.rst

@@ -9,9 +9,7 @@
 of *fixers* to transform it into valid Python 3.x code.  The standard library
 contains a rich set of fixers that will handle almost all code.  2to3 supporting
 library :mod:`lib2to3` is, however, a flexible and generic library, so it is
-possible to write your own fixers for 2to3.  :mod:`lib2to3` could also be
-adapted to custom applications in which Python code needs to be edited
-automatically.
+possible to write your own fixers for 2to3.
 
 
 .. _2to3-using:
@@ -466,9 +464,17 @@ and off individually.  They are described here in more detail.
 
 --------------
 
+.. deprecated:: 3.10
+   Python 3.9 will switch to a PEG parser (see :pep:`617`), and Python 3.10 may
+   include new language syntax that is not parsable by lib2to3's LL(1) parser.
+   The ``lib2to3`` module may be removed from the standard library in a future
+   Python version. Consider third-party alternatives such as `LibCST`_ or
+   `parso`_.
+
 .. note::
 
    The :mod:`lib2to3` API should be considered unstable and may change
    drastically in the future.
 
-.. XXX What is the public interface anyway?
+.. _LibCST: https://libcst.readthedocs.io/
+.. _parso: https://parso.readthedocs.io/

Lib/lib2to3/__init__.py

@@ -1 +1,8 @@
-#empty
+import warnings
+
+
+warnings.warn(
+    "lib2to3 package is deprecated and may not be able to parse Python 3.10+",
+    PendingDeprecationWarning,
+    stacklevel=2,
+)

Lib/test/test___all__.py

@@ -17,6 +17,7 @@ def check_all(self, modname):
         names = {}
         with support.check_warnings(
             (".* (module|package)", DeprecationWarning),
+            (".* (module|package)", PendingDeprecationWarning),
             ("", ResourceWarning),
             quiet=True):
             try:

Misc/NEWS.d/next/Library/2020-04-22-20-55-17.bpo-40360.Er8sv-.rst

@@ -0,0 +1 @@
+The :mod:`lib2to3` module is pending deprecation due to :pep:`617`.