Updates based on review feedback

Author: ncoghlan

Doc/howto/argparse-optparse.rst

@@ -1,10 +1,11 @@
 .. currentmodule:: argparse
 
 .. _upgrading-optparse-code:
+.. _migrating-optparse-code:
 
-==========================
-Upgrading optparse code
-==========================
+============================================
+Migrating ``optparse`` code to ``argparse``
+============================================
 
 The :mod:`argparse` module offers several higher level features not natively
 provided by the :mod:`optparse` module, including:
@@ -17,13 +18,22 @@ provided by the :mod:`optparse` module, including:
 * Providing a much simpler interface for custom ``type`` and ``action``.
 
 Originally, the :mod:`argparse` module attempted to maintain compatibility
-with :mod:`optparse`.  However, :mod:`optparse` was difficult to extend
-transparently, particularly with the changes required to support
-``nargs=`` specifiers and better usage messages.  When most everything in
-:mod:`optparse` had either been copy-pasted over or monkey-patched, it no
-longer seemed practical to try to maintain the backwards compatibility.
+with :mod:`optparse`.  However, the fundamental design differences between
+supporting declarative command line option processing (while leaving positional
+argument processing to application code), and supporting both named options
+and positional arguments in the declarative interface mean that the
+API has diverged from that of ``optparse`` over time.
 
-A partial upgrade path from :mod:`optparse` to :mod:`argparse`:
+As described in :ref:`choosing-an-argument-parser`, applications that are
+currently using :mod:`optparse` and are happy with the way it works can
+just continue to use ``optparse``.
+
+Application developers that are considering migrating should also review
+the list of intrinsic behavioural differences described in that section
+before deciding whether or not migration is desirable.
+
+For applications that do choose to migrate from :mod:`optparse` to :mod:`argparse`,
+the following suggestions should be helpful:
 
 * Replace all :meth:`optparse.OptionParser.add_option` calls with
   :meth:`ArgumentParser.add_argument` calls.

Doc/howto/argparse.rst

@@ -18,7 +18,8 @@ recommended command-line parsing module in the Python standard library.
    module (which may require more code to configure for a given application,
    but also allows an application to request behaviors that ``argparse``
    doesn't support), and the very low level :mod:`getopt` (which specifically
-   serves as an equivalent to :c:func:`!getopt` from the C language).
+   serves as an equivalent to the :c:func:`!getopt` family of functions
+   available to C programmers).
    While neither of those modules is covered directly in this guide, many of
    the core concepts in ``argparse`` first originated in ``optparse``, so
    some aspects of this tutorial will also be relevant to ``optparse`` users.

Doc/library/argparse.rst

@@ -13,17 +13,15 @@
 
 .. note::
 
-   While :mod:`argparse` is the recommended standard library module for
-   *implementing* basic command line applications, authors of third party
-   command line argument processing libraries may find that the
-   lower level :mod:`optparse` module serves as a better foundation for
-   that use case. ``optparse`` (or one of the third party libraries
-   based on it) may also be worth considering for applications where
-   ``argparse`` doesn't support behaviors that the application requires
-   (such as entirely disabling support for interspersed options and
-   positional arguments, or stricter adherence to common Unix and Linux
-   command line interface conventions related to the handling of option
-   parameter values that start with ``-``).
+   While :mod:`argparse` is the default recommended standard library module
+   for implementing basic command line applications, authors with more
+   exacting requirements for exactly how their command line applications
+   behave may find it doesn't provide the necessary level of control.
+   Refer to :ref:`choosing-an-argument-parser` for alternatives to
+   consider when ``argparse`` doesn't support behaviors that the application
+   requires (such as entirely disabling support for interspersed options and
+   positional arguments, or accepting option parameter values that start
+   with ``-`` even when they correspond to another defined option).
 
 --------------
 

Doc/library/getopt.rst

@@ -9,7 +9,7 @@
 
 .. note::
 
-   This module is considered feature complete. A more object-oriented and
+   This module is considered feature complete. A more declarative and
    extensible alternative to this API is provided in the :mod:`optparse`
    module. Further functional enhancements for command line parameter
    processing are provided either as third party modules on PyPI,
@@ -27,7 +27,8 @@ Users who are unfamiliar with the Unix :c:func:`!getopt` function should conside
 using the :mod:`argparse` module instead. Users who are familiar with the Unix
 :c:func:`!getopt` function, but would like to get equivalent behavior while
 writing less code and getting better help and error messages should consider
-using the :mod:`optparse` module.
+using the :mod:`optparse` module. See :ref:`choosing-an-argument-parser` for
+additional details.
 
 This module provides two functions and an
 exception:
@@ -215,47 +216,28 @@ and more informative help and error messages by using the :mod:`optparse` module
          process(args, output=opts.output, verbose=opts.verbose)
 
 A roughly equivalent command line interface for this case can also be
-produced by using the :mod:`argparse` module::
-
-import argparse
-
-if __name__ == '__main__':
-      parser = argparse.ArgumentParser()
-      parser.add_argument('-o', '--output')
-      parser.add_argument('-v', dest='verbose', action='store_true')
-      parser.add_argument('rest', nargs='*')
-      args = parser.parse_args()
-      process(args.rest, output=args.output, verbose=args.verbose)
-
-However, unlike the ``optparse`` example, this ``argparse`` example will
-handle some parameter combinations differently from the way the ``getopt``
-version would handle them. For example (amongst other differences):
-
-* supplying ``-o -v`` gives ``output="-v"`` and ``verbose=False``
-  for both ``getopt`` and ``optparse``,
-  but a usage error with ``argparse``
-  (complaining that no value has been supplied for ``-o/--output``,
-  since ``-v`` is interpreted as meaning the verbosity flag)
-* similarly, supplying ``-o --`` gives ``output="--"`` and ``args=()``
-  for both ``getopt`` and ``optparse``,
-  but a usage error with ``argparse``
-  (also complaining that no value has been supplied for ``-o/--output``,
-  since ``--`` is interpreted as terminating the option processing
-  and treating all remaining values as positional arguments)
-* supplying ``-o=foo`` gives ``output="=foo"``
-  for both ``getopt`` and ``optparse``,
-  but gives ``output="foo"`` with ``argparse``
-  (since ``=`` is special cased as an alternative separator for
-  option parameter values)
-
-Whether these differing behaviors in the ``argparse`` version are
-considered desirable or a problem will depend on the specific command line
-application use case.
+produced by using the :mod:`argparse` module:
+
+.. testcode::
+
+   import argparse
+
+   if __name__ == '__main__':
+         parser = argparse.ArgumentParser()
+         parser.add_argument('-o', '--output')
+         parser.add_argument('-v', dest='verbose', action='store_true')
+         parser.add_argument('rest', nargs='*')
+         args = parser.parse_args()
+         process(args.rest, output=args.output, verbose=args.verbose)
+
+See :ref:`choosing-an-argument-parser` for details on how the ``argparse``
+version of this code differs in behaviour from the ``optparse`` (and
+``getopt``) version.
 
 .. seealso::
 
    Module :mod:`optparse`
-      More object-oriented command line option parsing.
+      Declarative command line option parsing.
 
    Module :mod:`argparse`
       More opinionated command line option and argument parsing library.

Doc/library/optparse.rst

@@ -3,41 +3,114 @@
 
 .. module:: optparse
    :synopsis: Command-line option parsing library.
-   :deprecated:
 
 .. moduleauthor:: Greg Ward <[email protected]>
 .. sectionauthor:: Greg Ward <[email protected]>
 
 **Source code:** :source:`Lib/optparse.py`
 
-.. note::
-
-   This module is considered feature complete. Further functional enhancements for
-   command line parameter processing are provided either as third party modules on
-   PyPI, or else as features in the :mod:`argparse` module.
-
-.. note::
-
-   The higher level :mod:`argparse` (rather than this module) is the recommended
-   standard library module for implementing command line applications unless one
-   of the following caveats applies:
-
-   * the application requires additional control over the way options and
-     positional parameters are interleaved on the command line (including
-     the ability to disable the interleaving feature completely)
-   * the application requires additional control over the incremental parsing
-     of command line elements (while ``argparse`` does support this, the
-     exact way it works in practice is undesirable for some use cases)
-   * the application requires additional control over the handling of options
-     which accept parameter values that may start with ``-`` (such as delegated
-     options to be passed to invoked subprocesses)
-   * the application requires some other command line parameter processing
-     behavior which ``argparse`` does not support, but which can be implemented
-     in terms of the lower level interface offered by ``optparse``
+--------------
 
-   These ``argparse`` caveats also mean that :mod:`optparse` is likely to
-   provide a better foundation for library authors *writing* third party
-   command line argument processing libraries.
+.. _choosing-an-argument-parser:
+
+Choosing an argument parsing library
+------------------------------------
+
+The standard library includes three argument parsing libraries:
+
+* :mod:`getopt`: a module that closely mirrors the procedural C ``getopt`` API.
+  Included in the standard library since before the initial Python 1.0 release.
+* :mod:`optparse`: a declarative replacement for ``getopt`` that
+  provides equivalent functionality without requiring each application
+  to implement its own procedural option parsing logic. First included
+  in the standard library since the Python 2.3 release.
+* :mod:`argparse`: a more opinionated alternative to ``optparse`` that
+  provides more functionality by default, at the expense of reduced application
+  flexibility in controlling exactly how arguments are processed. Included in
+  the standard library since the Python 2.7 and Python 3.2 releases.
+
+In the absence of more specific argument parsing design constraints, :mod:`argparse`
+is the recommended choice for implementing command line applications, as it offers
+the highest level of baseline functionality with the least application level code.
+
+:mod:`getopt` is retained almost entirely for backwards compatibility reasons.
+However, it also serves a niche use case as a tool for prototyping and testing
+command line argument handling in ``getopt``-based C applications.
+
+:mod:`optparse` should be considered as an alternative to :mod:`argparse` in the
+following cases:
+
+* an application is already using :mod:`optparse` and doesn't want to risk the
+  subtle behavioural changes that may arise when migrating to :mod:`argparse`
+* the application requires additional control over the way options and
+  positional parameters are interleaved on the command line (including
+  the ability to disable the interleaving feature completely)
+* the application requires additional control over the incremental parsing
+  of command line elements (while ``argparse`` does support this, the
+  exact way it works in practice is undesirable for some use cases)
+* the application requires additional control over the handling of options
+  which accept parameter values that may start with ``-`` (such as delegated
+  options to be passed to invoked subprocesses)
+* the application requires some other command line parameter processing
+  behavior which ``argparse`` does not support, but which can be implemented
+  in terms of the lower level interface offered by ``optparse``
+
+These considerations also mean that :mod:`optparse` is likely to provide a
+better foundation for library authors writing third party command line
+argument processing libraries.
+
+As a concrete example, consider the following two command line argument
+parsing configurations, the first using ``optparse``, and the second
+using ``argparse``:
+
+.. testcode::
+
+   import optparse
+
+   if __name__ == '__main__':
+         parser = optparse.OptionParser()
+         parser.add_option('-o', '--output')
+         parser.add_option('-v', dest='verbose', action='store_true')
+         opts, args = parser.parse_args()
+         process(args, output=opts.output, verbose=opts.verbose)
+
+.. testcode::
+
+   import argparse
+
+   if __name__ == '__main__':
+         parser = argparse.ArgumentParser()
+         parser.add_argument('-o', '--output')
+         parser.add_argument('-v', dest='verbose', action='store_true')
+         parser.add_argument('rest', nargs='*')
+         args = parser.parse_args()
+         process(args.rest, output=args.output, verbose=args.verbose)
+
+The most obvious difference is that in the ``optparse`` version, the non-option
+arguments are processed separately by the application after the option processing
+is complete. In the ``argparse`` version, positional arguments are declared and
+processed in the same way as the named options.
+
+However, the ``argparse`` version will also handle some parameter combination
+differently from the way the ``optparse`` version would handle them.
+For example (amongst other differences):
+
+* supplying ``-o -v`` gives ``output="-v"`` and ``verbose=False``
+  when using ``optparse``, but a usage error with ``argparse``
+  (complaining that no value has been supplied for ``-o/--output``,
+  since ``-v`` is interpreted as meaning the verbosity flag)
+* similarly, supplying ``-o --`` gives ``output="--"`` and ``args=()``
+  when using ``optparse``, but a usage error with ``argparse``
+  (also complaining that no value has been supplied for ``-o/--output``,
+  since ``--`` is interpreted as terminating the option processing
+  and treating all remaining values as positional arguments)
+* supplying ``-o=foo`` gives ``output="=foo"`` when using ``optparse``,
+  but gives ``output="foo"`` with ``argparse`` (since ``=`` is special
+  cased as an alternative separator for option parameter values)
+
+Whether these differing behaviors in the ``argparse`` version are
+considered desirable or a problem will depend on the specific command line
+application use case.
 
 .. seealso::
 
@@ -46,10 +119,12 @@
     developed as a set of decorated command implementation functions.
 
     Other third party libraries, such as :pypi:`typer` or :pypi:`msgspec-click`,
-    allow command line interfaces to be specified in ways that effectively
+    allow command line interfaces to be specified in ways that more effectively
     integrate with static checking of Python type annotations.
 
---------------
+
+Introduction
+------------
 
 :mod:`optparse` is a more convenient, flexible, and powerful library for parsing
 command-line options than the minimalist :mod:`getopt` module.
@@ -117,10 +192,11 @@ Background
 ----------
 
 :mod:`optparse` was explicitly designed to encourage the creation of programs
-with straightforward, conventional command-line interfaces.  To that end, it
-supports only the most common command-line syntax and semantics conventionally
-used under Unix.  If you are unfamiliar with these conventions, read this
-section to acquaint yourself with them.
+with straightforward command-line interfaces that follow the conventions
+established by the :c:func`!getopt` family of functions available to C developers.
+To that end, it supports only the most common command-line syntax and semantics
+conventionally used under Unix.  If you are unfamiliar with these conventions,
+reading this section will allow you to acquaint yourself with them.
 
 
 .. _optparse-terminology: