Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
2b57bd0
allow initial as keyword, update docs and news
sayandipdutta Oct 24, 2024
b7617c8
Merge branch 'main' into allow_initial_keyword_reduce
sayandipdutta Oct 24, 2024
47abbd1
Apply suggestions from code review
sayandipdutta Oct 24, 2024
2fc8841
Use Argument Clinic
sayandipdutta Oct 24, 2024
7b795ba
fix functools.reduce signature for pure python
sayandipdutta Oct 24, 2024
11e7d13
initial defaults to _functools._initial_missing
sayandipdutta Oct 24, 2024
25c35e3
update docstring for python version
sayandipdutta Oct 24, 2024
ede87bf
Merge branch 'main' into allow_initial_keyword_reduce
sayandipdutta Oct 24, 2024
d8a5538
Apply suggestions from code review
sayandipdutta Oct 24, 2024
84f0a04
Update Doc/library/functools.rst
sayandipdutta Oct 24, 2024
8b3c2e5
review remove private API usage for PyObject_New
sayandipdutta Oct 24, 2024
2705ffa
inspecting reduce signature unsupported
sayandipdutta Oct 24, 2024
0d81eb2
Update Lib/functools.py
sayandipdutta Oct 25, 2024
46979d0
resolve conflicts
sayandipdutta Nov 1, 2024
7e05bc7
remove _initial_missing
sayandipdutta Nov 1, 2024
f1b5994
Update Misc/NEWS.d/next/Library/2024-10-24-13-40-20.gh-issue-126916.M…
sayandipdutta Nov 1, 2024
7cc052f
update whatsnew entry
sayandipdutta Nov 2, 2024
4130df8
Merge branch 'main' into allow_initial_keyword_reduce
sayandipdutta Nov 2, 2024
b4fb15c
Merge branch 'main' into allow_initial_keyword_reduce
sayandipdutta Nov 11, 2024
0cda3b6
Merge branch 'main' into allow_initial_keyword_reduce
sayandipdutta Nov 12, 2024
85e5012
Merge branch 'main' into allow_initial_keyword_reduce
erlend-aasland Nov 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions Doc/library/functools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,7 @@ The :mod:`functools` module defines the following functions:
.. versionadded:: 3.4


.. function:: reduce(function, iterable[, initial], /)
.. function:: reduce(function, iterable, /[, initial])

Apply *function* of two arguments cumulatively to the items of *iterable*, from
left to right, so as to reduce the iterable to a single value. For example,
Expand All @@ -468,7 +468,7 @@ The :mod:`functools` module defines the following functions:

initial_missing = object()

def reduce(function, iterable, initial=initial_missing, /):
def reduce(function, iterable, /, initial=initial_missing):
it = iter(iterable)
if initial is initial_missing:
value = next(it)
Expand All @@ -481,6 +481,9 @@ The :mod:`functools` module defines the following functions:
See :func:`itertools.accumulate` for an iterator that yields all intermediate
values.

.. versionchanged:: 3.14
*initial* is now supported as a keyword argument.

.. decorator:: singledispatch

Transform a function into a :term:`single-dispatch <single
Expand Down
2 changes: 1 addition & 1 deletion Lib/functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ def __ge__(self, other):

def reduce(function, sequence, initial=_initial_missing):
"""
reduce(function, iterable[, initial], /) -> value
reduce(function, iterable, /, initial=None) -> value
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use ellipsis:

Suggested change
reduce(function, iterable, /, initial=None) -> value
reduce(function, iterable, /, initial=...) -> value

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See PEP 661:)

Copy link
Contributor

@nineteendo nineteendo Oct 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the sentinel is private and doesn't even exist in the C implementation. Ellipsis is frequently used for unspecified default values in typeshed. We could use multiple signatures though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the sentinel is private and doesn't even exist in the C implementation.

It's easy to add, see #125917 (comment)

Ellipsis is frequently used for unspecified default values in typeshed.

@Eclips4?

We could use multiple signatures though.

Yes, I think it's fine for the sphinx docs. But help will looks like this (as for pure-Python version):

>>> help(functools.reduce)
Help on built-in function reduce in module _functools:

reduce(function, iterable, /,
       initial=_functools._initial_missing)
    Apply a function of two arguments cumulatively to an iterable, from left to right.

    [...]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I've ever seen =... in the docs. Do we have precedent for that?

Copy link
Contributor Author

@sayandipdutta sayandipdutta Oct 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like the signature is giving inspect a hard time. But it is autogenerated by AC. Did I do something wrong?

reduce(function, iterable, /,
       initial=_functools._initial_missing)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the sentinel is private and doesn't even exist in the C implementation. Ellipsis is frequently used for unspecified default values in typeshed. We could use multiple signatures though.

Multiple signatures for a docs sounds like a good solution.
Using ... for default values is essentially the same as using None, and it's just wrong since users can pass ... as the initial value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I've ever seen =... in the docs. Do we have precedent for that?

Yeah, e.g. for the int.from_bytes, for example.

it seems like the signature is giving inspect a hard time. But it is autogenerated by AC. Did I do something wrong?

First, note that reduce() has no correct signature in the current main.

Now AC adds one, but it can't be parsed by inspect._signature_fromstr(): this helper has own opinion on what can be specified as a default value (e.g. it can't be a complex number).


Apply a function of two arguments cumulatively to the items of a sequence
or iterable, from left to right, so as to reduce the iterable to a single
Expand Down
23 changes: 23 additions & 0 deletions Lib/test/test_functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -1005,6 +1005,29 @@ def __getitem__(self, i):
d = {"one": 1, "two": 2, "three": 3}
self.assertEqual(self.reduce(add, d), "".join(d.keys()))

# test correctness of keyword usage of `initial` in `reduce`
def test_initial_keyword(self):
def add(x, y):
return x + y
self.assertEqual(
self.reduce(add, ['a', 'b', 'c'], ''),
self.reduce(add, ['a', 'b', 'c'], initial=''),
)
self.assertEqual(
self.reduce(add, [['a', 'c'], [], ['d', 'w']], []),
self.reduce(add, [['a', 'c'], [], ['d', 'w']], initial=[]),
)
self.assertEqual(
self.reduce(lambda x, y: x*y, range(2,8), 1),
self.reduce(lambda x, y: x*y, range(2,8), initial=1),
)
self.assertEqual(
self.reduce(lambda x, y: x*y, range(2,21), 1),
self.reduce(lambda x, y: x*y, range(2,21), initial=1),
)
self.assertRaises(TypeError, self.reduce, add, [0, 1], initial="")
self.assertEqual(self.reduce(42, "", initial="1"), "1") # func is never called with one item


@unittest.skipUnless(c_functools, 'requires the C _functools module')
class TestReduceC(TestReduce, unittest.TestCase):
Expand Down
1 change: 1 addition & 0 deletions Misc/ACKS
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,7 @@ Luke Dunstan
Virgil Dupras
Bruno Dupuis
Andy Dustman
Sayandip Dutta
Gary Duzan
Eugene Dvurechenski
Karmen Dykstra
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Allow the *initial* argument of :func:`functools.reduce` to be a keyword.
Patch by Sayandip Dutta.
39 changes: 22 additions & 17 deletions Modules/_functoolsmodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -932,15 +932,30 @@ _functools_cmp_to_key_impl(PyObject *module, PyObject *mycmp)

/* reduce (used to be a builtin) ********************************************/

// Not converted to argument clinic, because of `args` in-place modification.
// AC will affect performance.
/*[clinic input]
_functools.reduce

function as func: object
iterable as seq: object
/
initial as result: object(c_default="NULL") = None

Apply a function of two arguments cumulatively.

Apply it to the items of a sequence or iterable, from left to right, so as to
reduce the iterable to a single value. For example, reduce(lambda x, y: x+y,
[1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). If initial is present, it is
placed before the items of the iterable in the calculation, and serves as a
default when the iterable is empty.
[clinic start generated code]*/

static PyObject *
functools_reduce(PyObject *self, PyObject *args)
_functools_reduce_impl(PyObject *module, PyObject *func, PyObject *seq,
PyObject *result)
/*[clinic end generated code: output=30d898fe1267c79d input=b7082b8b1473fdc2]*/
{
PyObject *seq, *func, *result = NULL, *it;
PyObject *args, *it;

if (!PyArg_UnpackTuple(args, "reduce", 2, 3, &func, &seq, &result))
return NULL;
if (result != NULL)
Py_INCREF(result);

Expand Down Expand Up @@ -1006,16 +1021,6 @@ functools_reduce(PyObject *self, PyObject *args)
return NULL;
}

PyDoc_STRVAR(functools_reduce_doc,
"reduce(function, iterable[, initial], /) -> value\n\
\n\
Apply a function of two arguments cumulatively to the items of a sequence\n\
or iterable, from left to right, so as to reduce the iterable to a single\n\
value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates\n\
((((1+2)+3)+4)+5). If initial is present, it is placed before the items\n\
of the iterable in the calculation, and serves as a default when the\n\
iterable is empty.");

/* lru_cache object **********************************************************/

/* There are four principal algorithmic differences from the pure python version:
Expand Down Expand Up @@ -1720,7 +1725,7 @@ PyDoc_STRVAR(_functools_doc,
"Tools that operate on functions.");

static PyMethodDef _functools_methods[] = {
{"reduce", functools_reduce, METH_VARARGS, functools_reduce_doc},
_FUNCTOOLS_REDUCE_METHODDEF
_FUNCTOOLS_CMP_TO_KEY_METHODDEF
{NULL, NULL} /* sentinel */
};
Expand Down
73 changes: 72 additions & 1 deletion Modules/clinic/_functoolsmodule.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading