Support `enumerate` with `reversed` function

[muddyfish]

Feature or enhancement

Proposal:

Currently, the reversed function doesn't support enumerate objects. This would be useful any time you want to traverse a list backwards whilst keeping indexes.

The implementation would ensure that enumerate objects support reversal only if the iterable argument also supports reversal and len.

lst = ["foo", "bar"]
for i, data in reversed(enumerate(lst)):
    print(i, data)

Currently, what I see being done is something like:

for i in range(len(lst)-1, -1, -1):
  data = lst[i]

Which is less readable.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

[skirpichev]

IIUC, you want:

>>> for i in range(len(lst)-1, -1, -1):
...    print(i, lst[i])
...    
1 bar
0 foo

Another solution is:

>>> lst = ["foo", "bar"]
... for i, data in reversed(list(enumerate(lst))):
...     print(i, data)
...     
1 bar
0 foo

But I doubt that you idea can be realized for all iterators, see e.g. #63026. I think this issue should be closed.

CC @vstinner

[muddyfish]

Using reversed(list(enumerate(lst))) materialises a new list, and isn't ideal for ease of writing or memory usage.

I know that there are workarounds possible, however what I was requesting was that in the cases where the underlying object passed to enumerate supports reversed (__reversed__, or __getitem__ and __len__ implemented), enumerate should also support reversed (additional constraint that __len__ is implemented).

This would be possible by implementing a __reversed__ method on enumerate objects which do the proper validity checks at runtime

[kacpereqo]

Using reversed(list(enumerate(lst))) materialises a new list, and isn't ideal for ease of writing or memory usage.

I know that there are workarounds possible, however what I was requesting was that in the cases where the underlying object passed to enumerate supports reversed (__reversed__, or __getitem__ and __len__ implemented), enumerate should also support reversed (additional constraint that __len__ is implemented).

This would be possible by implementing a __reversed__ method on enumerate objects which do the proper validity checks at runtime

How reversed should work on enumerating infinite structures?

def foo():
    temp = 0
    while True:
        yield temp
        temp += 1


for idx, num in enumerate(foo()):
    print(idx, num)

[muddyfish]

foo() doesn't support __len__ and either __getitem__ or __reversed__, so this should raise an exception like it does today.

[muddyfish]

Some pseudo code would be something like:

class enumerate:
  def __init__(self, it):
    self.it = it
  def __reversed__(self):
    total = len(self.it)
    for val in reversed(self.it):
      total -= 1
      yield total, val

The point here is that enumerate doesn't do anything special to the iterable that stops it being reversible aside from adding another constraint that it must be len-able

[kacpereqo]

that's make sense, cool

[sobolevn]

What would you get from list(reversed(enumerate(range(3))))? Options:

1. [(0, 2), (1, 1), (2, 1)]
2. [(2, 2), (1, 1), (0, 0)]

Why can't we simply use this code to achieve the same?

>>> it = range(3)
>>> len_it = len(it)
>>> for i, o in enumerate(reversed(it)):
...    print('index', i)
...    print('reverse index', len_it - i - 1)
...    print('object', o)
...    print()
... 
index 0
reverse index 2
object 2

index 1
reverse index 1
object 1

index 2
reverse index 0
object 0

[muddyfish]

You'd get option 2, as otherwise you'd do enumerate(reversed(...)).

You could do what you're suggesting in that code block, but it's harder to work out what's going on at a glance. However that approach wouldn't work if you wanted to do further in-line calls, such as itertools.islice(reversed(enumerate(...))).

[terryjreedy]

I think that this request that enumerate forward the issue of reversability to its input iterable, as in #124178 (comment), is reasonable. If the user knows that iter cannot be reversed, reversed(enumerate(list(iter))) would generally be more space efficient than `reversed(list(enumerate(iter))).

Adding enumerate.reversed would raise the issue of whether we should do the same for other built-in functions of iterables. How about filter? For this, listing an non-reversable input iterable first would take more space than listing the filtered result, but much more than if the input iterable is reversable. But here, filter(f, reversed(iter)) is the same as would be reversed(filter(f, iter)), so no need for the addition.

There is also a question of how to document. Dicts and their views are documented as reversable; lists and tuples are not, probably as it is obvious. Perhaps adding "Enumerations are reversable with reversed() if the input iterable is." would be enough.

[rhettinger]

I'm -1 on this one.

A few thoughts:

• The use case is not common. We barely had enough use cases to warrant adding reversed() at all. (IIRC, Guido said something like, "looping forward is more common by a factor of thousands"). So of course, the intersection of the need to reverse and enumerate at the same time would be even more rare.
• People benefit by having a simple mental model for their tooling. Currently, enumerate and reversed both have simple models. Reaching through an input iterator to access something other than iter(iterable) is surprising and out of line with almost any of our other tooling.
• I've previously experimented with "iterator transparency" (i.e. passing through a length-hint) and quickly found myself on thin ice where it was hard to avoid bugs and/or surprising behavior.
• I can't say that I really like zip(reversed(range(len(iterable))), reversed(iterable)) but it does demonstrate that the need can easily be met with our current offerings.
• It is really helpful to users that the code equivalent in the docs is simple, easily digested, and easily customized to other use cases. I don't think we should give up that simplicity.
• You can check with Jelle, but I suspect this would complexify the type annotations, making editor tool tips harder to understand and less useful.
• As a reviewer, I would have to stop and think carefully about this every time I saw it. It wouldn't make my life easier.
• Lastly, Python is a language that mostly avoids implicit optimizations that cross API boundaries. We've assiduously avoided the idea that "we could do this" implies "we should do this".

[serhiy-storchaka]

One issue is that implementing __reversed__ would make isinstance(enumerate, collections.abc.Reversible) returning true even if the enumerate iterator is not reversible in general. There will also be similar issues in static typing -- if the parameter is declared as Reversible, should it accept enumerate?

Note also that the enumerate object is an iterator, not just iterable. Do you know other examples of reversible iterators?

[muddyfish]

At least in terms of use cases, I think this is the second time I've wanted to recommend it for the simple case on lists in the last 3 months or so.

I disagree that it makes the mental model more complicated to use, at least in the simple case where it's used with lists, as reversing a list is something that's obviously possible. I also wasn't aware that enumerate returned an iterator rather than an iterable, which does make it more weird to handle.

With regards to typing, I don't think Python's current model for annotations would make this easy to express.

[brettcannon]

I'm -1 on this. I don't think it's common enough and enumerate() is designed to work w/ infinite iterators which this won't work with. I also don't think working around it by getting a concrete data structure upfront is arduous.