Skip to content

stdlib: adds itertools, composable templates for inline iterators#25680

Open
ZoomRmc wants to merge 5 commits intonim-lang:develfrom
ZoomRmc:itertools
Open

stdlib: adds itertools, composable templates for inline iterators#25680
ZoomRmc wants to merge 5 commits intonim-lang:develfrom
ZoomRmc:itertools

Conversation

@ZoomRmc
Copy link
Copy Markdown
Contributor

@ZoomRmc ZoomRmc commented Mar 28, 2026

This PR implements RFC #562 , it was given a green light long ago.

The code is based on beef's idea and impl in slicerator/itermacros.

For convenient syntax that allows writing "foo".split.mapIt(it).collect() and not mapIt("foo".split, it).collect()" it requires merging #25679 (or an alternative fix) but it doesn't rely on it inherently (except some examples and test cases assume this form being available), so workable even without it.

This example works now:

import itertools
import std/[strutils, options]

# Find the first element in a sequence of transformed numbers above 35.
# Note using `Slice[int].items` instead of `CountUp` (also supported).
doAssert (-25..25).items.mapIt(it * 10 div 7).findIt(it > 35) == none(int)

# Filter philosophers by country, compose sentences, join to a string.
let philosophers = {
  "Plato": "Greece", "Aristotle": "Greece", "Socrates": "Greece",
  "Confucius": "China", "Descartes": "France"}

const Phrase = "$1 is a famous philosopher from $2."
let facts = philosophers.items()
              .filterIt(it[1] != "Greece")
              .mapIt([it[0], it[1]])
              .mapIt(Phrase % it)
              .foldIt("", acc & it & '\n')
doAssert facts == """
Confucius is a famous philosopher from China.
Descartes is a famous philosopher from France.
"""

The implementation is straightforward, API is bog-standard and shouldn't be surprising to anyone familiar with this style, tests cover everything. The only interesting thing in the code is that it uses concepts for overloads without any when compiles hacks, feels pretty great.

The set of provided templates is basic and the naming is all common, so I hope there's not much place for bikeshedding. I've been craving for this feature since 2018 or so and didn't hear much pushback against the idea, so I hope this PR doesn't go to waste.

requires #25679

@ZoomRmc
Copy link
Copy Markdown
Contributor Author

ZoomRmc commented Mar 28, 2026

Test failures all should get green after #25679.

@arnetheduck
Copy link
Copy Markdown
Contributor

Very nice, specially that things compose and don't have to be materialized!

  1. Can closure iterators be supported as well out of the box via some standard helper? feels like a pretty sad omission
  2. does the whole thing evaluate to an iterator itself, ie does for x in (-25..25).items.mapIt(it * 10 div 7): echo x work?

@ZoomRmc
Copy link
Copy Markdown
Contributor Author

ZoomRmc commented Apr 1, 2026

Thanks!

1. Can closure iterators be supported as well out of the box via some standard helper? feels like a pretty sad omission

Well, it's a language omission first (iterator (): T isnot iterable[T]) and I don't think it doable now without instantiating an inline iterator that uses the closure first.

This works:

  let infinite = iterator(): int =
     while true: yield 1

  template asIterable[T](it: iterator(): T {.closure.}): untyped =
    iterator wrap(): T =
      for x in it():
        yield x
    wrap()

  var sum = 0
  for i in infinite.asIterable().take(3):
    sum.inc i

  doAssert sum == 3

We could provide such template in the module, but it's a hack and we should just make iterator {.closure.} satisfy iterable[T].

2. does the whole thing evaluate to an iterator itself, ie  does `for x in (-25..25).items.mapIt(it * 10 div 7): echo x` work?

Absolutely! Otherwise, the templates wouldn't compose. Added a test case for the chain in the regular iterator context (the for loop).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants