Surfacing iterables in argument names and docstrings

[sogaiu]

As a follow-up to this recent comment, I've started going through boot.janet looking for argument names and docstrings that might benefit from making it explicit that an iterable is involved. Please see the parent issue for a bit more background.

Since empty? now uses iter instead of xs and this change was seen as worthwhile, it seems possible that other functions / macros might be improved in a similar fashion.

One caveat might be if a fiber doesn't stop yielding many (all?) of the functions / macros may not terminate in some cases (at least not normally?).

I'll try to report in this issue what I find. Additional help and comments welcome :)

[sogaiu]

The following work with at least fibers:

• distinct
• geomean
• mean
• product
• sum

They all use xs in their docstrings, e.g.

(geomean xs)
    
Returns the geometric mean of xs. If empty, returns NaN.

Except for distinct, they also all use the word empty in their docstrings.

[sogaiu]

The following also seem to work with fibers:

• accumulate
• accumulate2
• all
• count
• filter
• keep
• map
• mapcat
• reduce
• reduce2
• some

Note that these all use ind in their signatures and docstrings.

[sogaiu]

Although it might look like:

• find-index
• index-of

work with fibers, in practice, because of fiber behavior with respect to in, they don't really produce results one might want:

(find-index even?
            (coro (yield 1) (yield 3) (yield 2)))
# =>
0

(index-of 2
          (coro (yield 1) (yield 3) (yield 2)))
# =>
0

It looks like find might be ok though:

(find even?
      (coro (yield 1) (yield 3) (yield 2)))
# =>
2

[amano-kenji]

Fiber is a single-use iterable. Tuple, array, table, and struct are re-iterables. The distinction between single-use iterable data types and re-iterable data types is important. Basilisp's iterator-seq function distinguishes single-use iterables(iterators) and re-iterables.

Basilisp's iterator-seq is based on python's iter function which produces an iterator from an iterable. You can call next on an iterator.

Docstring and the main documentation should not confuse the two.

It seems I should have mentioned empty? is not for single-use iterables like fibers... empty? resumes fiber.... once.

In java, there is a distinction between iterable and iterator. An iterable type can produce an iterator which can only be traversed once with next method. Perhaps, janet can make the distinction between iterable and iterator?

In janet, you don't exactly produce an iterator from tuple or array, but they are certainly re-iterable. Perhaps, we should call fibers single-use iterables rather than iterators.

[amano-kenji]

I think single-use iterable is too long. We should just call it iterable. Let's say janet has iterable types and re-iterable types.

Then, function argument names can be iter and riter.

[sogaiu]

I presume by "single-use", one doesn't mean "can only call resume once" but rather, the traversal through the iterable cannot be performed a second time.

For "iterator", may be "single-use" is ok [1], but "single-use" doesn't seem like a good choice to be combining with the term "iterable".

I think the distinction of whether a traversal can be performed more than once may be a relevant one to make, but the phrase "single-use iterable" might sound a bit like you can only use it one time, which may not be the case fairly often (because typically resume can be called more than once with a fiber -- ofc, not always).

Also, fibers don't necessarily "end" -- specific fibers may be "infinite", in that some may be implemented to keep yielding each time they are resumed (e.g. generating random values). This could lead to another sort of confusion in combination with the phrase "single-use".

On a related note, fibers are not the only things that have this property of "can be traversed at most once" [2].

Some abstract types can be written to be that way too.

---

[1] I'm not sure it is TBH.
[2] "Single-pass" and "input" might be alternative candidate terms that are less confusing. See here, here, and here for some background.

[amano-kenji]

That's why we should make the distinction between iterable and re-iterable?

[sogaiu]

I don't disagree that a distinction could be worthwhile:

I think the distinction of whether a traversal can be performed more than once may be a relevant one to make

I'm not into using the term "single-use" though.

I don't know yet about "iterable" vs "re-iterable". Will think about it. The term "iterable" is already used in Janet source code and the docs, though not much. I would interpret "iterable" to not imply "single-pass" but rather "unspecified".

In the mean time there is plenty to go over in boot.janet. Identifying where changes might be made can be done without settling on the specific terms to use.

[amano-kenji]

I think it's okay to see re-iterable as a subset of iterable.

[sogaiu]

Sure.

The question remains though what to do about naming "single-pass" or "input" types of iterables.

I'm not in a hurry though. There are quite a few functions identified above and there may still be more. Perhaps some good ideas will come about while investigating boot.janet.

[amano-kenji]

What is input iterable?

[sogaiu]

The following is from an earlier comment. The links point to relevant information:

[2] "Single-pass" and "input" might be alternative candidate terms that are less confusing. See here, here, and here for some background.

FWIW, "input iterable" may not be a term in use -- it is based on "input iterator".

[amano-kenji]

According to Output Iterators in C++, output iterators are also single-pass iterators.

In different languages, input/output iterators can be multi-pass iterators.

I think single-pass iterable and multi-pass iterable are more accurate. Perhaps, their function argument names can be siter and miter. If the distinction is not important, you can just call it iter.

[sogaiu]

ATM I prefer the names "single-pass iterable" and "multi-pass iterable" over "input iterable" and "output iterable" [1]. The latter names seem less clear to me while the former ones seem like things I might be more likely to guess appropriate meanings from. However, I find "multi-pass" a bit strange. The term "re-iterable" that you suggested before seems better to me.

An alternative to using siter and miter as argument names is to just use iter and to make a distinction if needed in the docstring.

---

In any case, as I said before, IMO there is no urgency. Whatever is discussed now may help, but further ideas may appear down the line. I don't think it makes sense to try to commit to something at the moment as there are plenty of things to review yet in boot.janet.

It could also be that as additional items turn up there are additional concerns that are identified...just like what happened in this case. It might have seemed like things were conclusive from examining empty? on its own but when we went looking further it turned out there was more to the story.

I think another concern may have been identified above -- that of whether the iterable "halts":

Also, fibers don't necessarily "end" -- specific fibers may be "infinite", in that some may be implemented to keep yielding each time they are resumed (e.g. generating random values).

May be there are at least three kinds of fibers:

• those that always have a last value
• those that never have a last value
• those that sometimes have a last value

Some (most?) of the functions listed above [2] won't return if the passed in iterable doesn't "have a last value".

---

[1] I don't think we have iterating constructs in Janet that correspond to the "output" type. If that's correct, the term doesn't seem like it would be a candidate name for us.

[2] These and these.

[sogaiu]

Making a note that what is discovered / explored here and elsewhere can inform this issue regarding documenting iterable at the janet-lang.org repository.

Possibly working on that in parallel could be helpful.