Skip to content

Make IteratorSize(::Iterators.Cycle) not always IsInfinite add to docstring as example #54187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Make IteratorSize(::Iterators.Cycle) not always IsInfinite add to docstring as example #54187

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
63f3575
make `IteratorSize(::Iterators.Cycle)` not always `IsInfinite` and do…
LilithHafner Apr 22, 2024
7b0eba0
Fix typo
LilithHafner Apr 22, 2024
f7bf6f1
Apply triage's suggestion
LilithHafner Sep 25, 2025
42b6c7b
Fixup jldoctest
LilithHafner Sep 25, 2025
7c19b7f
Know that `HasShape{0}()` is not empty
LilithHafner Sep 25, 2025
eaad675
Test that HasShape
LilithHafner Sep 25, 2025
43fb545
Merge branch 'master' into lh/unknown-cycle-size
LilithHafner Sep 25, 2025
356a868
Merge branch 'master' into lh/unknown-cycle-size
LilithHafner Oct 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions base/generator.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,24 @@ This means that most iterators are assumed to implement [`length`](@ref).
This trait is generally used to select between algorithms that pre-allocate space for their
result, and algorithms that resize their result incrementally.

Most iterators' `IteratorSize` category can be determined from their type. In this case
the generic `IteratorSize(iterator) = IteratorSize(typeof(iterator))` fallback applies.
However, some iterators (e.g. [`Iterators.cycle`](@ref)) have a size category that can only
be determined at runtime. In this case the `IteratorSize(itr)` method will not be type
stable and the `IteratorSize(::Type)` method will return `SizeUnknown`.

```jldoctest
julia> Base.IteratorSize(1:5)
Base.HasShape{1}()

julia> Base.IteratorSize((2,3))
Base.HasLength()

julia> Base.IteratorSize(Iterators.cycle(1:5))
Base.IsInfinite()

julia> Base.IteratorSize(Iterators.cycle(1:0))
Base.HasLength()
```
"""
IteratorSize(x) = IteratorSize(typeof(x))
Expand Down
8 changes: 7 additions & 1 deletion base/iterators.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -988,7 +988,13 @@ cycle(xs, n::Integer) = flatten(repeated(xs, n))

eltype(::Type{Cycle{I}}) where {I} = eltype(I)
IteratorEltype(::Type{Cycle{I}}) where {I} = IteratorEltype(I)
IteratorSize(::Type{Cycle{I}}) where {I} = IsInfinite() # XXX: this is false if iterator ever becomes empty
function IteratorSize(::Type{Cycle{I}}) where I
# TODO: find a better way of communicating the size of a cycle
# IsInfinite() would be false if iterator ever becomes empty
IteratorSize(I) === IsInfinite() ? IsInfinite() : SizeUnknown()
end
IteratorSize(it::Cycle) = isempty(it.xs) ? HasLength() : IsInfinite()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have other examples where IteratorSize(it) and IteratorSize(typeof(it)) (potentially) disagree? Makes me a bit uncomfortable.

And if it.xs is stateful, it seems that IsInfinite might turn out to be wrong, because isempty(it.xs) might become true.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have other examples where IteratorSize(it) and IteratorSize(typeof(it)) (potentially) disagree? Makes me a bit uncomfortable.

No, this is the first case, and changes the docs to suggest the possibility.

And if it.xs is stateful, it seems that IsInfinite might turn out to be wrong, because isempty(it.xs) might become true.

IMO that's because Iterators.cycle on stateful iterators is broken. It is explicitly documented to be infinite for non-empty arguments.

Copy link
Contributor

@blegat blegat Sep 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am facing the same issue in JuliaAlgebra/MultivariatePolynomials.jl#343. There are probably many cases where an iterator is infinite except in degenerate cases. We could also document that when an iterator type says that it is IsInfinite, it means that it may be infinite and calling IteratorSize on an instance may (it could also be difficult to know in advance whether it is infinite, say for instance that you define an iterator for this iteration that stops when it reaches 1) distinguish whether it is infinite or not. These iterators are also said to be infinite but they may still end if the corresponding input closes I guess.

If we decide that IsInfinite only means "may be infinite", then the IteratorSize(::Type{Cycle}) should probably return IsInfinite in all cases, and never SizeUnknown.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thoughts regarding defining IteratorSize on instances of a type:

  • I interpret the iteration interface like so:

    • Callers of IteratorSize(::Any) should be able to assume the call is "free", in the sense it gets constant folded and does not exist at run time when everything is concretely inferred.

    • The intention behind the IteratorSize(x) = IteratorSize(typeof(x)) is as a mere convenience for making it unnecessary to call typeof. So this method is supposed to be the only method of IteratorSize that accepts non-Type.

    • Callers should be able to assume IteratorSize(x) === IteratorSize(typeof(x)), if !isa(x, Type).

    • If one wants a trait like IteratorSize, but defined on instances, they should create a new function instead of adding methods to IteratorSize.

  • Another point possibly worth considering is abstract inference: when the argument types to a call are not concretely known, Julia will possibly consider a set of more than one matching method. Thus adding a methods to a callable can negatively affect the code generation (and affinity to invalidation) for unrelated loaded packages. This is especially bad if the method is of an uncommon type.

length(it::Cycle) = isempty(it.xs) ? 0 : throw(ArgumentError("Cannot compute length of infinite iterator"))

iterate(it::Cycle) = iterate(it.xs)
isdone(it::Cycle) = isdone(it.xs)
Expand Down
12 changes: 12 additions & 0 deletions test/iterators.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1049,3 +1049,15 @@ end
@testset "Iterators docstrings" begin
@test isempty(Docs.undocumented_names(Iterators))
end

@testset "cycle IteratorSize (#53169)" begin
@test Base.IteratorSize(Iterators.cycle(1:3)) == Base.IsInfinite()
@test Base.IteratorSize(Iterators.cycle(1:0)) == Base.HasLength()
@test length(Iterators.cycle(1:0)) == 0
@test collect(Iterators.cycle(1:0))::Vector{Int} == Int[]
@test Base.IteratorSize(Iterators.cycle(Iterators.Repeated(6))) == Base.IsInfinite()

@test Base.IteratorSize(typeof(Iterators.cycle(1:3))) == Base.SizeUnknown()
@test Base.IteratorSize(typeof(Iterators.cycle(1:0))) == Base.SizeUnknown()
@test Base.IteratorSize(typeof(Iterators.cycle(Iterators.Repeated(6)))) == Base.IsInfinite()
end