Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions base/generator.jl
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,24 @@ This means that most iterators are assumed to implement [`length`](@ref).
This trait is generally used to select between algorithms that pre-allocate space for their
result, and algorithms that resize their result incrementally.

Most iterators' `IteratorSize` category can be determined from their type. In this case
the generic `IteratorSize(iterator) = IteratorSize(typeof(iterator))` fallback applies.
However, some iterators (e.g. [`Iterators.cycle`](@ref)) have a size category that can only
be determined at runtime. In this case the `IteratorSize(itr)` method will not be type
stable and the `IteratorSize(::Type)` method will return `SizeUnkown`.

```jldoctest
julia> Base.IteratorSize(1:5)
Base.HasShape{1}()

julia> Base.IteratorSize((2,3))
Base.HasLength()

julia> Base.IteratorSize(Iterators.cycle(1:5))
Base.IsInfinite()

julia> Base.IteratorSize(Iterators.cycle(1:0))
Base.HasLength()
```
"""
IteratorSize(x) = IteratorSize(typeof(x))
Expand Down
8 changes: 7 additions & 1 deletion base/iterators.jl
Original file line number Diff line number Diff line change
Expand Up @@ -988,7 +988,13 @@ cycle(xs, n::Integer) = flatten(repeated(xs, n))

eltype(::Type{Cycle{I}}) where {I} = eltype(I)
IteratorEltype(::Type{Cycle{I}}) where {I} = IteratorEltype(I)
IteratorSize(::Type{Cycle{I}}) where {I} = IsInfinite() # XXX: this is false if iterator ever becomes empty
function IteratorSize(::Type{Cycle{I}}) where I
# TODO: find a better way of communicating the size of a cycle
# IsInfinite() would be false if iterator ever becomes empty
IteratorSize(I) === IsInfinite() ? IsInfinite() : SizeUnknown()
end
IteratorSize(it::Cycle) = isempty(it.xs) ? HasLength() : IsInfinite()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have other examples where IteratorSize(it) and IteratorSize(typeof(it)) (potentially) disagree? Makes me a bit uncomfortable.

And if it.xs is stateful, it seems that IsInfinite might turn out to be wrong, because isempty(it.xs) might become true.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have other examples where IteratorSize(it) and IteratorSize(typeof(it)) (potentially) disagree? Makes me a bit uncomfortable.

No, this is the first case, and changes the docs to suggest the possibility.

And if it.xs is stateful, it seems that IsInfinite might turn out to be wrong, because isempty(it.xs) might become true.

IMO that's because Iterators.cycle on stateful iterators is broken. It is explicitly documented to be infinite for non-empty arguments.

Copy link
Contributor

@blegat blegat Sep 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am facing the same issue in JuliaAlgebra/MultivariatePolynomials.jl#343. There are probably many cases where an iterator is infinite except in degenerate cases. We could also document that when an iterator type says that it is IsInfinite, it means that it may be infinite and calling IteratorSize on an instance may (it could also be difficult to know in advance whether it is infinite, say for instance that you define an iterator for this iteration that stops when it reaches 1) distinguish whether it is infinite or not. These iterators are also said to be infinite but they may still end if the corresponding input closes I guess.

If we decide that IsInfinite only means "may be infinite", then the IteratorSize(::Type{Cycle}) should probably return IsInfinite in all cases, and never SizeUnknown.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thoughts regarding defining IteratorSize on instances of a type:

  • I interpret the iteration interface like so:

    • Callers of IteratorSize(::Any) should be able to assume the call is "free", in the sense it gets constant folded and does not exist at run time when everything is concretely inferred.

    • The intention behind the IteratorSize(x) = IteratorSize(typeof(x)) is as a mere convenience for making it unnecessary to call typeof. So this method is supposed to be the only method of IteratorSize that accepts non-Type.

    • Callers should be able to assume IteratorSize(x) === IteratorSize(typeof(x)), if !isa(x, Type).

    • If one wants a trait like IteratorSize, but defined on instances, they should create a new function instead of adding methods to IteratorSize.

  • Another point possibly worth considering is abstract inference: when the argument types to a call are not concretely known, Julia will possibly consider a set of more than one matching method. Thus adding a methods to a callable can negatively affect the code generation (and affinity to invalidation) for unrelated loaded packages. This is especially bad if the method is of an uncommon type.

length(it::Cycle) = isempty(it.xs) ? 0 : throw(ArgumentError("Cannot compute length of infinite iterator"))

iterate(it::Cycle) = iterate(it.xs)
isdone(it::Cycle) = isdone(it.xs)
Expand Down
12 changes: 12 additions & 0 deletions test/iterators.jl
Original file line number Diff line number Diff line change
Expand Up @@ -1049,3 +1049,15 @@ end
@testset "Iterators docstrings" begin
@test isempty(Docs.undocumented_names(Iterators))
end

@testset "cycle IteratorSize (#53169)" begin
@test Base.IteratorSize(Iterators.cycle(1:3)) == Base.IsInfinite()
@test Base.IteratorSize(Iterators.cycle(1:0)) == Base.HasLength()
@test length(Iterators.cycle(1:0)) == 0
@test collect(Iterators.cycle(1:0))::Vector{Int} == Int[]
@test Base.IteratorSize(Iterators.cycle(Iterators.Repeated(6))) == Base.IsInfinite()

@test Base.IteratorSize(typeof(Iterators.cycle(1:3))) == Base.SizeUnknown()
@test Base.IteratorSize(typeof(Iterators.cycle(1:0))) == Base.SizeUnknown()
@test Base.IteratorSize(typeof(Iterators.cycle(Iterators.Repeated(6)))) == Base.IsInfinite()
end