Update for Base.stack, Julia 1.9 (#14)

* update for Base.stack

* don't test on 1.3

* tidy

Author: mcabbott

.github/workflows/ci.yaml

@@ -16,7 +16,7 @@ jobs:
       fail-fast: false
       matrix:
         version:
-          - '1.3'
+          - '1.6'
           - '1' # Leave this line unchanged. '1' will automatically expand to the latest stable 1.x release of Julia.
           - 'nightly'
         os:

.travis.yml

@@ -1,23 +1,26 @@
 name = "LazyStack"
 uuid = "1fad7336-0346-5a1a-a56f-a06ba010965b"
 authors = ["Michael Abbott"]
-version = "0.0.8"
+version = "0.1.0"
 
 [deps]
 ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
+Compat = "34da2185-b29b-5c13-b0c7-acf172513d20"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
-NamedDims = "356022a1-0364-5f58-8944-0da4b18d706f"
-OffsetArrays = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
+# NamedDims = "356022a1-0364-5f58-8944-0da4b18d706f"
+# OffsetArrays = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
 
 [compat]
-ChainRulesCore = "0.10.9, 1"
-NamedDims = "0.2.16, 0.3, 1.0"
-OffsetArrays = "1"
-julia = "1.3"
+ChainRulesCore = "1"
+Compat = "3.46, 4.2"
+# NamedDims = "0.2.16, 0.3, 1.0"  # try to remove
+# OffsetArrays = "1"  # try to remove
+julia = "1.6"
 
 [extras]
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
+OffsetArrays = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
 Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
 
 [targets]
-test = ["Test", "Zygote"]
+test = ["Test", "OffsetArrays", "Zygote"]

Project.toml

@@ -1,67 +1,72 @@
 # LazyStack.jl
 
-[![Travis CI](https://travis-ci.org/mcabbott/LazyStack.jl.svg?branch=master)](https://travis-ci.org/mcabbott/LazyStack.jl)
 [![Github CI](https://github.com/mcabbott/LazyStack.jl/workflows/CI/badge.svg)](https://github.com/mcabbott/LazyStack.jl/actions?query=workflow%3ACI+branch%3Amaster)
 
-This package exports one function, `stack`, for turning a list of arrays 
+This package exports one function, `lazystack`, for turning a list of arrays 
 into one `AbstractArray`. Given several arrays with the same `eltype`, 
 or an array of such arrays, it returns a lazy `Stacked{T,N}` view of these:
 
 ```julia
-stack([zeros(2,2), ones(2,2)])  # isa Stacked{Float64, 3, <:Vector{<:Matrix}}
-stack([1,2,3], 4:6)             # isa Stacked{Int, 2, <:Tuple{<:Vector, <:UnitRange}}
+julia> lazystack([1:2, 3:4, 5:6])
+2×3 lazystack(::Vector{UnitRange{Int64}}) with eltype Int64:
+ 1  3  5
+ 2  4  6
+
+julia> lazystack([pi^ℯ], [ℯ^pi])
+1×2 lazystack(::Tuple{Vector{Float64}, Vector{Float64}}) with eltype Float64:
+ 22.4592  23.1407
 ```
 
-Given a generator, it instead iterates through the elements and writes into a new array.
-Given a function and then some arrays, it behaves like `map(f, A, B)` but immediately writes
-into a new array:
+Before v0.1 this function used to be called `stack`, but that name is now exported by Base (from Julia 1.9).
+Like this package, `Base.stack` makes an array with `size(result) = (size(inner)..., size(outer)...)`.
+It always returns a new dense array, not a lazy container.
+And instead of two vectors (in the above example) it would want a tuple `stack(([pi^ℯ], [ℯ^pi]))`.
 
-```julia
-stack([i,2i] for i in 1:5)            # isa Matrix{Int}     # size(ans) == (2, 5)
-stack(*, eachcol(ones(2,4)), 1:4)     # == Matrix(stack(map(*, eachcol(...), 1:4)))
-```
-
-The same `stack_iter` method is also used for any list of arrays of heterogeneous element type,
-and for arrays of tuples. Notice that like `map(identity, Any[1, 1.0, 5im])`, this promotes using 
-`promote_typejoin`, to `Number` here, rather than to `Complex{Float64}`:
-
-```julia
-stack([1,2], [3.0, 4.0], [5im, 6im])  # isa Matrix{Number}  # size(ans) == (2, 3)
-stack([(i,2.0,3//j) for i=1:4, j=1:5])# isa Array{Real, 3}  # size(ans) == (3, 4, 5)
-```
+Generators such as `lazystack([i,2i] for i in 1:5)` and arrays of mixed eltype like `lazystack([1,2], [3.0, 4.0], [5im, 6im])` used to be be handled here, making a dense array, but are now simply passed through to `Base.stack`.
 
-The slices must all have the same `size`, but they (and the container) 
-can have any number of dimensions. `stack` always places the slice dimensions first.
-There are no options.
+When the individual slices aren't backed by an `Array`, as for instance with `CuArray`s on a GPU, then again `Base.stack` is called. 
+This should make one big `CuArray`, since scalar indexing of individual slices won't work well.
 
 ### Ragged stack
 
-There is also a version which does not demand that slices have equal `size` (or equal `ndims`),
-which always returns a new `Array`. You can control the position of slices `using OffsetArrays`:
+There is also a version which does not demand that slices have equal `size` (or equal `ndims`).
+For now this is not lazy:
 
 ```julia
-rstack([1:n for n in 1:10])           # upper triangular Matrix{Int}
-rstack(OffsetArray(fill(n,4), rand(-2:2)) for n in 1:10; fill=NaN)
+julia> raggedstack([10:10+n for n in 1:3])
+4×3 Matrix{Int64}:
+ 10  10  10
+ 11  11  11
+  0  12  12
+  0   0  13
+
+julia> using OffsetArrays
+
+julia> raggedstack(OffsetArray(fill(1.0n, 3), rand(-1:1)) for n in 1:10; fill=NaN)
+5×10 OffsetArray(::Matrix{Float64}, 0:4, 1:10) with eltype Float64 with indices 0:4×1:10:
+ NaN      2.0  NaN      4.0  NaN      6.0    7.0  NaN      9.0  NaN
+   1.0    2.0    3.0    4.0    5.0    6.0    7.0  NaN      9.0   10.0
+   1.0    2.0    3.0    4.0    5.0    6.0    7.0    8.0    9.0   10.0
+   1.0  NaN      3.0  NaN      5.0  NaN    NaN      8.0  NaN     10.0
+ NaN    NaN    NaN    NaN    NaN    NaN    NaN      8.0  NaN    NaN
 ```
 
 ### Other packages
 
-This one plays well with [OffsetArrays.jl](https://github.com/JuliaArrays/OffsetArrays.jl),
-[NamedDims.jl](https://github.com/invenia/NamedDims.jl), and 
-[Zygote.jl](https://github.com/FluxML/Zygote.jl).
+This one plays well with [OffsetArrays.jl](https://github.com/JuliaArrays/OffsetArrays.jl), and [ChainRules.jl](https://github.com/JuliaDiff/ChainRules.jl)-compatible AD such as [Zygote.jl](https://github.com/FluxML/Zygote.jl). It's also used internally by [TensorCast.jl](https://github.com/mcabbott/TensorCast.jl).
 
 Besides which, there are several other ways to achieve similar things:
 
 * For an array of arrays, you can also use [`JuliennedArrays.Align`](https://bramtayl.github.io/JuliennedArrays.jl/latest/#JuliennedArrays.Align). This requires (or enables) you to specify which dimensions of the output belong to the sub-arrays, instead of writing `PermutedDimsArray(stack(...), ...)`. 
-* There is also [`RecursiveArrayTools.VectorOfArray`](https://github.com/JuliaDiffEq/RecursiveArrayTools.jl#vectorofarray) which as its name hints only allows a one-dimensional container. Linear indexing retreives a slice, not an element, which is sometimes surprising.
+* There is also [`RecursiveArrayTools.VectorOfArray`](https://github.com/JuliaDiffEq/RecursiveArrayTools.jl#vectorofarray) which as its name hints only allows a one-dimensional container. (And unlike the package name, nothing is recursive.) Linear indexing retreives a slice, not an element, which is sometimes surprising.
 * For a tuple of arrays, [`LazyArrays.Hcat`](https://github.com/JuliaArrays/LazyArrays.jl#concatenation) is at present faster to index than `stack`, but doesn't allow arbitrary dimensions.
-* For a generator of arrays, the built-in `reduce(hcat,...)` may work, but it slow compared to `stack`: see [test/speed.jl](test/speed.jl) for some examples.
 
 And a few more:
 
 * When writing this I missed [`SplitApplyCombine.combinedimsview`](https://github.com/JuliaData/SplitApplyCombine.jl#combinedimsviewarray), which is very similar to `stack`, but doesn't handle tuples.
 * Newer than this package is [StackViews.jl](https://github.com/JuliaArrays/StackViews.jl) handles both, with `StackView(A,B,dims=4) == StackView([A,B],4)` creating a 4th dimension; the container is always one-dimensional. 
 * [`Flux.stack`](https://fluxml.ai/Flux.jl/stable/utilities/#Flux.stack) similarly takes a dimension, but eagerly creates an `Array`.
+* Finally, [CatViews.jl](https://github.com/ahwillia/CatViews.jl) offers a lazy `vcat`. But the package is old and I think not so fast.
 
 The lazy inverse:
 
@@ -71,10 +76,10 @@ The lazy inverse:
 
 * As does [`PackedVectorsOfVectors`](https://github.com/synchronoustechnologies/PackedVectorsOfVectors.jl), although only 1+1 dimensions. Also has an eager `pack` method which turns a vector of vectors into view of a single larger matrix. 
 
-* [`Base.eachslice`](https://docs.julialang.org/en/v1/base/arrays/#Base.eachslice) also views one large array as many slices. This is a generator, but [JuliaLang#32310](https://github.com/JuliaLang/julia/pull/32310) should upgrade it to a multi-dimensional container indexable container.
+* [`Base.eachslice`](https://docs.julialang.org/en/v1/base/arrays/#Base.eachslice) also views one large array as many slices. This was a generator, but [JuliaLang#32310](https://github.com/JuliaLang/julia/pull/32310) upgrades it to a multi-dimensional indexable container, in Julia 1.9.
 
 Eager:
 
 * After writing this I learned of [JuliaLang#31644](https://github.com/JuliaLang/julia/pull/31644) which extends `reduce(hcat,...)` to work on generators. 
 
-* Later, [JuliaLang#31644](https://github.com/JuliaLang/julia/pull/43334) proposes to add the eager `stack_iter` method of this package to Base.
+* Later, [JuliaLang#43334](https://github.com/JuliaLang/julia/pull/43334) has added a better version of this package's `stack_iter` method to Base.