Feature request: `cumsum`

[briochemc]

It's fairly common to take cumulative sums in Earth Science, but YAXArrays doesn't seem to cover this use case... MWE that throws:

a = YAXArray(rand(10, 20, 5))
b = mapslices(cumsum, a, dims="Dim_1") # all seems fine (because it's lazy)
b[1,1,1] # throws

Not sure what the issue is but this seems like a bug to me? (YAXArrays v0.7.0)

---

Edit: Adding full stack trace of the error:

ERROR: DimensionMismatch: cannot broadcast array to have fewer non-singleton dimensions
Stacktrace:
  [1] check_broadcast_shape
    @ ./broadcast.jl:547 [inlined]
  [2] check_broadcast_axes
    @ ./broadcast.jl:555 [inlined]
  [3] instantiate
    @ ./broadcast.jl:310 [inlined]
  [4] materialize!
    @ ./broadcast.jl:883 [inlined]
  [5] materialize!
    @ ./broadcast.jl:880 [inlined]
  [6] apply_function(f::DiskArrayEngine.ElementFunction{typeof(cumsum), DiskArrayEngine.NonMutating}, xout::Tuple{SubArray{…}}, xin::Tuple{SubArray{…}})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/zj17T/src/userfuncs.jl:87
  [7] innercode
    @ ~/.julia/packages/DiskArrayEngine/zj17T/src/runner.jl:35 [inlined]
  [8] run_block_single(loopRanges::Tuple{…}, f::DiskArrayEngine.ElementFunction{…}, inbuffers::Tuple{…}, outbuffers::Tuple{…})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/zj17T/src/runner.jl:50
  [9] run_block
    @ ~/.julia/packages/DiskArrayEngine/zj17T/src/runner.jl:41 [inlined]
 [10] run_block(op::DiskArrayEngine.GMDWop{…}, loopRanges::Tuple{…}, xin::Tuple{…}, xout::Tuple{…}, threaded::Bool)
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/zj17T/src/runner.jl:80
 [11] default_loopbody(inow::Tuple{…}, op::DiskArrayEngine.GMDWop{…}, inbuffers_pure::Tuple{…}, outbuffers::Tuple{…}, threaded::Bool, outars::Tuple{…}, cb::Tuple{…}, runfilter::Tuple{}, piddir::Nothing)
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/zj17T/src/runner.jl:176
 [12] run_loop(::DiskArrayEngine.LocalRunner, op::DiskArrayEngine.GMDWop{…}, inbuffers_pure::Tuple{…}, outbuffers::Tuple{…}, threaded::Bool, outars::Tuple{…}, loopranges::DiskArrayEngine.ProductArray{…}, cb::Tuple{…}, runfilter::Tuple{}; groupspecs::Nothing)
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/zj17T/src/runner.jl:186
 [13] run_loop(runner::DiskArrayEngine.LocalRunner, loopranges::DiskArrayEngine.ProductArray{Tuple{…}, 3, Tuple{…}}; groupspecs::Nothing)
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/zj17T/src/runner.jl:166
 [14] run_loop(runner::DiskArrayEngine.LocalRunner, loopranges::DiskArrayEngine.ProductArray{Tuple{UnitRange{…}, UnitRange{…}, UnitRange{…}}, 3, Tuple{Vector{…}, Vector{…}, Vector{…}}})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/zj17T/src/runner.jl:165
 [15] readblock!(::DiskArrayEngine.GMWOPResult{Float64, 3, DiskArrayEngine.GMDWop{…}, DiskArrays.Unchunked{…}, 1}, ::Array{Float64, 3}, ::UnitRange{Int64}, ::Vararg{UnitRange{…}})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/zj17T/src/diskarrayresults.jl:52
 [16] getindex_disk(::DiskArrayEngine.GMWOPResult{Float64, 3, DiskArrayEngine.GMDWop{…}, DiskArrays.Unchunked{…}, 1}, ::Int64, ::Int64, ::Vararg{Int64})
    @ DiskArrays ~/.julia/packages/DiskArrays/xakzf/src/indexing.jl:42
 [17] getindex
    @ ~/.julia/packages/DiskArrays/xakzf/src/indexing.jl:312 [inlined]
 [18] getindex(A::YAXArray{Float64, 3, DiskArrayEngine.GMWOPResult{…}, Tuple{…}, Dict{…}}, i1::Int64, i2::Int64, I::Int64)
    @ DimensionalData ~/.julia/packages/DimensionalData/hv9KC/src/array/indexing.jl:24
 [19] top-level scope
    @ REPL[370]:1
Some type information was truncated. Use `show(err)` to see complete types.

[felixcremer]

Could you please always copy paste the full error message, because sometimes we are reading this on our phones and then we might already help but can't without a full error message.

[Balinus]

Note that I tested on v0.6.1 and the code is working. The mapslices function return a loaded in memory struct though versus a lazy eval on v0.7.0

[briochemc]

Thanks @Balinus for pointing this out! It seems to me that v0.7.0 also "broke" other things that used to work in v0.6.1, at least:

• Bug: mean over dimension does not remove dimension #545
• Request: Return in-memory outputs by default for in-memory inputs #544
  but I am guessing there are more cases. Maybe all of these should be added as tests?

[felixcremer]

This was closed in #553. The problem was the assumption that the function in mapslices would always make a full reduction.