Skip to content

Conversation

mkitti
Copy link
Member

@mkitti mkitti commented Mar 12, 2025

This replaces #150 by implementing the dimension_separator as a type
parameter.

I also intend to add Zarr specification version as a type parameter in forthcoming commits.

@mkitti
Copy link
Member Author

mkitti commented Mar 12, 2025

cc: @bjarthur @StephanPreibisch

@coveralls
Copy link

coveralls commented Mar 13, 2025

Pull Request Test Coverage Report for Build 15430506466

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 71 of 134 (52.99%) changed or added relevant lines in 5 files are covered.
  • 5 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-4.6%) to 81.444%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/metadata.jl 11 13 84.62%
src/Storage/http.jl 8 11 72.73%
src/ZArray.jl 6 9 66.67%
src/ZGroup.jl 15 18 83.33%
src/Storage/formattedstore.jl 31 83 37.35%
Files with Coverage Reduction New Missed Lines %
src/ZGroup.jl 1 81.32%
src/Compressors/Compressors.jl 2 88.0%
src/Storage/Storage.jl 2 76.92%
Totals Coverage Status
Change from base Build 13680422725: -4.6%
Covered Lines: 970
Relevant Lines: 1191

💛 - Coveralls

@mkitti
Copy link
Member Author

mkitti commented Mar 18, 2025

@bjarthur I believe the dimension separator of / is now working with Zarr v2.

julia> using Zarr

julia> readdir("ts_test/zarr_v2_raw")
9-element Vector{String}:
 ".zarray"
 "0"
 "1"
 "2"
 "3"
 "4"
 "5"
 "6"
 "7"

julia> read("ts_test/zarr_v2_raw/.zarray", String) |> print
{"chunks":[8,8,8],"compressor":null,"dimension_separator":"/","dtype":"<f8","fill_value":null,"filters":null,"order":"C","shape":[64,64,64],"zarr_format":2}
julia> g = zopen("ts_test")
ZarrGroup at DirectoryStore{2, '/'}("ts_test") and path 
Variables: zarr_v2_raw 

julia> typeof(g)
ZGroup{DirectoryStore{2, '/'}}

julia> arr = g["zarr_v2_raw"]
ZArray{Float64} of size 64 x 64 x 64

julia> typeof(arr)
ZArray{Float64, 3, Zarr.NoCompressor, DirectoryStore{2, '/'}}

julia> arr[1:10,1,1]
10-element Vector{Float64}:
 0.5031057549501186
 0.30753242290354865
 0.7604873102093034
 0.3274341519944033
 0.016695504839365882
 0.4110998920786101
 0.2731260969810403
 0.5038009325254829
 0.1788864699667917
 0.8449113642679142

@mkitti mkitti mentioned this pull request Apr 1, 2025
@mkitti
Copy link
Member Author

mkitti commented Apr 1, 2025

I have prototyped Zarr v3 on top of this and #180
https://github.com/mkitti/Zarr.jl/tree/mkitti-v3-prototype

@felixcremer
Copy link
Contributor

The failures on nightly seem to come from CondaPkg not being able to install pixi.
Unfortunately CondaPkg doesn't test on nightly.

@mkitti mkitti marked this pull request as ready for review May 8, 2025 13:23
@mkitti
Copy link
Member Author

mkitti commented May 8, 2025

Currently, I have the following storage wrapper which adds separator and zarr format version information to a storage container. Below the parameters are as follows:

  • V is a the Zarr format version
  • SEP is either the dimension separator (a Char) or a some other thing modification of the chunk key encoding
  • STORE is the storage that is being "versioned"
struct VersionedStore{V,SEP,STORE <: AbstractStore} <: AbstractStore
    parent::STORE
end

This may be be confused for some kind of storage that keeps track of versions. I am considering renaming this to be ForrmattedStore. Here CKE replaces SEP since CKE can be any kind of chunk key encoding scheme or extension.

struct FormattedStore{V,CKE,STORE <: AbstractStore} <: AbstractStore
    parent::STORE
end

@mkitti mkitti mentioned this pull request Jun 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants