-
Notifications
You must be signed in to change notification settings - Fork 13
Suppor dimensional metadata #62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 8 commits
5ae5ab0
47daf4c
e1b08c5
19287e0
10ca2f0
bbe4fc7
ffbb582
3a0466f
1a418e3
115962f
7cb0aef
0d5bb5c
4a11c37
8225a7e
2ae2768
57e4f8c
e68ece2
c35f7a1
a3ecbcd
b519d26
ab256ec
838047a
c0ae557
2e343f4
b01d2c2
6a0af43
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
name = "DataAPI" | ||
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a" | ||
authors = ["quinnj <[email protected]>"] | ||
version = "1.15.0" | ||
version = "1.16.0" | ||
|
||
[compat] | ||
julia = "1" | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -472,7 +472,7 @@ end | |
colmetadatakeys(x, [col]) | ||
|
||
If `col` is passed return an iterator of metadata keys for which | ||
`metadata(x, col, key)` returns a metadata value. Throw an error if `x` does not | ||
`colmetadata(x, col, key)` returns a metadata value. Throw an error if `x` does not | ||
support reading column metadata or if `col` is not a column of `x`. | ||
|
||
`col` must have a type that is supported by table `x` for column indexing. | ||
|
@@ -515,6 +515,113 @@ Throw an error if `x` does not support metadata deletion for column `col`. | |
""" | ||
function emptycolmetadata! end | ||
|
||
""" | ||
dimmetadatasupport(T::Type, dim::Int) | ||
|
||
Return a `NamedTuple{(:read, :write), Tuple{Bool, Bool}}` indicating whether | ||
values of type `T` support metadata correspond to dimension `dim`. | ||
|
||
The `read` field indicates whether reading metadata with the [`dimmetadata`](@ref) | ||
and [`dimmetadatakeys`]](@ref) functions is supported. | ||
|
||
The `write` field indicates whether modifying metadata with the [`dimmetadata!`](@ref), | ||
[`deletemetadata!`](@ref), and [`emptymetadata!`](@ref) functions is supported. | ||
""" | ||
dimmetadatasupport(::Type, i::Int) = (read=false, write=false) | ||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
""" | ||
dimmetadata(x, dim::Int, key::AbstractString, [default]; style::Bool=false) | ||
|
||
Return metadata value associated with `x` for dimension `dim` and key `key`. | ||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Throw an error if `x` does not support reading metadata for dimension `dim` or `x` | ||
supports reading metadata, but does not have a mapping for dimension `dim` for `key`. | ||
|
||
If `style=true` return a tuple of metadata value and metadata style. Metadata | ||
style is an additional information about the kind of metadata that is stored for | ||
the `key`. | ||
|
||
$STYLE_INFO | ||
|
||
If `default` is passed then return it if `x` supports reading metadata and has | ||
dimension `dim` but mapping for `key` is missing. | ||
If `style=true` return `(default, :default)`. | ||
""" | ||
function dimmetadata end | ||
|
||
""" | ||
dimmetadata(x, dim::Int; style::Bool=false) | ||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
Return a dictionary of metadata corresponding to dimension `dim` of object `x`. If `dim` | ||
is not passed return a collection mapping each dimension to its metadata so that | ||
`collection[dim][key] == dimmetadata(x, dim, key)`. | ||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Throw an error if `x` does not support reading metadata corresponding to dimension `dim`. | ||
nalimilan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
If `style=true` values are tuples of metadata value and metadata style. Metadata | ||
style is an additional information about the kind of metadata that is stored for | ||
the `key`. | ||
|
||
$STYLE_INFO | ||
|
||
The returned dictionary may be freshly allocated on each call to `dimmetadata` and | ||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
is considered to be owned by `x` so it must not be modified. | ||
""" | ||
function dimmetadata(x::T, dim::Int; style::Bool=false) where {T} | ||
if !dimmetadatasupport(T, dim).read | ||
throw(ArgumentError("Objects of type $T do not support reading dimension metadata")) | ||
end | ||
return Dict(key => dimmetadata(x, dim, key, style=style) for key in dimmetadatakeys(x, dim)) | ||
end | ||
function dimmetadata(x::T; style::Bool=false) where {T} | ||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Tuple(dimmetadata(x, dim; style=style) for dim in 1:ndims(x)) | ||
end | ||
|
||
""" | ||
dimmetadatakeys(x, [dim::Int]) | ||
|
||
If `dim` is passed return an iterator of metadata keys for which | ||
`dimmetadata(x, dim, key)` returns a metadata value. Throw an error if `x` does not | ||
support reading dimension metadata or if `dim` is not a dimension of `x`. | ||
|
||
If `dim` is not passed return an iterator of `dim => dimmetadatakeys(x, dim)` | ||
pairs for all dimensions that have metadata. | ||
If `x` does not support dimension metadata return `()`. | ||
""" | ||
function dimmetadatakeys(x, dim::Int) | ||
dimmetadatasupport(typeof(x), dim).read || return () | ||
throw(MethodError(dimmetadatakeys, (x, dim))) | ||
end | ||
function dimmetadatakeys(x) | ||
Dict(dim => dimmetadatakeys(x, dim) for dim in 1:ndims(x)) | ||
end | ||
|
||
|
||
""" | ||
dimmetadata!(x, dim::Int, key::AbstractString, value; style::Symbol=:default) | ||
|
||
Set metadata for `x` for dimension `dim` for key `key` to have value `value` | ||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
and style `style` (`:default` by default) and return `x`. | ||
Throw an error if `x` does not support setting metadata for dimension `dim`. | ||
|
||
$STYLE_INFO | ||
""" | ||
function dimmetadata! end | ||
|
||
""" | ||
deletedimmetadata!(x, dim::Int, key::AbstractString) | ||
|
||
Delete metadata for corresponding to dimension `dim` and key `key` of `x` and return `x` | ||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
(if metadata for `key` is not present do not perform any action). | ||
Throw an error if `x` does not support metadata deletion for dimension `dim`. | ||
""" | ||
function deletedimmetadata! end | ||
|
||
""" | ||
emptydimmetadata!(x, dim::Int) | ||
|
||
Delete all metadata for `x` corresponding to dimension `dim` and return `x`. | ||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Throw an error if `x` does not support metadata deletion for dimension `dim`. | ||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||
""" | ||
function emptydimmetadata! end | ||
|
||
""" | ||
rownumber(row) | ||
|
||
|
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -17,12 +17,23 @@ DataAPI.levels(x::TestArray) = reverse(DataAPI.levels(x.x)) | |||||||||||||
struct TestMeta | ||||||||||||||
table::Dict{String, Any} | ||||||||||||||
col::Dict{Symbol, Dict{String, Any}} | ||||||||||||||
dims::Tuple{Dict{String, Any}, Dict{String, Any}} | ||||||||||||||
|
||||||||||||||
TestMeta() = new(Dict{String, Any}(), Dict{Symbol, Dict{String, Any}}()) | ||||||||||||||
function TestMeta() | ||||||||||||||
new(Dict{String, Any}(), Dict{Symbol, Dict{String, Any}}(), (Dict{String, Any}(), Dict{String, Any}())) | ||||||||||||||
end | ||||||||||||||
end | ||||||||||||||
Base.ndims(::TestMeta) = 2 | ||||||||||||||
|
||||||||||||||
DataAPI.metadatasupport(::Type{TestMeta}) = (read=true, write=true) | ||||||||||||||
DataAPI.colmetadatasupport(::Type{TestMeta}) = (read=true, write=true) | ||||||||||||||
function DataAPI.dimmetadatasupport(::Type{TestMeta}, dim::Int) | ||||||||||||||
if dim == 1 || dim == 2 | ||||||||||||||
(read=true, write=true) | ||||||||||||||
else | ||||||||||||||
(read=false, write=false) | ||||||||||||||
end | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.metadata(x::TestMeta, key::AbstractString; style::Bool=false) | ||||||||||||||
return style ? x.table[key] : x.table[key][1] | ||||||||||||||
|
@@ -93,6 +104,38 @@ end | |||||||||||||
|
||||||||||||||
DataAPI.emptycolmetadata!(x::TestMeta) = empty!(x.col) | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata(x::TestMeta, dim::Int, key::AbstractString; style::Bool=false) | ||||||||||||||
return style ? x.dims[dim][key] : x.dims[dim][key][1] | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata(x::TestMeta, dim::Int, key::AbstractString, default; style::Bool=false) | ||||||||||||||
haskey(x.dims[dim], key) && return DataAPI.metadata(x, dim, key, style=style) | ||||||||||||||
return style ? (default, :default) : default | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadatakeys(x::TestMeta, dim::Int) | ||||||||||||||
if DataAPI.dimmetadatasupport(TestMeta, dim).read | ||||||||||||||
return keys(x.dims[dim]) | ||||||||||||||
else | ||||||||||||||
return () | ||||||||||||||
end | ||||||||||||||
Comment on lines
+117
to
+121
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The docstring says we should throw an error when
Suggested change
|
||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata!(x::TestMeta, dim::Int, key::AbstractString, value; style::Symbol=:default) | ||||||||||||||
x.dims[dim][key] = (value, style) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.deletedimmetadata!(x::TestMeta, dim::Int, key::AbstractString) | ||||||||||||||
delete!(x.dims[dim], key) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
function DataAPI.emptydimmetadata!(x::TestMeta, dim::Int) | ||||||||||||||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
empty!(x.dims[dim]) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
|
||||||||||||||
# An example implementation of a table (without the Tables.jl interface) | ||||||||||||||
# for testing DataAPI.rownumber | ||||||||||||||
struct TestTable{T} | ||||||||||||||
|
@@ -301,12 +344,23 @@ end | |||||||||||||
@test_throws MethodError DataAPI.colmetadatakeys(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.colmetadatakeys(1) | ||||||||||||||
|
||||||||||||||
@test_throws MethodError DataAPI.dimmetadata!(1, 1, "a", 10, style=:default) | ||||||||||||||
@test_throws MethodError DataAPI.deletedimmetadata!(1, 1, "a") | ||||||||||||||
@test_throws MethodError DataAPI.emptydimmetadata!(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.dimmetadata(1, 1, "a") | ||||||||||||||
@test_throws ArgumentError DataAPI.dimmetadata(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.dimmetadata(1, 1, "a", style=true) | ||||||||||||||
@test_throws ArgumentError DataAPI.dimmetadata(1, 1, style=true) | ||||||||||||||
|
||||||||||||||
@test DataAPI.metadatasupport(Int) == (read=false, write=false) | ||||||||||||||
@test DataAPI.colmetadatasupport(Int) == (read=false, write=false) | ||||||||||||||
@test DataAPI.dimmetadatasupport(Int, 1) == (read=false, write=false) | ||||||||||||||
|
||||||||||||||
tm = TestMeta() | ||||||||||||||
@test DataAPI.metadatasupport(TestMeta) == (read=true, write=true) | ||||||||||||||
@test DataAPI.colmetadatasupport(TestMeta) == (read=true, write=true) | ||||||||||||||
@test DataAPI.dimmetadatasupport(TestMeta, 1) == (read=true, write=true) | ||||||||||||||
@test DataAPI.dimmetadatasupport(TestMeta, 0) == (read=false, write=false) | ||||||||||||||
|
||||||||||||||
@test isempty(DataAPI.metadatakeys(tm)) | ||||||||||||||
@test DataAPI.metadata(tm) == Dict() | ||||||||||||||
|
@@ -353,6 +407,28 @@ end | |||||||||||||
@test DataAPI.colmetadata!(tm, :col, "a", "100", style=:note) == tm | ||||||||||||||
DataAPI.emptycolmetadata!(tm) | ||||||||||||||
@test isempty(DataAPI.colmetadatakeys(tm)) | ||||||||||||||
|
||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm)[1]) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1) == Dict() | ||||||||||||||
@test DataAPI.dimmetadata(tm) == (Dict(), Dict()) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, style=true) == Dict() | ||||||||||||||
@test DataAPI.dimmetadata!(tm, 1, "a", "100", style=:note) == tm | ||||||||||||||
@test collect(DataAPI.dimmetadatakeys(tm, 1)) == ["a"] | ||||||||||||||
@test_throws KeyError DataAPI.dimmetadata(tm, 1, "b") | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "b", 123) == 123 | ||||||||||||||
@test_throws KeyError DataAPI.dimmetadata(tm, 1, "b", style=true) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "b", 123, style=true) == (123, :default) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "a") == "100" | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1) == Dict("a" => "100") | ||||||||||||||
@test DataAPI.dimmetadata(tm)[1] == Dict("a" => "100") | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "a", style=true) == ("100", :note) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, style=true) == Dict("a" => ("100", :note)) | ||||||||||||||
DataAPI.deletedimmetadata!(tm, 1, "a") | ||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
@test DataAPI.dimmetadata!(tm, 1, "a", "100", style=:note) == tm | ||||||||||||||
DataAPI.emptydimmetadata!(tm, 1) | ||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
@testset "rownumber" begin | ||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I missed this substantial point:
colmetadatasupport
takes a single argument, i.e. it asks whether column metadata is supported at all, without considering a particular column. Shouldn't we do the same here? Once a type supports dimension metadata, it can returnnothing
for dimensions that do not have any metadata.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is could be OK as is proposed here.
colmetadatasupport
takes one argument, because there is exactly one dimension always.But what @nalimilan is also OK - it depends on what @Tokazama thinks is better (i.e. do we imagine that only selected dimensions would have metadata?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you mean by "there is exactly one dimension always"? :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK I get it now, you mean that columns are the second dimension so
colmetadatasupport(T)
is similar todimmetadatasupport(T, 2)
.I still think there's a problem though:
dimmetadata(x)
returns a tuple of metadata for each dimension. But how can you check whether you can calldimmetadata(x)
without getting an error? For that you would need to be able to calldimmetadatasupport(T)
.