-
Notifications
You must be signed in to change notification settings - Fork 13
Suppor dimensional metadata #62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 5 commits
5ae5ab0
47daf4c
e1b08c5
19287e0
10ca2f0
bbe4fc7
ffbb582
3a0466f
1a418e3
115962f
7cb0aef
0d5bb5c
4a11c37
8225a7e
2ae2768
57e4f8c
e68ece2
c35f7a1
a3ecbcd
b519d26
ab256ec
838047a
c0ae557
2e343f4
b01d2c2
6a0af43
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
name = "DataAPI" | ||
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a" | ||
authors = ["quinnj <[email protected]>"] | ||
version = "1.15.0" | ||
version = "1.16.0" | ||
|
||
[compat] | ||
julia = "1" | ||
|
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -515,6 +515,97 @@ Throw an error if `x` does not support metadata deletion for column `col`. | |||||||
""" | ||||||||
function emptycolmetadata! end | ||||||||
|
||||||||
""" | ||||||||
dimmetadatasupport(T::Type, dim::Int) | ||||||||
|
||||||||
Return a `NamedTuple{(:read, :write), Tuple{Bool, Bool}}` indicating whether | ||||||||
values of type `T` support metadata correspond to dimension `dim`. | ||||||||
|
||||||||
The `read` field indicates whether reading metadata with the [`dimmetadata`](@ref) | ||||||||
and [`dimmetadatakeys`]](@ref) functions is supported. | ||||||||
|
||||||||
The `write` field indicates whether modifying metadata with the [`dimmetadata!`](@ref), | ||||||||
[`deletemetadata!`](@ref), and [`emptymetadata!`](@ref) functions is supported. | ||||||||
""" | ||||||||
dimmetadatasupport(::Type, i::Int) = (read=false, write=false) | ||||||||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
""" | ||||||||
dimmetadata(x, dim::Int, key::AbstractString, [default]; style::Bool=false) | ||||||||
|
||||||||
Return metadata value associated with `x` for dimension `dim` and key `key`. | ||||||||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
Throw an error if `x` does not support reading metadata for dimension `dim` or `x` | ||||||||
supports reading metadata, but does not have a mapping for dimension `dim` for `key`. | ||||||||
|
||||||||
If `style=true` return a tuple of metadata value and metadata style. Metadata | ||||||||
style is an additional information about the kind of metadata that is stored for | ||||||||
the `key`. | ||||||||
|
||||||||
$STYLE_INFO | ||||||||
|
||||||||
If `default` is passed then return it if `x` supports reading metadata and has | ||||||||
dimension `dim` but mapping for `key` is missing. | ||||||||
If `style=true` return `(default, :default)`. | ||||||||
""" | ||||||||
function dimmetadata end | ||||||||
|
||||||||
""" | ||||||||
dimmetadata(x, dim::Int; style::Bool=false) | ||||||||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
|
||||||||
Return a dictionary of metadata corresponding to dimension `dim` of object `x`. | ||||||||
Throw an error if `x` does not support reading metadata corresponding to dimension `dim`. | ||||||||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
|
||||||||
If `style=true` values are tuples of metadata value and metadata style. Metadata | ||||||||
style is an additional information about the kind of metadata that is stored for | ||||||||
the `key`. | ||||||||
|
||||||||
$STYLE_INFO | ||||||||
|
||||||||
The returned dictionary may be freshly allocated on each call to `metadata` and | ||||||||
is considered to be owned by `x` so it must not be modified. | ||||||||
""" | ||||||||
function dimmetadata(x::T, dim::Int; style::Bool=false) where {T} | ||||||||
if !dimmetadatasupport(T, dim).read | ||||||||
throw(ArgumentError("Objects of type $T do not support reading dimension metadata")) | ||||||||
end | ||||||||
return Dict(key => dimmetadata(x, dim, key, style=style) for key in dimmetadatakeys(x, dim)) | ||||||||
end | ||||||||
|
||||||||
""" | ||||||||
dimmetadatakeys(x, dim::Int) | ||||||||
|
||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's do the same as for
Suggested change
|
||||||||
Return an iterator of keys for corresponding to metadata for dimension `dim` of `x`. | ||||||||
If `x` does not support dimension metadata return `()`. | ||||||||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
""" | ||||||||
function dimmetadatakeys end | ||||||||
|
||||||||
""" | ||||||||
dimmetadata!(x, dim::Int, key::AbstractString, value; style::Symbol=:default) | ||||||||
|
||||||||
Set metadata for `x` for dimension `dim` for key `key` to have value `value` | ||||||||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
and style `style` (`:default` by default) and return `x`. | ||||||||
Throw an error if `x` does not support setting metadata for dimension `dim`. | ||||||||
|
||||||||
$STYLE_INFO | ||||||||
""" | ||||||||
function dimmetadata! end | ||||||||
|
||||||||
""" | ||||||||
deletedimmetadata!(x, dim::Int, key::AbstractString) | ||||||||
|
||||||||
Delete metadata for corresponding to dimension `dim` and key `key` of `x` and return `x` | ||||||||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
(if metadata for `key` is not present do not perform any action). | ||||||||
Throw an error if `x` does not support metadata deletion for dimension `dim`. | ||||||||
""" | ||||||||
function deletedimmetadata! end | ||||||||
|
||||||||
""" | ||||||||
emptydimmetadata!(x, dim::Int) | ||||||||
|
||||||||
Delete all metadata for `x` corresponding to dimension `dim` and return `x`. | ||||||||
Tokazama marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
Throw an error if `x` does not support metadata deletion for dimension `dim`. | ||||||||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
""" | ||||||||
function emptydimmetadata! end | ||||||||
|
||||||||
""" | ||||||||
rownumber(row) | ||||||||
|
||||||||
|
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -17,12 +17,22 @@ DataAPI.levels(x::TestArray) = reverse(DataAPI.levels(x.x)) | |||||||||||||
struct TestMeta | ||||||||||||||
table::Dict{String, Any} | ||||||||||||||
col::Dict{Symbol, Dict{String, Any}} | ||||||||||||||
dims::Tuple{Dict{String, Any}, Dict{String, Any}} | ||||||||||||||
|
||||||||||||||
TestMeta() = new(Dict{String, Any}(), Dict{Symbol, Dict{String, Any}}()) | ||||||||||||||
function TestMeta() | ||||||||||||||
new(Dict{String, Any}(), Dict{Symbol, Dict{String, Any}}(), (Dict{String, Any}(), Dict{String, Any}())) | ||||||||||||||
end | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
DataAPI.metadatasupport(::Type{TestMeta}) = (read=true, write=true) | ||||||||||||||
DataAPI.colmetadatasupport(::Type{TestMeta}) = (read=true, write=true) | ||||||||||||||
function DataAPI.dimmetadatasupport(::Type{TestMeta}, dim::Int) | ||||||||||||||
if dim == 1 || dim == 2 | ||||||||||||||
(read=true, write=true) | ||||||||||||||
else | ||||||||||||||
(read=false, write=false) | ||||||||||||||
end | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.metadata(x::TestMeta, key::AbstractString; style::Bool=false) | ||||||||||||||
return style ? x.table[key] : x.table[key][1] | ||||||||||||||
|
@@ -93,6 +103,38 @@ end | |||||||||||||
|
||||||||||||||
DataAPI.emptycolmetadata!(x::TestMeta) = empty!(x.col) | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata(x::TestMeta, dim::Int, key::AbstractString; style::Bool=false) | ||||||||||||||
return style ? x.dims[dim][key] : x.dims[dim][key][1] | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata(x::TestMeta, dim::Int, key::AbstractString, default; style::Bool=false) | ||||||||||||||
haskey(x.dims[dim], key) && return DataAPI.metadata(x, dim, key, style=style) | ||||||||||||||
return style ? (default, :default) : default | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadatakeys(x::TestMeta, dim::Int) | ||||||||||||||
if DataAPI.dimmetadatasupport(TestMeta, dim).read | ||||||||||||||
return keys(x.dims[dim]) | ||||||||||||||
else | ||||||||||||||
return () | ||||||||||||||
end | ||||||||||||||
Comment on lines
+117
to
+121
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The docstring says we should throw an error when
Suggested change
|
||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata!(x::TestMeta, dim::Int, key::AbstractString, value; style::Symbol=:default) | ||||||||||||||
x.dims[dim][key] = (value, style) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.deletedimmetadata!(x::TestMeta, dim::Int, key::AbstractString) | ||||||||||||||
delete!(x.dims[dim], key) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
function DataAPI.emptydimmetadata!(x::TestMeta, dim::Int) | ||||||||||||||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
empty!(x.dims[dim]) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
|
||||||||||||||
# An example implementation of a table (without the Tables.jl interface) | ||||||||||||||
# for testing DataAPI.rownumber | ||||||||||||||
struct TestTable{T} | ||||||||||||||
|
@@ -301,12 +343,24 @@ end | |||||||||||||
@test_throws MethodError DataAPI.colmetadatakeys(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.colmetadatakeys(1) | ||||||||||||||
|
||||||||||||||
@test_throws MethodError DataAPI.dimmetadata!(1, 1, "a", 10, style=:default) | ||||||||||||||
@test_throws MethodError DataAPI.deletedimmetadata!(1, 1, "a") | ||||||||||||||
@test_throws MethodError DataAPI.emptydimmetadata!(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.dimmetadata(1, 1, "a") | ||||||||||||||
@test_throws ArgumentError DataAPI.dimmetadata(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.dimmetadata(1, 1, "a", style=true) | ||||||||||||||
@test_throws ArgumentError DataAPI.dimmetadata(1, 1, style=true) | ||||||||||||||
@test_throws MethodError DataAPI.dimmetadatakeys(1, 1) | ||||||||||||||
|
||||||||||||||
@test DataAPI.metadatasupport(Int) == (read=false, write=false) | ||||||||||||||
@test DataAPI.colmetadatasupport(Int) == (read=false, write=false) | ||||||||||||||
@test DataAPI.dimmetadatasupport(Int, 1) == (read=false, write=false) | ||||||||||||||
|
||||||||||||||
tm = TestMeta() | ||||||||||||||
@test DataAPI.metadatasupport(TestMeta) == (read=true, write=true) | ||||||||||||||
@test DataAPI.colmetadatasupport(TestMeta) == (read=true, write=true) | ||||||||||||||
@test DataAPI.dimmetadatasupport(TestMeta, 1) == (read=true, write=true) | ||||||||||||||
@test DataAPI.dimmetadatasupport(TestMeta, 0) == (read=false, write=false) | ||||||||||||||
|
||||||||||||||
@test isempty(DataAPI.metadatakeys(tm)) | ||||||||||||||
@test DataAPI.metadata(tm) == Dict() | ||||||||||||||
|
@@ -353,6 +407,25 @@ end | |||||||||||||
@test DataAPI.colmetadata!(tm, :col, "a", "100", style=:note) == tm | ||||||||||||||
DataAPI.emptycolmetadata!(tm) | ||||||||||||||
@test isempty(DataAPI.colmetadatakeys(tm)) | ||||||||||||||
|
||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1) == Dict() | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, style=true) == Dict() | ||||||||||||||
@test DataAPI.dimmetadata!(tm, 1, "a", "100", style=:note) == tm | ||||||||||||||
@test collect(DataAPI.dimmetadatakeys(tm, 1)) == ["a"] | ||||||||||||||
@test_throws KeyError DataAPI.dimmetadata(tm, 1, "b") | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "b", 123) == 123 | ||||||||||||||
@test_throws KeyError DataAPI.dimmetadata(tm, 1, "b", style=true) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "b", 123, style=true) == (123, :default) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "a") == "100" | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1) == Dict("a" => "100") | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "a", style=true) == ("100", :note) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, style=true) == Dict("a" => ("100", :note)) | ||||||||||||||
DataAPI.deletedimmetadata!(tm, 1, "a") | ||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
@test DataAPI.dimmetadata!(tm, 1, "a", "100", style=:note) == tm | ||||||||||||||
DataAPI.emptydimmetadata!(tm, 1) | ||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
@testset "rownumber" begin | ||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I missed this substantial point:
colmetadatasupport
takes a single argument, i.e. it asks whether column metadata is supported at all, without considering a particular column. Shouldn't we do the same here? Once a type supports dimension metadata, it can returnnothing
for dimensions that do not have any metadata.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is could be OK as is proposed here.
colmetadatasupport
takes one argument, because there is exactly one dimension always.But what @nalimilan is also OK - it depends on what @Tokazama thinks is better (i.e. do we imagine that only selected dimensions would have metadata?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you mean by "there is exactly one dimension always"? :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK I get it now, you mean that columns are the second dimension so
colmetadatasupport(T)
is similar todimmetadatasupport(T, 2)
.I still think there's a problem though:
dimmetadata(x)
returns a tuple of metadata for each dimension. But how can you check whether you can calldimmetadata(x)
without getting an error? For that you would need to be able to calldimmetadatasupport(T)
.