diff --git a/Project.toml b/Project.toml index 658d732..e2b850d 100644 --- a/Project.toml +++ b/Project.toml @@ -1,7 +1,7 @@ name = "StatisticalMeasuresBase" uuid = "c062fc1d-0d66-479b-b6ac-8b44719de4cc" authors = ["Anthony D. Blaom "] -version = "0.1.1" +version = "0.1.2" [deps] CategoricalArrays = "324d7699-5711-5eae-9e2f-1d82baa6b597" diff --git a/README.md b/README.md index abc47e4..6e24cea 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@ A Julia package for building production-ready measures (metrics) for statistics Related: [StatisticalMeasures.jl](https://juliaai.github.io/StatisticalMeasures.jl/dev/) +[List](https://github.com/FluxML/FluxML-Community-Call-Minutes/discussions/38) of Julia packages providing metrics. ## The main idea @@ -47,18 +48,18 @@ julia> multitarget_L1(ŷ, y, weights) ``` ```julia -using Tables -t = y' |> Tables.table |> Tables.rowtable -t̂ = ŷ' |> Tables.table |> Tables.rowtable +using DataFrames +df = DataFrame(y', :auto) +df̂ = DataFrame(ŷ', :auto) -julia> multitarget_L1(t̂, t, weights) +julia> multitarget_L1(df̂, df, weights) 39 ``` -Generate measurements *for each observation* with the `measurement` method: +Generate measurements *for each observation* with the `measurements` method: ```julia -julia> measurements(multitarget_L1, t̂, t, weights) +julia> measurements(multitarget_L1, df̂, df, weights) 3-element Vector{Int64}: 3 9 diff --git a/docs/Project.toml b/docs/Project.toml index 76e32ef..f7b9b80 100644 --- a/docs/Project.toml +++ b/docs/Project.toml @@ -1,4 +1,5 @@ [deps] +DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4" ScientificTypesBase = "30f210dd-8aff-4c5f-94ba-8e64358c1161" Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" diff --git a/docs/src/index.md b/docs/src/index.md index d895972..4151960 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -47,16 +47,16 @@ multitarget_L1(ŷ, y, weights) ``` ```@example 01 -using Tables -t = y' |> Tables.table |> Tables.rowtable -t̂ = ŷ' |> Tables.table |> Tables.rowtable -multitarget_L1(t̂, t, weights) +using DataFrames +df = DataFrame(y', :auto) +df̂ = DataFrame(ŷ', :auto) +multitarget_L1(df̂, df, weights) ``` -Generate measurements *for each observation* with the `measurement` method: +Generate measurements *for each observation* with the `measurements` method: ```@example 01 -measurements(multitarget_L1, t̂, t, weights) +measurements(multitarget_L1, df̂, df, weights) ``` # Overview @@ -69,7 +69,7 @@ measures. For a package that does, based on this interface, see can also be applied to measures provided by other packages, such as [LossFunctions.jl](https://github.com/JuliaML/LossFunctions.jl). -Specically, this package provides: +Specifically, this package provides: - A measure wrapper [`multimeasure`](@ref) that leverages MLUtils.jl to broadcast a simple measure over multiple observations; the main use case is for extending a measure (e.g., diff --git a/docs/src/tutorial.md b/docs/src/tutorial.md index e0a4de9..e936948 100644 --- a/docs/src/tutorial.md +++ b/docs/src/tutorial.md @@ -115,10 +115,10 @@ instead of an aggregate. Mutli-targets (as tables): ```@example 42 -using Tables -t = y' |> Tables.table |> Tables.rowtable -t̂ = ŷ' |> Tables.table |> Tables.rowtable -@assert MultitargetHuberLoss()(t̂, t, weights) ≈ MultitargetHuberLoss()(ŷ, y, weights) +using DataFrames +df = DataFrame(y', :auto) +df̂ = DataFrame(ŷ', :auto) +@assert MultitargetHuberLoss()(df̂, df, weights) ≈ MultitargetHuberLoss()(df̂, df, weights) ``` Multi-dimensional arrays: diff --git a/src/fussy_measure.jl b/src/fussy_measure.jl index f33a291..82384a0 100644 --- a/src/fussy_measure.jl +++ b/src/fussy_measure.jl @@ -31,7 +31,7 @@ Return a new measure, `fussy`, with the same behavior as `measure`, except that `extra_check==nothing`. Note the first argument here is `measure`, not `atomic_measure`. Do not use `fussy_measure` unless both `y` and `ŷ` are expected to implement the -MLUtils.jl `getobs`/`numbos` interface (e.g., are `AbstractArray`s) +MLUtils.jl `getobs`/`numobs` interface (e.g., are `AbstractArray`s) See also [`$API.measurements`](@ref), [`$API.is_measure`](@ref) diff --git a/src/multimeasure.jl b/src/multimeasure.jl index 2dd5aef..9359f39 100644 --- a/src/multimeasure.jl +++ b/src/multimeasure.jl @@ -67,7 +67,7 @@ Advanced Examples below. using StatisticalMeasuresBase # define an atomic measure: -struct L2OnScalars +struct L2OnScalars end (::L2OnScalars)(ŷ, y) = (ŷ - y)^2 julia> $API.external_aggregation_mode(L2OnScalars()) diff --git a/src/traits.jl b/src/traits.jl index 83b5952..073bc16 100644 --- a/src/traits.jl +++ b/src/traits.jl @@ -131,8 +131,8 @@ can_report_unaggregated(measure) = false Return the kind of proxy `ŷ` for target predictions expected in calls of the form `measure(ŷ, y, args...; kwargs...)`. -Typical return values are `LearnAPI.LiteralTarget()`, when `ŷ` is expected to have the -same form as `ŷ`, or `LearnAPI.Distribution()`, when the observations in `ŷ` are expected +Typical return values are `LearnAPI.Point()`, when `ŷ` is expected to have the +same form as `y`, or `LearnAPI.Distribution()`, when the observations in `ŷ` are expected to represent probability density/mass functions. For other kinds of proxy, see the [LearnAPI.jl](https://juliaai.github.io/LearnAPI.jl/dev/) documentation. @@ -176,7 +176,7 @@ value has no meaning. # New implementations -Optional but strongly recommended for measure than consume multiple observations. The +Optional but strongly recommended for measures that consume multiple observations. The fallback returns `Union{}`. Examples of return values are `Union{Finite,Missing}`, for `CategoricalValue` observations diff --git a/test/integration.jl b/test/integration.jl index 3089bf3..5c53e38 100644 --- a/test/integration.jl +++ b/test/integration.jl @@ -84,7 +84,6 @@ const HuberLossType = API.Multimeasure{ @trait( HuberLossType, observation_scitype = Union{ST.Continuous,Missing}, -# kind_of_proxy = LearnAPI.LiteralTarget(), human_name = "Huber loss", ) @@ -93,7 +92,6 @@ const MultitargetHuberLossType = API.FussyMeasure{<:API.Multimeasure{<:HuberLoss MultitargetHuberLossType, observation_scitype = AbstractArray{<:Union{Continuous,Missing}}, can_consume_tables = true, -# kind_of_proxy = LearnAPI.LiteralTarget(), human_name = "multi-target Huber loss", ) diff --git a/test/multimeasure.jl b/test/multimeasure.jl index 3cbf8b4..89abd91 100644 --- a/test/multimeasure.jl +++ b/test/multimeasure.jl @@ -4,7 +4,6 @@ struct MAEOnScalars end @trait( MAEOnScalars, - kind_of_proxy = LiteralTarget(), orientation = Loss(), )