Skip to content

Commit 41d632b

Browse files
ablaomablaom
committed
In the beginning there was Zero. Zero counted itself, "One", ...
more stuff more stuff more stuff more more more more before dumping tools more more more more more more add registry check spelling and fix doc generation issues add callability and callable_measure tweaks dump registry; add @fix_show add @Combination tool tests and adding to manual more polish polish mulitmeasure option weights -> atomic_weights fix some method ambiguity revolving around `call` fallback & polish polish modify CompositeWeights to eliminate missing elements add support for aggregating dict-valued measurements more doc tweaks add extra_check arg to fussy_measure tweak oops introduce HonestMean() and change meaning of Mean() polish polish add [compat] to Project.toml fix callable measures to forward measurement(...) method typo improve display of wrapped objects and add type param: Wrapper{M} doc fixes avoid global constant with a let block in @Combination allow wrapping of arbtirary callables ("hide call") hide call further in docs more doc fixes get rid of call as public facing method doc polish oops add precompilation workload commit forgotten file re-organize some files fix docstring oops
0 parents  commit 41d632b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+6041
-0
lines changed

‎.github/workflows/CompatHelper.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
name: CompatHelper
2+
on:
3+
schedule:
4+
- cron: '00 00 * * *'
5+
workflow_dispatch:
6+
jobs:
7+
CompatHelper:
8+
runs-on: ubuntu-latest
9+
steps:
10+
- name: Pkg.add("CompatHelper")
11+
run: julia -e 'using Pkg; Pkg.add("CompatHelper")'
12+
- name: CompatHelper.main()
13+
env:
14+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
15+
COMPATHELPER_PRIV: ${{ secrets.COMPATHELPER_PRIV}}
16+
run: julia -e 'using CompatHelper; CompatHelper.main()'

‎.github/workflows/TagBot.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
name: TagBot
2+
on:
3+
issue_comment:
4+
types:
5+
- created
6+
workflow_dispatch:
7+
jobs:
8+
TagBot:
9+
if: github.event_name == 'workflow_dispatch' || github.actor == 'JuliaTagBot'
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: JuliaRegistries/TagBot@v1
13+
with:
14+
token: ${{ secrets.GITHUB_TOKEN }}
15+
ssh: ${{ secrets.DOCUMENTER_KEY }}

‎.github/workflows/ci.yml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
name: CI
2+
on:
3+
pull_request:
4+
branches:
5+
- master
6+
- dev
7+
push:
8+
branches:
9+
- master
10+
- dev
11+
tags: '*'
12+
jobs:
13+
test:
14+
name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}
15+
runs-on: ${{ matrix.os }}
16+
strategy:
17+
fail-fast: false
18+
matrix:
19+
version:
20+
- '1.6'
21+
- '1' # automatically expands to the latest stable 1.x release of Julia.
22+
os:
23+
- ubuntu-latest
24+
arch:
25+
- x64
26+
steps:
27+
- uses: actions/checkout@v2
28+
- uses: julia-actions/setup-julia@v1
29+
with:
30+
version: ${{ matrix.version }}
31+
arch: ${{ matrix.arch }}
32+
- uses: actions/cache@v1
33+
env:
34+
cache-name: cache-artifacts
35+
with:
36+
path: ~/.julia/artifacts
37+
key: ${{ runner.os }}-test-${{ env.cache-name }}-${{ hashFiles('**/Project.toml') }}
38+
restore-keys: |
39+
${{ runner.os }}-test-${{ env.cache-name }}-
40+
${{ runner.os }}-test-
41+
${{ runner.os }}-
42+
- uses: julia-actions/julia-buildpkg@v1
43+
- uses: julia-actions/julia-runtest@v1
44+
env:
45+
JULIA_NUM_THREADS: 2
46+
- uses: julia-actions/julia-processcoverage@v1
47+
- uses: codecov/codecov-action@v1
48+
with:
49+
file: lcov.info
50+
docs:
51+
name: Documentation
52+
runs-on: ubuntu-latest
53+
steps:
54+
- uses: actions/checkout@v2
55+
- uses: julia-actions/setup-julia@v1
56+
with:
57+
version: '1'
58+
- uses: julia-actions/julia-buildpkg@v1
59+
- uses: julia-actions/julia-docdeploy@v1
60+
env:
61+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
62+
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }}
63+
- run: |
64+
julia --project=docs -e '
65+
using Documenter: DocMeta, doctest
66+
using StatisticalMeasuresAPI
67+
DocMeta.setdocmeta!(StatisticalMeasuresAPI, :DocTestSetup, :(using StatisticalMeasuresAPI); recursive=true)
68+
doctest(StatisticalMeasuresAPI)'

‎.gitignore‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
Manifest.toml
2+
.ipynb_checkpoints
3+
*~
4+
#*
5+
.DS_Store
6+
sandbox/
7+
/docs/build/
8+
/docs/site/
9+
/docs/Manifest.toml
10+
/test/Manifest.toml
11+
.vscode

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
MIT License Copyright (c) 2021 - JuliaAI
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎Project.toml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name = "StatisticalMeasuresBase"
2+
uuid = "c062fc1d-0d66-479b-b6ac-8b44719de4cc"
3+
authors = ["Anthony D. Blaom <[email protected]>"]
4+
version = "0.1.0"
5+
6+
[deps]
7+
CategoricalArrays = "324d7699-5711-5eae-9e2f-1d82baa6b597"
8+
InteractiveUtils = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
9+
MLUtils = "f1d291b0-491e-4a28-83b9-f70985020b54"
10+
MacroTools = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
11+
OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
12+
PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
13+
ScientificTypesBase = "30f210dd-8aff-4c5f-94ba-8e64358c1161"
14+
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
15+
16+
[compat]
17+
CategoricalArrays = "0.10"
18+
MLUtils = "0.4"
19+
MacroTools = "0.5"
20+
OrderedCollections = "1.6"
21+
PrecompileTools = "1.1"
22+
ScientificTypesBase = "3"
23+
julia = "1.6"

‎README.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# StatisticalMeasuresBase.jl
2+
3+
A Julia package for building production-ready measures (metrics) for statistics and machine learning.
4+
5+
[![Build Status](https://github.com/JuliaAI/StatisticalMeasuresBase.jl/workflows/CI/badge.svg)](https://github.com/JuliaAI/StatisticalMeasuresBase.jl/actions)
6+
[![Coverage](https://codecov.io/gh/JuliaAI/StatisticalMeasuresBase.jl/branch/master/graph/badge.svg)](https://codecov.io/github/JuliaAI/StatisticalMeasuresBase.jl?branch=master)
7+
[![Docs](https://img.shields.io/badge/docs-dev-blue.svg)](https://juliaai.github.io/StatisticalMeasuresBase.jl/dev/)
8+
9+
10+
## The main idea
11+
12+
Here's an example of a simple statistical measure that can be applied to a pair of scalars:
13+
14+
```julia
15+
l1(ŷ, y) = abs(ŷ - y)
16+
y = 5 # ground truth
17+
ŷ = 2 # prediction
18+
19+
julia> l1(ŷ, y)
20+
3
21+
```
22+
23+
Wrappers provided in this package extend the functionality of such measures. For example:
24+
25+
```julia
26+
using StatisticalMeasuresBase
27+
L1 = multimeasure(supports_missings_measure(l1))
28+
y = [1, 2, missing]
29+
ŷ = [2, 3, 4]
30+
weights = [8, 7, 6]
31+
32+
julia> L1(ŷ, y, weights)
33+
5.0
34+
```
35+
36+
```julia
37+
multitarget_L1 = multimeasure(L1, transform=vec∘collect)
38+
# 3 observations (last index is observation index):
39+
y = [1 2 3; 2 4 6]
40+
ŷ = [2 3 4; 4 6 8]
41+
42+
julia> multitarget_L1(ŷ, y, weights)
43+
10.5
44+
```
45+
46+
```julia
47+
using Tables
48+
t = y' |> Tables.table |> Tables.rowtable
49+
t̂ = ŷ' |> Tables.table |> Tables.rowtable
50+
51+
julia> multitarget_L1(t̂, t, weights)
52+
10.5
53+
```
54+
55+
Access per-observation measurements with the measurement method:
56+
57+
```julia
58+
julia> measurements(multitarget_L1, t̂, t, weights)
59+
60+
3-element Vector{Float64}:
61+
12.0
62+
10.5
63+
9.0
64+
```
65+

‎docs/Project.toml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
[deps]
2+
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
3+
ScientificTypesBase = "30f210dd-8aff-4c5f-94ba-8e64358c1161"
4+
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
5+
Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
6+
7+
[compat]
8+
Documenter = "^0.27"
9+
julia = "1"

0 commit comments

Comments
 (0)