add trait tests

Author: ablaom

docs/src/common_implementation_patterns.md

@@ -16,30 +16,30 @@
 This guide is intended to be consulted after reading [Anatomy of an Implementation](@ref),
 which introduces the main interface objects and terminology.
 
-Although an implementation is defined purely by the methods and traits it implements, most
+Although an implementation is defined purely by the methods and traits it implements, many
 implementations fall into one (or more) of the following informally understood patterns or
 "tasks":
 
 - [Regression](@ref): Supervised learners for continuous targets
 
-- [Classification](@ref): Supervised learners for categorical targets 
+- Classification: Supervised learners for categorical targets 
 
-- [Clusterering](@ref): Algorithms that group data into clusters for classification and
+- Clusterering: Algorithms that group data into clusters for classification and
   possibly dimension reduction. May be true learners (generalize to new data) or static.
 
-- [Gradient Descent](@ref): Including neural networks.
+- Gradient Descent: Including neural networks.
 
 - [Iterative Algorithms](@ref)
 
-- [Incremental Algorithms](@ref)
+- Incremental Algorithms
 
 - [Feature Engineering](@ref): Algorithms for selecting or combining features
 
-- [Dimension Reduction](@ref): Transformers that learn to reduce feature space dimension
+- Dimension Reduction: Transformers that learn to reduce feature space dimension
 
-- [Missing Value Imputation](@ref)
+- Missing Value Imputation
 
-- [Transformers](@ref): Other transformers, such as standardizers, and categorical
+- Transformers: Other transformers, such as standardizers, and categorical
   encoders.
 
 - [Static Algorithms](@ref): Algorithms that do not learn, in the sense they must be
@@ -48,26 +48,26 @@ implementations fall into one (or more) of the following informally understood p
 
 - [Ensemble Algorithms](@ref): Algorithms that blend predictions of multiple algorithms
 
-- [Time Series Forecasting](@ref)
+- Time Series Forecasting
 
-- [Time Series Classification](@ref)
+- Time Series Classification
 
-- [Survival Analysis](@ref)
+- Survival Analysis
 
-- [Density Estimation](@ref): Algorithms that learn a probability distribution
+- Density Estimation: Algorithms that learn a probability distribution
 
-- [Bayesian Algorithms](@ref)
+- Bayesian Algorithms
 
-- [Outlier Detection](@ref): Supervised, unsupervised, or semi-supervised learners for
+- Outlier Detection: Supervised, unsupervised, or semi-supervised learners for
   anomaly detection.
 
-- [Text Analysis](@ref)
+- Text Analysis
 
-- [Audio Analysis](@ref)
+- Audio Analysis
 
-- [Natural Language Processing](@ref)
+- Natural Language Processing
 
-- [Image Processing](@ref)
+- Image Processing
 
 - [Meta-algorithms](@ref)
 

docs/src/patterns/meta_algorithms.md

@@ -1 +1,6 @@
 # Meta-algorithms
+
+Many meta-algorithms are wrappers. An example is [this bagged ensemble
+algorithm](https://github.com/JuliaAI/LearnAPI.jl/blob/dev/test/integration/iterative_algorithms.jl)
+from tests.
+

docs/src/reference.md

@@ -141,7 +141,9 @@ for each.
     [`LearnAPI.algorithm`](@ref algorithm_minimize), [`LearnAPI.constructor`](@ref) and
     [`LearnAPI.functions`](@ref).
 
-Most algorithms will also implement [`predict`](@ref) and/or [`transform`](@ref).
+Most algorithms will also implement [`predict`](@ref) and/or [`transform`](@ref). For a
+bare minimum implementation, see the implementation of `SmallAlgorithm`
+[here](https://github.com/JuliaAI/LearnAPI.jl/blob/dev/test/traits.jl).
 
 ### List of methods
 

docs/src/traits.md

@@ -26,8 +26,8 @@ In the examples column of the table below, `Continuous` is a name owned the pack
 | [`LearnAPI.load_path`](@ref)`(algorithm)`                    | string locating name returned by `LearnAPI.constructor(algorithm)`, beginning with a package name                        | "unknown"`                                            | `FastTrees.LearnAPI.DecisionTreeClassifier`                |
 | [`LearnAPI.is_composite`](@ref)`(algorithm)`                 | `true` if one or more properties of `algorithm` may be an algorithm                                                      | `false`                                               | `true`                                                     |
 | [`LearnAPI.human_name`](@ref)`(algorithm)`                   | human name for the algorithm; should be a noun                                                                           | type name with spaces                                 | "elastic net regressor"                                    |
-| [`LearnAPI.data_interface`](@ref)`(algorithm)`               | Interface implemented by objects returned by [`obs`](@ref)                                                               | `Base.HasLength()` (supports `MLUtils.getobs/numobs`) | `Base.SizeUnknown()` (supports `iterate`)                  |
 | [`LearnAPI.iteration_parameter`](@ref)`(algorithm)`          | symbolic name of an iteration parameter                                                                                  | `nothing`                                             | :epochs                                                    |
+| [`LearnAPI.data_interface`](@ref)`(algorithm)`               | Interface implemented by objects returned by [`obs`](@ref)                                                               | `Base.HasLength()` (supports `MLUtils.getobs/numobs`) | `Base.SizeUnknown()` (supports `iterate`)                  |
 | [`LearnAPI.fit_observation_scitype`](@ref)`(algorithm)`      | upper bound on `scitype(observation)` for `observation` in `data` ensuring `fit(algorithm, data)` works                  | `Union{}`                                             | `Tuple{AbstractVector{Continuous}, Continuous}`            |
 | [`LearnAPI.target_observation_scitype`](@ref)`(algorithm)`   | upper bound on the scitype of each observation of the targget                                                            | `Any`                                                 | `Continuous`                                               |
 | [`LearnAPI.predict_or_transform_mutates`](@ref)`(algorithm)` | `true` if `predict` or `transform` mutates first argument                                                                | `false`                                               | `true`                                                     |
@@ -36,12 +36,12 @@ In the examples column of the table below, `Continuous` is a name owned the pack
 
 The following are provided for convenience but should not be overloaded by new algorithms:
 
-| trait                              | return value                                                         | example |
-|:-----------------------------------|:---------------------------------------------------------------------|:--------|
-| `LearnAPI.name(algorithm)`         | algorithm type name as string                                        | "PCA"   |
-| `LearnAPI.is_algorithm(algorithm)` | `true` if `algorithm` is LearnAPI.jl-compliant                          | `true`  |
-| `LearnAPI.target(algorithm)`       | `true` if [`LearnAPI.target(algorithm, data)`](@ref) is implemented  | `false` |
-| `LearnAPI.weights(algorithm)`      | `true` if [`LearnAPI.weights(algorithm, data)`](@ref) is implemented | `false` |
+| trait                              | return value                                                             | example |
+|:-----------------------------------|:-------------------------------------------------------------------------|:--------|
+| `LearnAPI.name(algorithm)`         | algorithm type name as string                                            | "PCA"   |
+| `LearnAPI.is_algorithm(algorithm)` | `true` if `algorithm` is LearnAPI.jl-compliant                           | `true`  |
+| `LearnAPI.target(algorithm)`       | `true` if `fit` sees a target variable; see [`LearnAPI.target`](@ref)    | `false` |
+| `LearnAPI.weights(algorithm)`      | `true` if `fit` supports per-observation; see [`LearnAPI.weights`](@ref) | `false` |
 
 ## Implementation guide
 

src/traits.jl

@@ -23,29 +23,6 @@ const DOC_EXPLAIN_EACHOBS =
 
     """
 
-const TRAITS = [
-    :constructor,
-    :functions,
-    :kinds_of_proxy,
-    :tags,
-    :is_pure_julia,
-    :pkg_name,
-    :pkg_license,
-    :doc_url,
-    :load_path,
-    :is_composite,
-    :human_name,
-    :iteration_parameter,
-    :data_interface,
-    :predict_or_transform_mutates,
-    :fit_observation_scitype,
-    :target_observation_scitype,
-    :name,
-    :is_algorithm,
-    :target,
-]
-
-
 # # OVERLOADABLE TRAITS
 
 """
@@ -426,7 +403,7 @@ variable. Specifically:
   variables) then "target" means anything returned by `LearnAPI.target(algorithm, data)`,
   where `data` is an admissible argument in the call `fit(algorithm, data)`.
 
-- `S` will always be an upper bound on the scitype of observations that could be
+- `S` will always be an upper bound on the scitype of (point) observations that could be
   conceivably extracted from the output of [`predict`](@ref).
 
 To illustate the second case, suppose we have

test/integration/iterative_algorithms.jl

@@ -7,56 +7,57 @@ using Random
 using Statistics
 using StableRNGs
 
-# # ENSEMBLE OF RIDGE REGRESSORS
-
-# We implement a toy algorithm that creates an bagged ensemble of ridge regressors (as
-# defined already in test/integration/regressors.jl), i.e, where each atomic model is
-# trained on a random sample of the training observations (same number, but sampled with
-# replacement). In particular this algorithm has an iteration parameter `n`, and we
-# implement `update` for warm restarts when `n` increases.
-
-# no docstring here - that goes with the constructor
-struct RidgeEnsemble
-    lambda::Float64
-    rng # leaving abstract for simplicity
+# # ENSEMBLE OF REGRESSORS (A MODEL WRAPPER)
+
+# We implement a toy algorithm that creates an bagged ensemble of regressors, i.e, where
+# each atomic model is trained on a random sample of the training observations (same
+# number, but sampled with replacement). In particular this algorithm has an iteration
+# parameter `n`, and we implement `update` for warm restarts when `n` increases.
+
+# no docstring here - that goes with the constructor; some fields left abstract for
+# simplicity
+#
+struct Ensemble
+    atom # the base regressor being bagged
+    rng
     n::Int
 end
 
+# Since the `atom` hyperparameter is another algorithm, it doesn't need a default in the
+# kwarg constructor, but we do need to overload the `LearnAPI.is_composite` trait (done
+# later).
+
 """
-    RidgeEnsemble(; lambda=0.1, rng=Random.default_rng(), n=10)
+    Ensemble(atom; rng=Random.default_rng(), n=10)
 
-Instantiate a RidgeEnsemble algorithm, bla, bla, bla...
+Instantiate a bagged ensemble of `n` regressors, with base regressor `atom`, etc
 
 """
-RidgeEnsemble(; lambda=0.1, rng=Random.default_rng(), n=10) =
-    RidgeEnsemble(lambda, rng, n) # LearnAPI.constructor defined later
+Ensemble(atom; rng=Random.default_rng(), n=10) =
+    Ensemble(atom, rng, n) # `LearnAPI.constructor` defined later
 
-struct RidgeEnsembleFitted
-    algorithm::RidgeEnsemble
+struct EnsembleFitted
+    algorithm::Ensemble
     atom::Ridge
     rng    # mutated copy of `algorithm.rng`
     models # leaving type abstract for simplicity
 end
 
-LearnAPI.algorithm(model::RidgeEnsembleFitted) = model.algorithm
+LearnAPI.algorithm(model::EnsembleFitted) = model.algorithm
 
-# We add the same data interface we provided for `Ridge` in regression.jl. This is an
-# optional step on which the later code does not depend.
-LearnAPI.obs(algorithm::RidgeEnsemble, data) = LearnAPI.obs(Ridge(), data)
-LearnAPI.obs(model::RidgeEnsembleFitted, data) = LearnAPI.obs(first(model.models), data)
-LearnAPI.target(algorithm::RidgeEnsemble, data) = LearnAPI.target(Ridge(), data)
-LearnAPI.features(algorithm::Ridge, data) = LearnAPI.features(Ridge(), data)
+# We add the same data interface that the atomic regressor uses:
+LearnAPI.obs(algorithm::Ensemble, data) = LearnAPI.obs(algorithm.atom, data)
+LearnAPI.obs(model::EnsembleFitted, data) = LearnAPI.obs(first(model.models), data)
+LearnAPI.target(algorithm::Ensemble, data) = LearnAPI.target(algorithm.atom, data)
+LearnAPI.features(algorithm::Ridge, data) = LearnAPI.features(algorithm.atom, data)
 
-function LearnAPI.fit(algorithm::RidgeEnsemble, data; verbosity=1)
+function LearnAPI.fit(algorithm::Ensemble, data; verbosity=1)
 
     # unpack hyperparameters:
-    lambda = algorithm.lambda
-    rng = deepcopy(algorithm.rng) # to prevent mutation of `algorithm`
+    atom = algorithm.atom
+    rng = deepcopy(algorithm.rng) # to prevent mutation of `algorithm`!
     n = algorithm.n
 
-    # instantiate atomic algorithm:
-    atom = Ridge(lambda)
-
     # ensure data can be subsampled using MLUtils.jl, and that we're feeding the atomic
     # `fit` data in an efficient (pre-processed) form:
 
@@ -80,15 +81,16 @@ function LearnAPI.fit(algorithm::RidgeEnsemble, data; verbosity=1)
     # make some noise, if allowed:
     verbosity > 0 && @info "Trained $n ridge regression models. "
 
-    return RidgeEnsembleFitted(algorithm, atom, rng, models)
+    return EnsembleFitted(algorithm, atom, rng, models)
 
 end
 
-# If `n` is increased, this `update` adds new regressors to the ensemble, including any
-# new # hyperparameter updates (e.g, `lambda`) when computing the new
-# regressors. Otherwise, update is equivalent to retraining from scratch, with the
-# provided hyperparameter updates.
-function LearnAPI.update(model::RidgeEnsembleFitted, data; verbosity=1, replacements...)
+# Consistent with the documented `update` contract, we implement this behaviour: If `n` is
+# increased, `update` adds new regressors to the ensemble, including any new
+# hyperparameter updates (e.g, new `atom`) when computing the new atomic
+# models. Otherwise, update is equivalent to retraining from scratch, with the provided
+# hyperparameter updates.
+function LearnAPI.update(model::EnsembleFitted, data; verbosity=1, replacements...)
     :n in keys(replacements) || return fit(model, data)
 
     algorithm_old = LearnAPI.algorithm(model)
@@ -97,7 +99,7 @@ function LearnAPI.update(model::RidgeEnsembleFitted, data; verbosity=1, replacem
     Δn = n - algorithm_old.n
     n < 0 && return fit(model, algorithm)
 
-    atom = Ridge(; lambda=algorithm.lambda)
+    atom = algorithm.atom
     observations = obs(atom, data)
     N = MLUtils.numobs(observations)
 
@@ -116,15 +118,15 @@ function LearnAPI.update(model::RidgeEnsembleFitted, data; verbosity=1, replacem
     # make some noise, if allowed:
     verbosity > 0 && @info "Trained $Δn additional ridge regression models. "
 
-    return RidgeEnsembleFitted(algorithm, atom, rng, models)
+    return EnsembleFitted(algorithm, atom, rng, models)
 end
 
-LearnAPI.predict(model::RidgeEnsembleFitted, ::Point, data) =
+LearnAPI.predict(model::EnsembleFitted, ::Point, data) =
     mean(model.models) do atomic_model
         predict(atomic_model, Point(), data)
     end
 
-LearnAPI.minimize(model::RidgeEnsembleFitted) = RidgeEnsembleFitted(
+LearnAPI.minimize(model::EnsembleFitted) = EnsembleFitted(
     model.algorithm,
     model.atom,
     model.rng,
@@ -133,9 +135,10 @@ LearnAPI.minimize(model::RidgeEnsembleFitted) = RidgeEnsembleFitted(
 
 # note the inclusion of `iteration_parameter`:
 @trait(
-    RidgeEnsemble,
-    constructor = RidgeEnsemble,
+    Ensemble,
+    constructor = Ensemble,
     iteration_parameter = :n,
+    is_composite = true,
     kinds_of_proxy = (Point(),),
     tags = ("regression", "ensemble algorithms", "iterative models"),
     functions = (
@@ -165,7 +168,8 @@ Xtest = Tables.subset(X, test)
 
 @testset "test an implementation of bagged ensemble of ridge regressors" begin
     rng = StableRNG(123)
-    algorithm = RidgeEnsemble(lambda=0.5, n=4; rng)
+    atom = Ridge()
+    algorithm = Ensemble(atom; n=4, rng)
     @test LearnAPI.clone(algorithm) == algorithm
     @test :(LearnAPI.obs) in LearnAPI.functions(algorithm)
     @test LearnAPI.target(algorithm, data) == y
@@ -190,7 +194,6 @@ Xtest = Tables.subset(X, test)
     # compare with cold restart:
     model = fit(LearnAPI.clone(algorithm; n=7), Xtrain, y[train]; verbosity=0);
     @test ŷ7 ≈ predict(model, Xtest)
-
 end
 
 true

test/traits.jl

@@ -1,6 +1,51 @@
-module FruitSalad
+using Test
 using LearnAPI
 
+# A MINIMUM IMPLEMENTATION OF AN ALGORITHM
+
+# does nothing useful
+struct SmallAlgorithm end
+LearnAPI.fit(algorithm::SmallAlgorithm, data; verbosity=1) = algorithm
+LearnAPI.algorithm(algorithm::SmallAlgorithm) = algorithm
+@trait(
+    SmallAlgorithm,
+    constructor = SmallAlgorithm,
+    functions = (
+        :(LearnAPI.fit),
+        :(LearnAPI.algorithm),
+    ),
+)
+######## END OF IMPLEMENTATION ##################
+
+# ZERO ARGUMENT METHODS
+
+@test :(LearnAPI.fit) in LearnAPI.functions()
+@test Point in LearnAPI.kinds_of_proxy()
+@test "regression" in LearnAPI.tags()
+
+# OVERLOADABLE TRAITS
+
+small = SmallAlgorithm()
+@test !LearnAPI.is_pure_julia(small)
+@test LearnAPI.pkg_name(small) == "unknown"
+@test LearnAPI.pkg_license(small) == "unknown"
+@test LearnAPI.load_path(small) == "unknown"
+@test !LearnAPI.is_composite(small)
+@test LearnAPI.human_name(small) == "small algorithm"
+@test isnothing(LearnAPI.iteration_parameter(small))
+@test LearnAPI.data_interface(small) == LearnAPI.RandomAccess()
+@test !(6 isa LearnAPI.fit_observation_scitype(small))
+@test 6 isa LearnAPI.target_observation_scitype(small)
+
+# DERIVED TRAITS
+
+@test LearnAPI.is_algorithm(small)
+@test !LearnAPI.target(small)
+@test !LearnAPI.weights(small)
+
+module FruitSalad
+import LearnAPI
+
 struct RedApple{T}
     x::T
 end