diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d71082e6..10293c22 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -29,7 +29,7 @@ jobs: with: version: ${{ matrix.version }} arch: ${{ matrix.arch }} - - uses: actions/cache@v1 + - uses: julia-actions/cache@v1 env: cache-name: cache-artifacts with: diff --git a/Project.toml b/Project.toml index 9a1351fc..44182acd 100644 --- a/Project.toml +++ b/Project.toml @@ -1,7 +1,7 @@ name = "LearnAPI" uuid = "92ad9a40-7767-427a-9ee6-6e577f1266cb" authors = ["Anthony D. Blaom "] -version = "1.0.1" +version = "2.0.0" [compat] julia = "1.10" diff --git a/README.md b/README.md index 2554142e..3c0d3693 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ Here `learner` specifies the configuration the algorithm (the hyperparameters) w `model` stores learned parameters and any byproducts of algorithm execution. LearnAPI.jl is mostly method stubs and lots of documentation. It does not provide -meta-algorithms, such as cross-validation or hyperparameter optimization, but does aim to +meta-algorithms, such as cross-validation, hyperparameter optimization, or model composition, but does aim to support such algorithms. ## Related packages @@ -37,6 +37,8 @@ support such algorithms. - [StatisticalMeasures.jl](https://github.com/JuliaAI/StatisticalMeasures.jl): Package providing metrics, compatible with LearnAPI.jl +- [StatsModels.jl](https://github.com/JuliaStats/StatsModels.jl): Provides the R-style formula implementation of data preprocessing handled by [LearnDataFrontEnds.jl](https://github.com/JuliaAI/LearnDataFrontEnds.jl) + ### Selected packages providing alternative API's The following alphabetical list of packages provide public base API's. Some provide diff --git a/ROADMAP.md b/ROADMAP.md index 98de84fe..0d66e90f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -14,7 +14,7 @@ "Common Implementation Patterns". As real-world implementations roll out, we could increasingly point to those instead, to conserve effort - [x] regression - - [ ] classification + - [x] classification - [ ] clustering - [x] gradient descent - [x] iterative algorithms diff --git a/docs/Project.toml b/docs/Project.toml index 4ffd503c..30f1981a 100644 --- a/docs/Project.toml +++ b/docs/Project.toml @@ -2,7 +2,6 @@ Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4" DocumenterInterLinks = "d12716ef-a0f6-4df4-a9f1-a5a34e75c656" LearnAPI = "92ad9a40-7767-427a-9ee6-6e577f1266cb" -LearnTestAPI = "3111ed91-c4f2-40e7-bb19-7f6c618409b8" MLCore = "c2834f40-e789-41da-a90e-33b280584a8c" ScientificTypesBase = "30f210dd-8aff-4c5f-94ba-8e64358c1161" Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c" diff --git a/docs/make.jl b/docs/make.jl index 66e71113..dbe4e333 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -2,12 +2,12 @@ using Documenter using LearnAPI using ScientificTypesBase using DocumenterInterLinks -using LearnTestAPI +# using LearnTestAPI const REPO = Remotes.GitHub("JuliaAI", "LearnAPI.jl") makedocs( - modules=[LearnAPI, LearnTestAPI], + modules=[LearnAPI, ], #LearnTestAPI], format=Documenter.HTML( prettyurls = true,#get(ENV, "CI", nothing) == "true", collapselevel = 1, @@ -18,6 +18,7 @@ makedocs( "Reference" => [ "Overview" => "reference.md", "Public Names" => "list_of_public_names.md", + "Kinds of learner" => "kinds_of_learner.md", "fit/update" => "fit_update.md", "predict/transform" => "predict_transform.md", "Kinds of Target Proxy" => "kinds_of_target_proxy.md", diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index 338f61b8..c3f58219 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -1,6 +1,7 @@ # Anatomy of an Implementation -The core LearnAPI.jl pattern looks like this: +LearnAPI.jl supports three core patterns. The default pattern, known as the +[`LearnAPI.Descriminative`](@ref) pattern, looks like this: ```julia model = fit(learner, data) @@ -10,38 +11,51 @@ predict(model, newdata) Here `learner` specifies [hyperparameters](@ref hyperparameters), while `model` stores learned parameters and any byproducts of algorithm execution. -Variations on this pattern: +[Transformers](@ref) ordinarily implement `transform` instead of `predict`. For more on +`predict` versus `transform`, see [Predict or transform?](@ref) -- [Transformers](@ref) ordinarily implement `transform` instead of `predict`. For more on - `predict` versus `transform`, see [Predict or transform?](@ref) +Two other `fit`/`predict`/`transform` patterns supported by LearnAPI.jl are: +[`LearnAPI.Generative`](@ref) which has the form: -- ["Static" (non-generalizing) algorithms](@ref static_algorithms), which includes some - simple transformers and some clustering algorithms, have a `fit` that consumes no - `data`. Instead `predict` or `transform` does the heavy lifting. +```julia +model = fit(learner, data) +predict(model) # a single distribution, for example +``` -- In [density estimation](@ref density_estimation), the `newdata` argument in `predict` is - missing. +and [`LearnAPI.Static`](@ref), which looks like this: + +```julia +model = fit(learner) # no `data` argument +predict(model, data) # may mutate `model` to record byproducts of computation +``` -These are the basic possibilities. +Do not read too much into the names for these patterns, which are formalized [here](@ref kinds_of_learner). Use may not always correspond to prior associations. -Elaborating on the core pattern above, this tutorial details an implementation of the -LearnAPI.jl for naive [ridge regression](https://en.wikipedia.org/wiki/Ridge_regression) -with no intercept. The kind of workflow we want to enable has been previewed in [Sample -workflow](@ref). Readers can also refer to the [demonstration](@ref workflow) of the -implementation given later. +Elaborating on the common `Descriminative` pattern above, this tutorial details an +implementation of the LearnAPI.jl for naive [ridge +regression](https://en.wikipedia.org/wiki/Ridge_regression) with no intercept. The kind of +workflow we want to enable has been previewed in [Sample workflow](@ref). Readers can also +refer to the [demonstration](@ref workflow) of the implementation given later. -## A basic implementation +!!! tip "Quick Start for new implementations" -See [here](@ref code) for code without explanations. + 1. From this tutorial, read at least "[A basic implementation](@ref)" below. + 1. Looking over the examples in "[Common Implementation Patterns](@ref patterns)", identify the appropriate core learner pattern above for your algorithm. + 1. Implement `fit` (probably following an existing example). Read the [`fit`](@ref) document string to see what else may need to be implemented, paying particular attention to the "New implementations" section. + 3. Rinse and repeat with each new method implemented. + 4. Identify any additional [learner traits](@ref traits) that have appropriate overloadings; use the [`@trait`](@ref) macro to define these in one block. + 5. Ensure your implementation includes the compulsory method [`LearnAPI.learner`](@ref) and compulsory traits [`LearnAPI.constructor`](@ref) and [`LearnAPI.functions`](@ref). Read and apply "[Testing your implementation](@ref)". -We suppose our algorithm's `fit` method consumes data in the form `(X, y)`, where -`X` is a suitable table¹ (the features) and `y` a vector (the target). + If you get stuck, refer back to this tutorial and the [Reference](@ref reference) sections. -!!! important - Implementations wishing to support other data - patterns may need to take additional steps explained under - [Other data patterns](@ref di) below. +## A basic implementation + +See [here](@ref code) for code without explanations. + +Let us suppose our algorithm's `fit` method is to consume data in the form `(X, y)`, where +`X` is a suitable table¹ (the features, a.k.a., covariates or predictors) and `y` a vector +(the target, a.k.a., labels or response). The first line below imports the lightweight package LearnAPI.jl whose methods we will be extending. The second imports libraries needed for the core algorithm. @@ -158,6 +172,22 @@ If the kind of proxy is omitted, as in `predict(model, Xnew)`, then a fallback g first element of the tuple returned by [`LearnAPI.kinds_of_proxy(learner)`](@ref), which we overload appropriately below. +### Data deconstructors: `target` and `features` + +LearnAPI.jl is flexible about the form of training `data`. However, to buy into +meta-functionality, such as cross-validation, we'll need to say something about the +structure of this data. We implement [`LearnAPI.target`](@ref) to say what +part of the data constitutes a [target variable](@ref proxy), and +[`LearnAPI.features`](@ref) to say what are the features (valid `newdata` in a +`predict(model, newdata)` call): + +```@example anatomy +LearnAPI.target(learner::Ridge, (X, y)) = y +LearnAPI.features(learner::Ridge, (X, y)) = X +``` + +Another data deconstructor, for learners that support per-observation weights in training, +is [`LearnAPI.weights`](@ref). ### [Accessor functions](@id af) @@ -241,15 +271,11 @@ the *type* of the argument. ### The `functions` trait The last trait, `functions`, above returns a list of all LearnAPI.jl methods that can be -meaningfully applied to the learner or associated model, with the exception of traits. You -always include the first five you see here: `fit`, `learner`, `clone` ,`strip`, -`obs`. Here [`clone`](@ref) is a utility function provided by LearnAPI that you never -overload, while [`obs`](@ref) is discussed under [Providing a separate data front -end](@ref) below and is always included because it has a meaningful fallback. The -`features` method, here provided by a fallback, articulates how the features `X` can be -extracted from the training data `(X, y)`. We must also include `target` here to flag our -model as supervised; again the method itself is provided by a fallback valid in the -present case. +meaningfully applied to the learner or the output of `fit` (denoted `model` above), with +the exception of traits. You always include the first five you see here: `fit`, `learner`, +`clone` ,`strip`, `obs`. Here [`clone`](@ref) is a utility function provided by LearnAPI +that you never overload, while [`obs`](@ref) is discussed under [Providing a separate data +front end](@ref) below and is always included because it has a meaningful fallback. See [`LearnAPI.functions`](@ref) for a checklist of what the `functions` trait needs to return. @@ -340,11 +366,6 @@ assumptions about data from those made above. under [Providing a separate data front end](@ref) below; or (ii) overload the trait [`LearnAPI.data_interface`](@ref) to specify a more relaxed data API. -- Where the form of data consumed by `fit` is different from that consumed by - `predict/transform` (as in classical supervised learning) it may be necessary to - explicitly overload the functions [`LearnAPI.features`](@ref) and (if supervised) - [`LearnAPI.target`](@ref). The same holds if overloading [`obs`](@ref); see below. - ## Providing a separate data front end @@ -414,7 +435,7 @@ The [`obs`](@ref) methods exist to: !!! important - While many new learner implementations will want to adopt a canned data front end, such as those provided by [LearnDataFrontEnds.jl](https://juliaai.github.io/LearnAPI.jl/dev/), we + While many new learner implementations will want to adopt a canned data front end, such as those provided by [LearnDataFrontEnds.jl](https://juliaai.github.io/LearnDataFrontEnds.jl/dev/), we focus here on a self-contained implementation of `obs` for the ridge example above, to show how it works. @@ -448,14 +469,14 @@ newobservations = MLCore.getobs(observations, test_indices) predict(model, newobservations) ``` -which works for any non-static learner implementing `predict`, no matter how one is -supposed to accesses the individual observations of `data` or `newdata`. See also the -demonstration [below](@ref advanced_demo). Furthermore, fallbacks ensure the above pattern -still works if we choose not to implement a front end at all, which is allowed, if -supported `data` and `newdata` already implement `getobs`/`numobs`. +which works for any [`LearnAPI.Descriminative`](@ref) learner implementing `predict`, no +matter how one is supposed to accesses the individual observations of `data` or +`newdata`. See also the demonstration [below](@ref advanced_demo). Furthermore, fallbacks +ensure the above pattern still works if we choose not to implement a front end at all, +which is allowed, if supported `data` and `newdata` already implement `getobs`/`numobs`. -Here we specifically wrap all the preprocessed data into single object, for which we -introduce a new type: +In the ridge regression example we specifically wrap all the preprocessed data into single +object, for which we introduce a new type: ```@example anatomy2 struct RidgeFitObs{T,M<:AbstractMatrix{T}} @@ -476,8 +497,8 @@ function LearnAPI.obs(::Ridge, data) end ``` -We informally refer to the output of `obs` as "observations" (see [The `obs` -contract](@ref) below). The previous core `fit` signature is now replaced with two +We informally refer to the output of `obs` as "observations" (see "[The `obs` +contract](@ref)" below). The previous core `fit` signature is now replaced with two methods - one to handle "regular" input, and one to handle the pre-processed data (observations) which appears first below: @@ -545,13 +566,10 @@ LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = predict(model, Point(), obs(model, Xnew)) ``` -### `features` and `target` methods +### Data deconstructors: `features` and `target` -Two methods [`LearnAPI.features`](@ref) and [`LearnAPI.target`](@ref) articulate how -features and target can be extracted from `data` consumed by LearnAPI.jl -methods. Fallbacks provided by LearnAPI.jl sufficed in our basic implementation -above. Here we must explicitly overload them, so that they also handle the output of -`obs(learner, data)`: +These methods must be able to handle any `data` supported by `fit`, which includes the +output of `obs(learner, data)`: ```@example anatomy2 LearnAPI.features(::Ridge, observations::RidgeFitObs) = observations.A @@ -573,7 +591,7 @@ LearnAPI.target(learner::Ridge, data) = LearnAPI.target(learner, obs(learner, da Since LearnAPI.jl provides fallbacks for `obs` that simply return the unadulterated data argument, overloading `obs` is optional. This is provided data in publicized -`fit`/`predict` signatures already consists only of objects implement the +`fit`/`predict` signatures already consists only of objects implementing the [`LearnAPI.RandomAccess`](@ref) interface (most tables¹, arrays³, and tuples thereof). To opt out of supporting the MLCore.jl interface altogether, an implementation must diff --git a/docs/src/common_implementation_patterns.md b/docs/src/common_implementation_patterns.md index 0c57ff50..db5ff1f2 100644 --- a/docs/src/common_implementation_patterns.md +++ b/docs/src/common_implementation_patterns.md @@ -9,8 +9,10 @@ This guide is intended to be consulted after reading [Anatomy of an Implementati which introduces the main interface objects and terminology. Although an implementation is defined purely by the methods and traits it implements, many -implementations fall into one (or more) of the following informally understood patterns or -tasks: +implementations fall into one (or more) of the informally understood patterns or tasks +below. While some generally fall into one of the core `Descriminative`, `Generative` or +`Static` patterns detailed [here](@id kinds of learner), there are exceptions (such as +clustering, which has both `Descriminative` and `Static` variations). - [Regression](@ref): Supervised learners for continuous targets diff --git a/docs/src/examples.md b/docs/src/examples.md index 49932084..fe690ff1 100644 --- a/docs/src/examples.md +++ b/docs/src/examples.md @@ -24,7 +24,6 @@ end Instantiate a ridge regression learner, with regularization of `lambda`. """ Ridge(; lambda=0.1) = Ridge(lambda) -LearnAPI.constructor(::Ridge) = Ridge # struct for output of `fit` struct RidgeFitted{T,F} @@ -58,6 +57,10 @@ end LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = Tables.matrix(Xnew)*model.coefficients +# data deconstructors: +LearnAPI.target(learner::Ridge, (X, y)) = y +LearnAPI.features(learner::Ridge, (X, y)) = X + # accessor functions: LearnAPI.learner(model::RidgeFitted) = model.learner LearnAPI.coefficients(model::RidgeFitted) = model.named_coefficients @@ -160,7 +163,7 @@ LearnAPI.predict(model::RidgeFitted, ::Point, observations::AbstractMatrix) = LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = predict(model, Point(), obs(model, Xnew)) -# methods to deconstruct training data: +# training data deconstructors: LearnAPI.features(::Ridge, observations::RidgeFitObs) = observations.A LearnAPI.target(::Ridge, observations::RidgeFitObs) = observations.y LearnAPI.features(learner::Ridge, data) = LearnAPI.features(learner, obs(learner, data)) diff --git a/docs/src/features_target_weights.md b/docs/src/features_target_weights.md index e2878672..2d383d05 100644 --- a/docs/src/features_target_weights.md +++ b/docs/src/features_target_weights.md @@ -4,7 +4,7 @@ Methods for extracting certain parts of `data` for all supported calls of the fo [`fit(learner, data)`](@ref). ```julia -LearnAPI.features(learner, data) -> +LearnAPI.features(learner, data) -> LearnAPI.target(learner, data) -> LearnAPI.weights(learner, data) -> ``` @@ -29,11 +29,11 @@ training_loss = sum(ŷ .!= y) # Implementation guide -| method | fallback return value | compulsory? | -|:-------------------------------------------|:---------------------------------------------:|--------------------------| -| [`LearnAPI.features(learner, data)`](@ref) | `first(data)` if `data` is tuple, else `data` | if fallback insufficient | -| [`LearnAPI.target(learner, data)`](@ref) | `last(data)` | if fallback insufficient | -| [`LearnAPI.weights(learner, data)`](@ref) | `nothing` | no | +| method | fallback return value | compulsory? | +|:-------------------------------------------|:---------------------:|-------------| +| [`LearnAPI.features(learner, data)`](@ref) | no fallback | no | +| [`LearnAPI.target(learner, data)`](@ref) | no fallback | no | +| [`LearnAPI.weights(learner, data)`](@ref) | no fallback | no | # Reference diff --git a/docs/src/fit_update.md b/docs/src/fit_update.md index 2329d494..40018986 100644 --- a/docs/src/fit_update.md +++ b/docs/src/fit_update.md @@ -1,15 +1,25 @@ # [`fit`, `update`, `update_observations`, and `update_features`](@id fit_docs) + ### Training ```julia fit(learner, data; verbosity=1) -> model +``` + +This is the typical `fit` pattern, applying in the case that [`LearnAPI.kind_of(learner)`](@ref) +returns one of: + +- [`LearnAPI.Descriminative()`](@ref) +- [`LearnAPI.Generative()`](@ref) + +``` fit(learner; verbosity=1) -> static_model ``` -A "static" algorithm is one that does not generalize to new observations (e.g., some -clustering algorithms); there is no training data and heavy lifting is carried out by -`predict` or `transform` which receive the data. See example below. +This pattern applies in the case [`LearnAPI.kind_of(learner)`](@ref) returns [`LearnAPI.Static()`](@ref). + +Examples appear below. ### Updating @@ -20,7 +30,10 @@ update_observations(model, new_data; verbosity=..., :param1=new_value1, ...) -> update_features(model, new_data; verbosity=..., :param1=new_value1, ...) -> updated_model ``` -## Typical workflows +[`LearnAPI.Static()`](@ref) learners cannot be updated. + + +## [Typical workflows](@id fit_workflows) ### Supervised models @@ -41,6 +54,8 @@ model = update(model; n=150) predict(model, Distribution(), X) ``` +In this case, `LearnAPI.kind_of(learner) == `[`LearnAPI.Descriminative()`](@ref). + See also [Classification](@ref) and [Regression](@ref). ### Transformers @@ -58,6 +73,9 @@ or, if implemented, using a single call: transform(learner, X) # `fit` implied ``` +In this case also, `LearnAPI.kind_of(learner) == `[`LearnAPI.Descriminative()`](@ref). + + ### [Static algorithms (no "learning")](@id static_algorithms) Suppose `learner` is some clustering algorithm that cannot be generalized to new data @@ -74,6 +92,8 @@ labels = predict(learner, X) LearnAPI.extras(model) ``` +In this case `LearnAPI.kind_of(learner) == `[`LearnAPI.Static()`](@ref). + See also [Static Algorithms](@ref) ### [Density estimation](@id density_estimation) @@ -83,7 +103,7 @@ which consumes no data, returns the learned density: ```julia model = fit(learner, y) # no features -predict(model) # shortcut for `predict(model, SingleDistribution())`, or similar +predict(model) # shortcut for `predict(model, Distribution())`, or similar ``` A one-liner will typically be implemented as well: @@ -92,6 +112,8 @@ A one-liner will typically be implemented as well: predict(learner, y) ``` +In this case `LearnAPI.kind_of(learner) == `[`LearnAPI.Generative()`](@ref). + See also [Density Estimation](@ref). diff --git a/docs/src/index.md b/docs/src/index.md index 6f6e1a01..a8fe9c4f 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -94,8 +94,8 @@ Some canned data front ends (implementations of [`obs`](@ref)) are provided by t ## Learning more -- [Anatomy of an Implementation](@ref): informal introduction to the main actors in a new - LearnAPI.jl implementation +- [Anatomy of an Implementation](@ref): informal tutorial introducing the main actors in a + new LearnAPI.jl implementation, including a **Quick Start** for new implementations. - [Reference](@ref reference): official specification diff --git a/docs/src/kinds_of_learner.md b/docs/src/kinds_of_learner.md new file mode 100644 index 00000000..bda42d91 --- /dev/null +++ b/docs/src/kinds_of_learner.md @@ -0,0 +1,8 @@ +# [Kinds of learner](@id kinds_of_learner) + +```@docs +LearnAPI.KindOfLearner +LearnAPI.Descriminative +LearnAPI.Static +LearnAPI.Generative +``` diff --git a/docs/src/kinds_of_target_proxy.md b/docs/src/kinds_of_target_proxy.md index ff9d3f4b..d90095a8 100644 --- a/docs/src/kinds_of_target_proxy.md +++ b/docs/src/kinds_of_target_proxy.md @@ -14,12 +14,6 @@ LearnAPI.KindOfProxy LearnAPI.IID ``` -## Proxies for density estimation algorithms - -```@docs -LearnAPI.Single -``` - ## Joint probability distributions ```@docs diff --git a/docs/src/list_of_public_names.md b/docs/src/list_of_public_names.md index 0e224fe2..f0359508 100644 --- a/docs/src/list_of_public_names.md +++ b/docs/src/list_of_public_names.md @@ -8,6 +8,8 @@ - [`update_observations`](@ref) +- [`update_features`](@ref) + - [`predict`](@ref) - [`transform`](@ref) @@ -34,6 +36,13 @@ See [here](@ref accessor_functions). See [here](@ref traits). +## Kinds of learner + +- [`LearnAPI.Descriminative`](@ref) + +- [`LearnAPI.Static`](@ref) + +- [`LearnAPI.Generative`](@ref) ## Kinds of target proxy diff --git a/docs/src/patterns/density_estimation.md b/docs/src/patterns/density_estimation.md index 74cad18f..e86a50ee 100644 --- a/docs/src/patterns/density_estimation.md +++ b/docs/src/patterns/density_estimation.md @@ -1,5 +1,26 @@ # Density Estimation -See these examples from the JuliaTestAPI.jl test suite: +In density estimators, `fit` is trained only on [target data](@ref proxy), and `predict` +consumes no data at all, a pattern flagged by the identities [`LearnAPI.target(learner, +y)`](@ref)` == y` and [`LearnAPI.kind_of(learner)`](@ref)` == +`[`LearnAPI.Generative()`](@ref), respetively. + + +Typically `predict` returns a single probability density/mass function. Here's a sample +workflow: + +```julia +model = fit(learner, y) # no features +predict(model) # shortcut for `predict(model, Distribution())`, or similar +``` + +A one-line convenience method will typically be implemented as well: + +```julia +predict(learner, y) +``` + +However, having the multi-line workflow enables the possibility of updating the model with +new data. See this example from the JuliaTestAPI.jl test suite: - [normal distribution estimator](https://github.com/JuliaAI/LearnTestAPI.jl/blob/dev/src/learners/incremental_algorithms.jl) diff --git a/docs/src/predict_transform.md b/docs/src/predict_transform.md index 2733fc64..a7db8251 100644 --- a/docs/src/predict_transform.md +++ b/docs/src/predict_transform.md @@ -1,12 +1,26 @@ # [`predict`, `transform` and `inverse_transform`](@id operations) ```julia -predict(model, kind_of_proxy, data) +predict(model, [kind_of_proxy,] data) transform(model, data) inverse_transform(model, data) ``` -Versions without the `data` argument may apply, for example in [density +The above signatures apply in the case that [`LearnAPI.kind_of(learner)`](@ref) +returns one of: + +- [`LearnAPI.Descriminative()`](@ref) +- [`LearnAPI.Static()`](@ref) + + +```julia +predict(model[, kind_of_proxy]) +transform(model) +inverse_transform(model) +``` + +The above signatures apply in the case that [`LearnAPI.kind_of(learner)`](@ref) +returns [`LearnAPI.Generative()`](@ref), as in [density estimation](@ref density_estimation). ## [Typical worklows](@id predict_workflow) diff --git a/docs/src/reference.md b/docs/src/reference.md index a4499e55..e77d8cdd 100644 --- a/docs/src/reference.md +++ b/docs/src/reference.md @@ -40,11 +40,11 @@ number of user-specified *hyperparameters*, such as the number of trees in a ran forest. Hyperparameters are understood in a rather broad sense. For example, one is allowed to have hyperparameters that are not data-generic. For example, a class weight dictionary, which will only make sense for a target taking values in the set of specified -dictionary keys, should be given as a hyperparameter. For simplicity and composability, -LearnAPI.jl discourages "run time" parameters (extra arguments to `fit`) such as -acceleration options (cpu/gpu/multithreading/multiprocessing). These should be included as -hyperparameters as far as possible. An exception is the compulsory `verbosity` keyword -argument of `fit`. +dictionary keys, should be given as a hyperparameter. For simplicity and easier +composability, LearnAPI.jl discourages "run time" parameters (extra arguments to `fit`) +such as acceleration options (cpu/gpu/multithreading/multiprocessing). These should be +included as hyperparameters as far as possible. An exception is the compulsory `verbosity` +keyword argument of `fit`. ### [Targets and target proxies](@id proxy) @@ -107,6 +107,16 @@ generally requires overloading `Base.==` for the struct. deep copies of RNG hyperparameters before using them in an implementation of [`fit`](@ref). + +#### Kinds of learner + +As previewed in [Anatomy of an Implementation](@ref), different +`fit`/`predict`/`transform` patterns lead to a division of learners into three distinct +kinds, [`LearnAPI.Descriminative()`](@ref), [`LearnAPI.Generative`](@ref), and +[`LearnAPI.Static`](@ref), which is detailed [here](@ref kinds_of_learner). See also +[these workflows](@ref fit_workflows) for concrete examples. + + #### Composite learners (wrappers) A *composite learner* is one with at least one property that can take other learners as @@ -122,19 +132,19 @@ Below is an example of a learner type with a valid constructor: ```julia struct GradientRidgeRegressor{T<:Real} - learning_rate::T - epochs::Int - l2_regularization::T + learning_rate::T + epochs::Int + l2_regularization::T end """ - GradientRidgeRegressor(; learning_rate=0.01, epochs=10, l2_regularization=0.01) + GradientRidgeRegressor(; learning_rate=0.01, epochs=10, l2_regularization=0.01) Instantiate a gradient ridge regressor with the specified hyperparameters. """ GradientRidgeRegressor(; learning_rate=0.01, epochs=10, l2_regularization=0.01) = - GradientRidgeRegressor(learning_rate, epochs, l2_regularization) + GradientRidgeRegressor(learning_rate, epochs, l2_regularization) LearnAPI.constructor(::GradientRidgeRegressor) = GradientRidgeRegressor ``` diff --git a/docs/src/testing_an_implementation.md b/docs/src/testing_an_implementation.md index cc0d58f6..6dd625cc 100644 --- a/docs/src/testing_an_implementation.md +++ b/docs/src/testing_an_implementation.md @@ -1,55 +1,57 @@ # Testing an Implementation -Testing is provided by the LearnTestAPI.jl package documented below. +Testing is provided by the LearnTestAPI.jl package. -## Quick start + -```@docs -LearnTestAPI -``` + -!!! warning + + + - New releases of LearnTestAPI.jl may add tests to `@testapi`, and - this may result in new failures in client package test suites, because - of previously undetected broken contracts. Adding a test to `@testapi` - is not considered a breaking change - to LearnTestAPI, unless it supports a breaking change to LearnAPI.jl. + + + + + + -## The @testapi macro -```@docs -LearnTestAPI.@testapi -``` + -## Learners for testing + + + -LearnTestAPI.jl provides some simple, tested, LearnAPI.jl implementations, which may be -useful for testing learner wrappers and meta-algorithms. + -```@docs -LearnTestAPI.Ridge -LearnTestAPI.BabyRidge -LearnTestAPI.ConstantClassifier -LearnTestAPI.TruncatedSVD -LearnTestAPI.Selector -LearnTestAPI.FancySelector -LearnTestAPI.NormalEstimator -LearnTestAPI.Ensemble -LearnTestAPI.StumpRegressor -``` + + -## Private methods + + + + + + + + + + + -For LearnTestAPI.jl developers only, and subject to breaking changes at any time: + -```@docs -LearnTestAPI.@logged_testset -LearnTestAPI.@nearly -LearnTestAPI.isnear -LearnTestAPI.learner_get -LearnTestAPI.model_get -LearnTestAPI.verb -LearnTestAPI.filter_out_verbosity -``` + + + + + + + + + + + diff --git a/docs/src/traits.md b/docs/src/traits.md index 890a53f9..16077a8f 100644 --- a/docs/src/traits.md +++ b/docs/src/traits.md @@ -1,4 +1,4 @@ -# [Learner Traits](@id traits) +2# [Learner Traits](@id traits) Learner traits are simply functions whose sole argument is a learner. @@ -13,24 +13,25 @@ training). They may also record more mundane information, such as a package lice In the examples column of the table below, `Continuous` is a name owned the package [ScientificTypesBase.jl](https://github.com/JuliaAI/ScientificTypesBase.jl/). -| trait | return value | fallback value | example | -|:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------|:---------------------------------------------------------------| -| [`LearnAPI.constructor`](@ref)`(learner)` | constructor for generating new or modified versions of `learner` | (no fallback) | `RidgeRegressor` | -| [`LearnAPI.functions`](@ref)`(learner)` | functions you can apply to `learner` or associated model (traits excluded) | `()` | `(:fit, :predict, :LearnAPI.strip, :(LearnAPI.learner), :obs)` | -| [`LearnAPI.kinds_of_proxy`](@ref)`(learner)` | instances `kind` of `KindOfProxy` for which an implementation of `LearnAPI.predict(learner, kind, ...)` is guaranteed. | `()` | `(Distribution(), Interval())` | -| [`LearnAPI.tags`](@ref)`(learner)` | lists one or more suggestive learner tags from `LearnAPI.tags()` | `()` | (:regression, :probabilistic) | -| [`LearnAPI.is_pure_julia`](@ref)`(learner)` | `true` if implementation is 100% Julia code | `false` | `true` | -| [`LearnAPI.pkg_name`](@ref)`(learner)` | name of package providing core code (may be different from package providing LearnAPI.jl implementation) | `"unknown"` | `"DecisionTree"` | -| [`LearnAPI.pkg_license`](@ref)`(learner)` | name of license of package providing core code | `"unknown"` | `"MIT"` | -| [`LearnAPI.doc_url`](@ref)`(learner)` | url providing documentation of the core code | `"unknown"` | `"https://en.wikipedia.org/wiki/Decision_tree_learning"` | -| [`LearnAPI.load_path`](@ref)`(learner)` | string locating name returned by `LearnAPI.constructor(learner)`, beginning with a package name | `"unknown"` | `FastTrees.LearnAPI.DecisionTreeClassifier` | -| [`LearnAPI.nonlearners`](@ref)`(learner)` | properties *not* corresponding to other learners | all properties | `(:K, :leafsize, :metric,)` | -| [`LearnAPI.human_name`](@ref)`(learner)` | human name for the learner; should be a noun | type name with spaces | "elastic net regressor" | -| [`LearnAPI.iteration_parameter`](@ref)`(learner)` | symbolic name of an iteration parameter | `nothing` | :epochs | +| trait | return value | fallback value | example | +|:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------|:---------------------------------------------------------------| +| [`LearnAPI.constructor`](@ref)`(learner)` | constructor for generating new or modified versions of `learner` | (no fallback) | `RidgeRegressor` | +| [`LearnAPI.functions`](@ref)`(learner)` | functions you can apply to `learner` or associated model (traits excluded) | `()` | `(:fit, :predict, :LearnAPI.strip, :(LearnAPI.learner), :obs)` | +| [`LearnAPI.kind_of`](@ref)`(learner)` | the `fit`/`predict`/`transform` pattern used by `learner` | `LearnAPI.Descriminative()` | `LearnAPI.Static()` | +| [`LearnAPI.kinds_of_proxy`](@ref)`(learner)` | instances `kind` of `KindOfProxy` for which an implementation of `LearnAPI.predict(learner, kind, ...)` is guaranteed. | `()` | `(Distribution(), Interval())` | +| [`LearnAPI.tags`](@ref)`(learner)` | lists one or more suggestive learner tags from `LearnAPI.tags()` | `()` | (:regression, :probabilistic) | +| [`LearnAPI.is_pure_julia`](@ref)`(learner)` | `true` if implementation is 100% Julia code | `false` | `true` | +| [`LearnAPI.pkg_name`](@ref)`(learner)` | name of package providing core code (may be different from package providing LearnAPI.jl implementation) | `"unknown"` | `"DecisionTree"` | +| [`LearnAPI.pkg_license`](@ref)`(learner)` | name of license of package providing core code | `"unknown"` | `"MIT"` | +| [`LearnAPI.doc_url`](@ref)`(learner)` | url providing documentation of the core code | `"unknown"` | `"https://en.wikipedia.org/wiki/Decision_tree_learning"` | +| [`LearnAPI.load_path`](@ref)`(learner)` | string locating name returned by `LearnAPI.constructor(learner)`, beginning with a package name | `"unknown"` | `FastTrees.LearnAPI.DecisionTreeClassifier` | +| [`LearnAPI.nonlearners`](@ref)`(learner)` | properties *not* corresponding to other learners | all properties | `(:K, :leafsize, :metric,)` | +| [`LearnAPI.human_name`](@ref)`(learner)` | human name for the learner; should be a noun | type name with spaces | "elastic net regressor" | +| [`LearnAPI.iteration_parameter`](@ref)`(learner)` | symbolic name of an iteration parameter | `nothing` | :epochs | | [`LearnAPI.data_interface`](@ref)`(learner)` | Interface implemented by objects returned by [`obs`](@ref) | `Base.HasLength()` (supports `MLCore.getobs/numobs`) | `Base.SizeUnknown()` (supports `iterate`) | -| [`LearnAPI.fit_scitype`](@ref)`(learner)` | upper bound on `scitype(data)` ensuring `fit(learner, data)` works | `Union{}` | `Tuple{AbstractVector{Continuous}, Continuous}` | -| [`LearnAPI.target_observation_scitype`](@ref)`(learner)` | upper bound on the scitype of each observation of the targget | `Any` | `Continuous` | -| [`LearnAPI.is_static`](@ref)`(learner)` | `true` if `fit` consumes no data | `false` | `true` | +| [`LearnAPI.fit_scitype`](@ref)`(learner)` | upper bound on `scitype(data)` ensuring `fit(learner, data)` works | `Union{}` | `Tuple{AbstractVector{Continuous}, Continuous}` | +| [`LearnAPI.target_observation_scitype`](@ref)`(learner)` | upper bound on the scitype of each observation of the targget | `Any` | `Continuous` | +| [`LearnAPI.is_static`](@ref)`(learner)` | `true` if `fit` consumes no data | `false` | `true` | ### Derived Traits @@ -94,6 +95,7 @@ informative (as in `LearnAPI.target_observation_scitype(learner) = Any`). ```@docs LearnAPI.constructor LearnAPI.functions +LearnAPI.kind_of LearnAPI.kinds_of_proxy LearnAPI.tags LearnAPI.is_pure_julia diff --git a/src/features_target_weights.jl b/src/features_target_weights.jl index 578772fa..9fae00f9 100644 --- a/src/features_target_weights.jl +++ b/src/features_target_weights.jl @@ -22,14 +22,16 @@ the LearnAPI.jl documentation. ## New implementations -A fallback returns `last(data)`. The method must be overloaded if [`fit`](@ref) consumes -data that includes a target variable and this fallback fails to fulfill the contract stated -above. +The method should be overloaded if [`fit`](@ref) consumes data that includes a target +variable (in the sense above). This will include both [`LearnAPI.Descriminative`](@ref) +and [`LearnAPI.Generative`](@ref) learners, but never [`LearnAPI.Static`](@ref) learners. +Implementation allows for certain meta-functionality, such as cross-validation in +supervised learning, supervised anomaly detection, and density estimation. If `obs` is being overloaded, then typically it suffices to overload `LearnAPI.target(learner, observations)` where `observations = obs(learner, data)` and `data` is any documented supported `data` in calls of the form [`fit(learner, -data)`](@ref), and to add a declaration of the form +data)`](@ref), and to then add a declaration of the form ```julia LearnAPI.target(learner, data) = LearnAPI.target(learner, obs(learner, data)) @@ -39,10 +41,10 @@ to catch all other forms of supported input `data`. Remember to ensure the return value of `LearnAPI.target` implements the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). -$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.target)"; overloaded=true)) +$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.target)"; overloaded=false)) """ -target(::Any, data) = last(data) +function target end """ LearnAPI.weights(learner, data) -> weights @@ -50,9 +52,8 @@ target(::Any, data) = last(data) Return, for each form of `data` supported by the call [`fit(learner, data)`](@ref), the per-observation weights part of `data`. -The returned object has the same number of observations -as `data` has and is guaranteed to implement the data interface specified by -[`LearnAPI.data_interface(learner)`](@ref). +The returned object has the same number of observations as `data` has and is guaranteed to +implement the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). Where `nothing` is returned, weighting is understood to be uniform. @@ -60,9 +61,9 @@ Where `nothing` is returned, weighting is understood to be uniform. # New implementations -Overloading is optional. A fallback returns `nothing`. +Implementing is optional. -If `obs` is being overloaded, then typically it suffices to overload +If `obs` is being implemented, then typically it suffices to overload `LearnAPI.weights(learner, observations)` where `observations = obs(learner, data)` and `data` is any documented supported `data` in calls of the form [`fit(learner, data)`](@ref), and to add a declaration of the form @@ -75,10 +76,10 @@ to catch all other forms of supported input `data`. Ensure the returned object, unless `nothing`, implements the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). -$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.weights)"; overloaded=true)) +$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.weights)"; overloaded=false)) """ -weights(::Any, data) = nothing +function weights end """ LearnAPI.features(learner, data) @@ -86,8 +87,8 @@ weights(::Any, data) = nothing Return, for each form of `data` supported by the call [`fit(learner, data)`](@ref), the features part `X` of `data`. -While "features" will typically have the commonly understood meaning, the only -learner-generic guaranteed properties of `X` are: +While "features" will typically have the commonly understood meaning ("covariates" or +"prediuctors"), the only learner-generic guaranteed properties of `X` are: - `X` can be passed to [`predict`](@ref) or [`transform`](@ref) when these are supported by `learner`, as in the call `predict(model, X)`, where `model = fit(learner, data)`. @@ -95,18 +96,13 @@ learner-generic guaranteed properties of `X` are: - `X` has the same number of observations as `data` has and is guaranteed to implement the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). -Where `nothing` is returned, `predict` and `transform` consume no data. - # Extended help # New implementations -A fallback returns `first(data)` if `data` is a tuple, and otherwise returns `data`. The -method has no meaning for static learners (where `data` is not an argument of `fit`) and -otherwise an implementation needs to overload this method if the fallback is inadequate. - -For density estimators, whose `fit` typically consumes *only* a target variable, you -should overload this method to always return `nothing`. +Implementation of this method allows for certain meta-functionality, such as +cross-validation. It can only be implemented for [`LearnAPI.Descriminative`](@ref) +learners. If `obs` is being overloaded, then typically it suffices to overload `LearnAPI.features(learner, observations)` where `observations = obs(learner, data)` and @@ -118,15 +114,14 @@ LearnAPI.features(learner, data) = LearnAPI.features(learner, obs(learner, data) ``` to catch all other forms of supported input `data`. -Ensure the returned object, unless `nothing`, implements the data interface specified by +Ensure the returned object, implements the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). `:(LearnAPI.features)` must be included in the return value of [`LearnAPI.functions(learner)`](@ref), unless the learner is static (`fit` consumes no data). +$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.target)"; overloaded=false)) + """ -features(learner, data) = _first(data) -_first(data) = data -_first(data::Tuple) = first(data) -# note the factoring above guards against method ambiguities +function features end diff --git a/src/fit_update.jl b/src/fit_update.jl index 39f78273..b72600a4 100644 --- a/src/fit_update.jl +++ b/src/fit_update.jl @@ -4,11 +4,11 @@ fit(learner, data; verbosity=1) fit(learner; verbosity=1) -Execute the machine learning or statistical algorithm with configuration `learner` using -the provided training `data`, returning an object, `model`, on which other methods, such -as [`predict`](@ref) or [`transform`](@ref), can be dispatched. -[`LearnAPI.functions(learner)`](@ref) returns a list of methods that can be applied to -either `learner` or `model`. +In the case of the first signature, execute the machine learning or statistical algorithm +with configuration `learner` using the provided training `data`, returning an object, +`model`, on which other methods, such as [`predict`](@ref) or [`transform`](@ref), can be +dispatched. [`LearnAPI.functions(learner)`](@ref) returns a list of methods that can be +applied to either `learner` or `model`. For example, a supervised classifier might have a workflow like this: @@ -17,38 +17,54 @@ model = fit(learner, (X, y)) ŷ = predict(model, Xnew) ``` -The signature `fit(learner; verbosity=...)` (no `data`) is provided by learners that do -not generalize to new observations (called *static algorithms*). In that case, -`transform(model, data)` or `predict(model, ..., data)` carries out the actual algorithm -execution, writing any byproducts of that operation to the mutable object `model` returned -by `fit`. Inspect the value of [`LearnAPI.is_static(learner)`](@ref) to determine whether -`fit` consumes `data` or not. - Use `verbosity=0` for warnings only, and `-1` for silent training. -See also [`predict`](@ref), [`transform`](@ref), -[`inverse_transform`](@ref), [`LearnAPI.functions`](@ref), [`obs`](@ref). +This `fit` signature applies to all learners for which [`LearnAPI.kind_of(learner)`](@ref) +returns [`LearnAPI.Descriminative()`](@ref) or [`LearnAPI.Generative()`](@ref). + +# Static learners + +In the case of a learner that does not generalize to new data, the second `fit` signature +can be used to wrap the `learner` in an object called `model` that the calls +`transform(model, data)` or `predict(model, ..., data)` may mutate, so as to record +byproducts of the core algorithm specified by `learner`, before returning the outcomes of +primary interest. + +Here's a sample workflow: + +```julia +model = fit(learner) # e.g, `learner` specifies DBSCAN clustering parameters +labels = predict(model, X) # compute and return cluster labels for `X` +LearnAPI.extras(model) # return outliers in the data `X` +``` +This `fit` signature applies to all learners for which +[`LearnAPI.kind_of(learner)`](@ref)` == `[`LearnAPI.Static()`](@ref). + +See also [`predict`](@ref), [`transform`](@ref), [`inverse_transform`](@ref), +[`LearnAPI.functions`](@ref), [`obs`](@ref), [`LearnAPI.kind_of`](@ref). # Extended help # New implementations -Implementation of exactly one of the signatures is compulsory. If `fit(learner; -verbosity=...)` is implemented, then the trait [`LearnAPI.is_static`](@ref) must be -overloaded to return `true`. - -The signature must include `verbosity` with `1` as default. +Implementation of exactly one of the signatures is compulsory. Unless implementing the +[`LearnAPI.Descriminative()`](@ref) `fit`/`predict`/`transform` pattern, +[`LearnAPI.kind_of(learner)`](@ref) will need to be suitably overloaded. -If `data` encapsulates a *target* variable, as defined in LearnAPI.jl documentation, then -[`LearnAPI.target`](@ref) must be implemented. If [`predict`](@ref) or [`transform`](@ref) -are implemented and consume data, then you made need to overload -[`LearnAPI.features`](@ref). +The `fit` signature must include `verbosity` with `1` as default. The LearnAPI.jl specification has nothing to say regarding `fit` signatures with more than two arguments. For convenience, for example, an implementation is free to implement a slurping signature, such as `fit(learner, X, y, extras...) = fit(learner, (X, y, extras...))` but LearnAPI.jl does not guarantee such signatures are actually implemented. +## The `target` and `features` methods + +If [`LearnAPI.kind_of(learner)`](@ref) returns [`LearnAPI.Descriminative()`](@ref) or +[`LearnAPI.Generative()`](@ref) then the methods [`LearnAPI.target`](@ref) and/or +[`LearnAPI.features`](@ref), which deconstruct the form of `data` consumed by `fit`, may +require overloading. Refer to their document strings for details. + $(DOC_DATA_INTERFACE(:fit)) """ @@ -92,6 +108,7 @@ Implementation is optional. The signature must include `verbosity`. It should be `LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and `newlearner = LearnAPI.clone(learner, replacements...)`. +Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.update)")) @@ -132,6 +149,8 @@ Implementation is optional. The signature must include `verbosity`. It should be `LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and `newlearner = LearnAPI.clone(learner, replacements...)`. +Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. + $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.update_observations)")) See also [`LearnAPI.clone`](@ref). @@ -162,6 +181,8 @@ Implementation is optional. The signature must include `verbosity`. It should be `LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and `newlearner = LearnAPI.clone(learner, replacements...)`. +Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. + $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.update_features)")) See also [`LearnAPI.clone`](@ref). diff --git a/src/predict_transform.jl b/src/predict_transform.jl index 0a92d3f5..f097a976 100644 --- a/src/predict_transform.jl +++ b/src/predict_transform.jl @@ -46,7 +46,7 @@ DOC_DATA_INTERFACE(method) = case then an implementation must either: (i) overload [`obs`](@ref) to articulate how provided data can be transformed into a form that does support [`LearnAPI.RandomAccess`](@ref); or (ii) overload the trait - [`LearnAPI.data_interface`](@ref) to specify a more relaxed data API. Refer tbo + [`LearnAPI.data_interface`](@ref) to specify a more relaxed data API. Refer to the document strings for details. """ diff --git a/src/traits.jl b/src/traits.jl index 54cdacfc..655b2289 100644 --- a/src/traits.jl +++ b/src/traits.jl @@ -84,23 +84,23 @@ functions owned by LearnAPI.jl. All new implementations must implement this trait. Here's a checklist for elements in the return value: -| expression | implementation compulsory? | include in returned tuple? | -|:----------------------------------|:---------------------------|:---------------------------------| -| `:(LearnAPI.fit)` | yes | yes | -| `:(LearnAPI.learner)` | yes | yes | -| `:(LearnAPI.clone)` | never overloaded | yes | -| `:(LearnAPI.strip)` | no | yes | -| `:(LearnAPI.obs)` | no | yes | -| `:(LearnAPI.features)` | no | yes, unless `learner` is static | -| `:(LearnAPI.target)` | no | only if implemented | -| `:(LearnAPI.weights)` | no | only if implemented | -| `:(LearnAPI.update)` | no | only if implemented | -| `:(LearnAPI.update_observations)` | no | only if implemented | -| `:(LearnAPI.update_features)` | no | only if implemented | -| `:(LearnAPI.predict)` | no | only if implemented | -| `:(LearnAPI.transform)` | no | only if implemented | -| `:(LearnAPI.inverse_transform)` | no | only if implemented | -| < accessor functions> | no | only if implemented | +| expression | implementation compulsory? | include in returned tuple? | +|:----------------------------------|:---------------------------|:---------------------------| +| `:(LearnAPI.fit)` | yes | yes | +| `:(LearnAPI.learner)` | yes | yes | +| `:(LearnAPI.clone)` | never overloaded | yes | +| `:(LearnAPI.strip)` | no | yes | +| `:(LearnAPI.obs)` | no | yes | +| `:(LearnAPI.features)` | no | only if implemented | +| `:(LearnAPI.target)` | no | only if implemented | +| `:(LearnAPI.weights)` | no | only if implemented | +| `:(LearnAPI.update)` | no | only if implemented | +| `:(LearnAPI.update_observations)` | no | only if implemented | +| `:(LearnAPI.update_features)` | no | only if implemented | +| `:(LearnAPI.predict)` | no | only if implemented | +| `:(LearnAPI.transform)` | no | only if implemented | +| `:(LearnAPI.inverse_transform)` | no | only if implemented | +| < accessor functions> | no | only if implemented | Also include any implemented accessor functions, both those owned by LearnaAPI.jl, and any learner-specific ones. The LearnAPI.jl accessor functions are: $ACCESSOR_FUNCTIONS_LIST @@ -152,6 +152,19 @@ macro functions(learner) end |> esc end +""" + LearnAPI.kind_of(learner) + +Return the `fit`/`predict`/`transform` signature pattern used by `learner`. See +[`KindOfLearner`](@ref) for details. + +# New implementations + +The fallback value is [`LearnAPI.Descriminative()`]. + +""" +kind_of(learner) = Descriminative() + """ LearnAPI.kinds_of_proxy(learner) diff --git a/src/types.jl b/src/types.jl index ec94fdb2..43dd420e 100644 --- a/src/types.jl +++ b/src/types.jl @@ -1,6 +1,136 @@ +# # KIND OF LEARNER + +# see later for doc-string: +abstract type KindOfLearner end + +""" + LearnAPI.Descriminative + +Type with a single instance, `LearnAPI.Descriminative()`. + +If [`LearnAPI.kind_of(learner)`](@ref)` == LearnAPI.Descriminative()`, then the only possible +signatures of [`fit`](@ref), [`predict`](@ref) and [`transform`](@ref) are those appearing +below, or variations on these in which keyword arguments are also supplied: + +``` +model = fit(learner, data) +predict(model, new_data) +predict(model, kop::KindOfProxy, new_data) +transform(model, new_data) +``` + +and the one-line convenience forms + +``` +predict(learner, data) +predict(learner, kop::KindOfProxy, new_data) +transform(learner, data) +``` + +See also [`LearnAPI.Static`](@ref), [`LearnAPI.Generative`](@ref). + +""" +struct Descriminative <: KindOfLearner end + +""" + LearnAPI.Static + +Type with a single instance, `LearnAPI.Static()`. + +If [`LearnAPI.kind_of(learner)`](@ref)` == LearnAPI.Static()`, then the only possible +signatures of [`fit`](@ref), [`predict`](@ref) and [`transform`](@ref) are those appearing +below, or variations on these in which keyword arguments are also supplied: + +``` +model = fit(learner) # no `data` argument +predict(model, data) # may mutate `model` +predict(model, kop::KindOfProxy, data) # may mutate `model` +transform(model, data) # may mutate `model` +``` + +and the one-line convenience forms + +``` +predict(learner, data) +predict(learner, kop::KindOfProxy) +transform(learner, data) +``` + +See also [`LearnAPI.Descriminative`](@ref), [`LearnAPI.Generative`](@ref). + +""" +struct Static <: KindOfLearner end + +""" + LearnAPI.Generative + +Type with a single instance, `LearnAPI.Generative()`. + +If [`LearnAPI.kind_of(learner)`](@ref)` == LearnAPI.Generative()`, then the only possible +signatures of [`fit`](@ref), [`predict`](@ref) and [`transform`](@ref) are those appearing +below, or variations on these in which keyword arguments are also supplied: + +``` +model = fit(learner, data) +predict(model) # no `newdata` argument +predict(model, kop::KindOfProxy) # no `newdata` argument +transform(model) # no `newdata` argument +``` + +and the one-line convenience forms + +``` +predict(learner, data) +predict(learner, kop::KindOfProxy, data) +transform(learner, data) +``` + +See also [`LearnAPI.Descriminative`](@ref), [`LearnAPI.Static`](@ref). +""" +struct Generative <: KindOfLearner end + + +""" + LearnAPI.KindOfLearner + +Abstract type whose instances are the possible values of +[`LearnAPI.kind_of(learner)`](@ref). All instances of this type, and brief indications of +their interpretation, appear below. + +[`LearnAPI.Descriminative()`](@ref): A typical workflow looks like: + +``` +model = fit(learner, data) +predict(learner, new_data) +# or +transform(learner, new_data) +``` + +[`LearnAPI.Static()`](@ref): A typical workflow looks like: + +``` +model = fit(learner) +predict(model, data) # may mutate `model` to record byproducts of computation +# or +transform(model, data) +``` + +[`LearnAPI.Generative()`](@ref): A typical workflow looks like: + +``` +model = fit(learner, data) +predict(learner) # e.g., returns a single probability distribution +``` + +For precise details, refer to the document strings for [`LearnAPI.Descriminative`](@ref), +[`LearnAPI.Static`](@ref), and [`LearnAPI.Generative`](@ref). +""" +KindOfLearner + + # # TARGET PROXIES -# see later for doc string: +# see later for doc-string: abstract type KindOfProxy end """ @@ -16,6 +146,12 @@ following must hold: - The ``j``th observation of `ŷ`, for any ``j``, depends only on the ``j``th observation of the provided `data` (no correlation between observations). +An exception holds in the case that [`LearnAPI.kind_of(learner)`](@ref)` == +[`LearnAPI.Generative()`](@ref): + +- `LearnAPI.predict(model, kind_of_proxy)` consists of a single observation (such as a + single probability distribution). + See also [`LearnAPI.KindOfProxy`](@ref). # Extended help @@ -113,40 +249,8 @@ for S in JOINT_SYMBOLS end |> eval end -""" - Single <: KindOfProxy - -Abstract subtype of [`LearnAPI.KindOfProxy`](@ref). It applies only to learners for -which `predict` has no data argument, i.e., is of the form `predict(model, -kind_of_proxy)`. An example is an algorithm learning a probability distribution from -samples, and we regard the samples as drawn from the "target" variable. If in this case, -`kind_of_proxy` is an instance of `LearnAPI.Single` then, `predict(learner)` returns a -single object representing a probability distribution. - -| type `T` | form of output of `predict(model, ::T)` | -|:--------------------------------:|:-----------------------------------------------------------------------| -| `SingleSampleable` | object that can be sampled to obtain a single target observation | -| `SingleDistribution` | explicit probability density/mass function for sampling the target | -| `SingleLogDistribution` | explicit log-probability density/mass function for sampling the target | - -""" -abstract type Single <: KindOfProxy end - -const SINGLE_SYMBOLS = [ - :SingleSampeable, - :SingleDistribution, - :SingleLogDistribution, -] - -for S in SINGLE_SYMBOLS - quote - struct $S <: Single end - end |> eval -end - const CONCRETE_TARGET_PROXY_SYMBOLS = [ IID_SYMBOLS..., - SINGLE_SYMBOLS..., JOINT_SYMBOLS..., ] @@ -166,17 +270,16 @@ are probability density/mass functions, assuming `learner = LearnAPI.learner(mod supports predictions of that form, which is true if `Distribution() in` [`LearnAPI.kinds_of_proxy(learner)`](@ref). -Proxy types are grouped under three abstract subtypes: +Proxy types are grouped under two abstract subtypes: - [`LearnAPI.IID`](@ref): The main type, for proxies consisting of uncorrelated individual - components, one for each input observation + components, one for each input observation. The type also applies to learners, such as + density estimators, that are trained on a target variable only (no features), and where + `predict` consumes no data and the returned target proxy is a single observation (e.g., + a single probability mass function) - [`LearnAPI.Joint`](@ref): For learners that predict a single probabilistic structure - encapsulating correlations between target predictions for different input observations - -- [`LearnAPI.Single`](@ref): For learners, such as density estimators, that are trained on - a target variable only (no features); `predict` consumes no data and the returned target - proxy is a single probabilistic structure. + encapsulating correlations between target predictions for different input observations. For lists of all concrete instances, refer to documentation for the relevant subtype. diff --git a/test/features_target_weights.jl b/test/features_target_weights.jl deleted file mode 100644 index 4809f5df..00000000 --- a/test/features_target_weights.jl +++ /dev/null @@ -1,11 +0,0 @@ -using Test -using LearnAPI - -struct Avocado end - -@test LearnAPI.target(Avocado(), (1, 2, 3)) == 3 -@test isnothing(LearnAPI.weights(Avocado(), "salsa")) -@test LearnAPI.features(Avocado(), "salsa") == "salsa" -@test LearnAPI.features(Avocado(), (:X, :y)) == :X - -true diff --git a/test/runtests.jl b/test/runtests.jl index e8117976..4b64816c 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -7,7 +7,6 @@ test_files = [ "predict_transform.jl", "obs.jl", "accessor_functions.jl", - "features_target_weights.jl", ] files = isempty(ARGS) ? test_files : ARGS diff --git a/test/traits.jl b/test/traits.jl index 0a7023dd..6a99a025 100644 --- a/test/traits.jl +++ b/test/traits.jl @@ -32,6 +32,7 @@ LearnAPI.learner(model::SmallLearner) = model small = SmallLearner() @test LearnAPI.constructor(small) == SmallLearner @test :(LearnAPI.learner) in LearnAPI.functions(small) +@test LearnAPI.kind_of(small) == LearnAPI.Descriminative() @test isempty(LearnAPI.kinds_of_proxy(small)) @test isempty(LearnAPI.tags(small)) @test !LearnAPI.is_pure_julia(small)