From fd672ca7eb070d327c6827a41c0df4386e4d3bed Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Fri, 11 Oct 2024 23:32:09 +1300 Subject: [PATCH 1/5] fix formatting mistake --- docs/src/common_implementation_patterns.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/src/common_implementation_patterns.md b/docs/src/common_implementation_patterns.md index 7f7641a4..528501b8 100644 --- a/docs/src/common_implementation_patterns.md +++ b/docs/src/common_implementation_patterns.md @@ -2,6 +2,8 @@ !!! warning +!!! warn + This section is only an implementation guide. The definitive specification of the Learn API is given in [Reference](@ref reference). From 0605baad809273d92c37b5e4a0db4fc2accfeb7e Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Fri, 11 Oct 2024 23:32:54 +1300 Subject: [PATCH 2/5] ditto --- docs/src/common_implementation_patterns.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/common_implementation_patterns.md b/docs/src/common_implementation_patterns.md index 528501b8..c554ca45 100644 --- a/docs/src/common_implementation_patterns.md +++ b/docs/src/common_implementation_patterns.md @@ -2,7 +2,7 @@ !!! warning -!!! warn +!!! warning This section is only an implementation guide. The definitive specification of the Learn API is given in [Reference](@ref reference). From ac5419b58d97c6009fe5acc8a9528cefcc131fe3 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Mon, 14 Oct 2024 11:18:43 +1300 Subject: [PATCH 3/5] remove slurping fallback signatures --- docs/src/anatomy_of_an_implementation.md | 39 +++++++++++++++----- docs/src/fit_update.md | 20 +++++++++-- docs/src/predict_transform.md | 7 ++-- src/fit_update.jl | 23 ++++++------ src/predict_transform.jl | 45 ++++++++++++------------ test/fit_update.jl | 14 -------- test/patterns/ensembling.jl | 7 ++++ test/patterns/gradient_descent.jl | 16 +++++++-- test/patterns/regression.jl | 20 +++++++++++ test/predict_transform.jl | 19 ---------- test/runtests.jl | 2 -- 11 files changed, 123 insertions(+), 89 deletions(-) delete mode 100644 test/fit_update.jl delete mode 100644 test/predict_transform.jl diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index a41325d4..28de88a3 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -1,20 +1,27 @@ # Anatomy of an Implementation -This section explains a detailed implementation of the LearnAPI for naive [ridge +This section explains a detailed implementation of the LearnAPI.jl for naive [ridge regression](https://en.wikipedia.org/wiki/Ridge_regression) with no intercept. The kind of workflow we want to enable has been previewed in [Sample workflow](@ref). Readers can also refer to the [demonstration](@ref workflow) of the implementation given later. -A transformer ordinarily implements `transform` instead of -`predict`. For more on `predict` versus `transform`, see [Predict or transform?](@ref) +The core LearnAPI.jl pattern looks like this: + +```julia +model = fit(algorithm, data) +predict(model, newdata) +``` + +A transformer ordinarily implements `transform` instead of `predict`. For more on +`predict` versus `transform`, see [Predict or transform?](@ref) !!! note New implementations of `fit`, `predict`, etc, - always have a *single* `data` argument, as in - `LearnAPI.fit(algorithm, data; verbosity=1) = ...`. - For convenience, user-calls, such as `fit(algorithm, X, y)`, automatically fallback - to `fit(algorithm, (X, y))`. + always have a *single* `data` argument as above. + For convenience, a signature such as `fit(algorithm, X, y)`, calling + `fit(algorithm, (X, y))`, can be added, but the LearnAPI.jl specification is + silent on the meaning or existence of signatures with extra arguments. !!! note @@ -52,7 +59,7 @@ nothing # hide Instances of `Ridge` will be [algorithms](@ref algorithms), in LearnAPI.jl parlance. -Associated with each new type of LearnAPI [algorithm](@ref algorithms) will be a keyword +Associated with each new type of LearnAPI.jl [algorithm](@ref algorithms) will be a keyword argument constructor, providing default values for all properties (struct fields) that are not other algorithms, and we must implement [`LearnAPI.constructor(algorithm)`](@ref), for recovering the constructor from an instance: @@ -244,6 +251,14 @@ in LearnAPI.functions(algorithm)`, for every instance `algorithm`. With [some exceptions](@ref trait_contract), the value of a trait should depend only on the *type* of the argument. +## Signatures added for convenience + +We add one `fit` signature for user-convenience only. The LearnAPI.jl specification has +nothing to say about `fit` signatures with more than two positional arguments. + +```@example anatomy +LearnAPI.fit(algorithm::Ridge, X, y; kwargs...) = fit(algorithm, (X, y); kwargs...) +``` ## [Demonstration](@id workflow) @@ -466,6 +481,14 @@ overload the trait, [`LearnAPI.data_interface(algorithm)`](@ref). See [Data interfaces](@ref data_interfaces) for details. +### Addition of signatures for user convenience + +As above, we add a signature which plays no role vis-à-vis LearnAPI.jl. + +```@exammple anatomy2 +LearnAPI.fit(algorithm::Ridge, X, y; kwargs...) = fit(algorithm, (X, y); kwargs...) +``` + ## Demonstration of an advanced `obs` workflow We now can train and predict using internal data representations, resampled using the diff --git a/docs/src/fit_update.md b/docs/src/fit_update.md index b49199c7..74ee1e0a 100644 --- a/docs/src/fit_update.md +++ b/docs/src/fit_update.md @@ -11,8 +11,6 @@ A "static" algorithm is one that does not generalize to new observations (e.g., clustering algorithms); there is no trainiing data and the algorithm is executed by `predict` or `transform` which receive the data. See example below. -When `fit` expects a tuple form of argument, `data = (X1, ..., Xn)`, then the signature -`fit(algorithm, X1, ..., Xn)` is also provided. ### Updating @@ -32,7 +30,7 @@ Supposing `Algorithm` is some supervised classifier type, with an iteration para ```julia algorithm = Algorithm(n=100) -model = fit(algorithm, (X, y)) # or `fit(algorithm, X, y)` +model = fit(algorithm, (X, y)) # Predict probability distributions: ŷ = predict(model, Distribution(), Xnew) @@ -76,6 +74,22 @@ labels = predict(algorithm, X) LearnAPI.extras(model) ``` +### Density estimation + +In density estimation, `fit` consumes no features, only a target variable; `predict`, +which consumes no data, returns the learned density: + +```julia +model = fit(algorithm, y) # no features +predict(model) # shortcut for `predict(model, Distribution())` +``` + +A one-liner will typically be implemented as well: + +```julia +predict(algorithm, y) +``` + ## Implementation guide ### Training diff --git a/docs/src/predict_transform.md b/docs/src/predict_transform.md index 1cf50f54..605ee27a 100644 --- a/docs/src/predict_transform.md +++ b/docs/src/predict_transform.md @@ -6,16 +6,15 @@ transform(model, data) inverse_transform(model, data) ``` -When a method expects a tuple form of argument, `data = (X1, ..., Xn)`, then a slurping -signature is also provided, as in `transform(model, X1, ..., Xn)`. - +Versions without the `data` argument may also appear, for example in [Density +estimation](@ref). ## [Typical worklows](@id predict_workflow) Train some supervised `algorithm`: ```julia -model = fit(algorithm, (X, y)) # or `fit(algorithm, X, y)` +model = fit(algorithm, (X, y)) ``` Predict probability distributions: diff --git a/src/fit_update.jl b/src/fit_update.jl index b6801359..75ae5d6e 100644 --- a/src/fit_update.jl +++ b/src/fit_update.jl @@ -14,13 +14,10 @@ The second signature is provided by algorithms that do not generalize to new obs ..., data)` carries out the actual algorithm execution, writing any byproducts of that operation to the mutable object `model` returned by `fit`. -Whenever `fit` expects a tuple form of argument, `data = (X1, ..., Xn)`, then the -signature `fit(algorithm, X1, ..., Xn)` is also provided. - -For example, a supervised classifier will typically admit this workflow: +For example, a supervised classifier might have a workflow like this: ```julia -model = fit(algorithm, (X, y)) # or `fit(algorithm, X, y)` +model = fit(algorithm, (X, y)) ŷ = predict(model, Xnew) ``` @@ -33,16 +30,16 @@ See also [`predict`](@ref), [`transform`](@ref), [`inverse_transform`](@ref), # New implementations -Implementation is compulsory. The signature must include `verbosity`. Fallbacks provide -the data slurping versions. A fallback for the first signature calls the second, ignoring -`data`: +Implementation of exactly one of the signatures is compulsory. If `fit(algorithm; +verbosity=1)` is implemented, then the trait [`LearnAPI.is_static`](@ref) must be +overloaded to return `true`. -```julia -fit(algorithm, data; kwargs...) = fit(algorithm; kwargs...) -``` +The signature must include `verbosity`. -If only the `fit(algorithm)` signature is expliclty implemented, then the trait -[`LearnAPI.is_static`](@ref) must be overloaded to return `true`. +The LearnAPI.jl specification has nothing to say regarding `fit` signatures with more than +two arguments. For convenience, for example, an algorithm is free to implement a slurping +signature, such as `fit(algorithm, X, y, extras...) = fit(algorithm, (X, y, extras...))` but +LearnAPI.jl does not guarantee such signatures are actually implemented. $(DOC_DATA_INTERFACE(:fit)) diff --git a/src/predict_transform.jl b/src/predict_transform.jl index 1fff01f3..16c6cabf 100644 --- a/src/predict_transform.jl +++ b/src/predict_transform.jl @@ -13,12 +13,20 @@ DOC_MUTATION(op) = """ +DOC_SLURPING(op) = + """ + + An algorithm is free to implement `$op` signatures with additional positional + arguments (eg., data-slurping signatures) but LearnAPI.jl is silent about their + interpretation or existence. + + """ DOC_MINIMIZE(func) = """ - If, additionally, [`LearnAPI.strip(model)`](@ref) is overloaded, then the following identity - must hold: + If, additionally, [`LearnAPI.strip(model)`](@ref) is overloaded, then the following + identity must hold: ```julia $func(LearnAPI.strip(model), args...) = $func(model, args...) @@ -63,7 +71,7 @@ which lists all supported target proxies. The argument `model` is anything returned by a call of the form `fit(algorithm, ...)`. If `LearnAPI.features(LearnAPI.algorithm(model)) == nothing`, then argument `data` is -omitted. An example is density estimators. +omitted in both signatures. An example is density estimators. # Example @@ -79,10 +87,7 @@ See also [`fit`](@ref), [`transform`](@ref), [`inverse_transform`](@ref). # Extended help -If `predict` supports data in the form of a tuple `data = (X1, ..., Xn)`, then a slurping -signature is also provided, as in `predict(model, X1, ..., Xn)`. - -Note `predict ` does not mutate any argument, except in the special case +Note `predict ` must not mutate any argument, except in the special case `LearnAPI.is_static(algorithm) == true`. # New implementations @@ -90,9 +95,12 @@ Note `predict ` does not mutate any argument, except in the special case If there is no notion of a "target" variable in the LearnAPI.jl sense, or you need an operation with an inverse, implement [`transform`](@ref) instead. -Implementation is optional. Only the first signature is implemented, but each -`kind_of_proxy` that gets an implementation must be added to the list returned by -[`LearnAPI.kinds_of_proxy`](@ref). +Implementation is optional. Only the first signature (with or without the `data` argument) +is implemented, but each `kind_of_proxy` that gets an implementation must be added to the +list returned by [`LearnAPI.kinds_of_proxy`](@ref). + +If `data` is not present in the implemented signature (eg., for density estimators) then +[`LearnAPI.features(algorithm, data)`](@ref) must return `nothing`. $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.predict)")) @@ -106,23 +114,12 @@ $(DOC_DATA_INTERFACE(:predict)) predict(model, data) = predict(model, kinds_of_proxy(algorithm(model)) |> first, data) predict(model) = predict(model, kinds_of_proxy(algorithm(model)) |> first) -# automatic slurping of multiple data arguments: -predict(model, k::KindOfProxy, data1, data2, datas...; kwargs...) = - predict(model, k, (data1, data2, datas...); kwargs...) -predict(model, data1, data2, datas...; kwargs...) = - predict(model, (data1, data2, datas...); kwargs...) - - - """ transform(model, data) Return a transformation of some `data`, using some `model`, as returned by [`fit`](@ref). -For `data` that consists of a tuple, a slurping version is also provided, i.e., you can do -`transform(model, X1, X2, X3)` in place of `transform(model, (X1, X2, X3))`. - # Example Below, `X` and `Xnew` are data of the same form. @@ -157,8 +154,10 @@ See also [`fit`](@ref), [`predict`](@ref), # New implementations -Implementation for new LearnAPI.jl algorithms is optional. A fallback provides the -slurping version. $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.transform)")) +Implementation for new LearnAPI.jl algorithms is +optional. $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.transform)")) + +$(DOC_SLURPING(:transform)) $(DOC_MINIMIZE(:transform)) diff --git a/test/fit_update.jl b/test/fit_update.jl deleted file mode 100644 index aa783432..00000000 --- a/test/fit_update.jl +++ /dev/null @@ -1,14 +0,0 @@ -using Test -using LearnAPI - -struct Gander end - -LearnAPI.update(::Gander, data) = sum(data) -LearnAPI.update_features(::Gander, data) = prod(data) - -@testset "update, update_features slurping" begin - @test update(Gander(), 2, 3, 4) == 9 - @test update_features(Gander(), 2, 3, 4) == 24 -end - -true diff --git a/test/patterns/ensembling.jl b/test/patterns/ensembling.jl index bcebacaa..8571818a 100644 --- a/test/patterns/ensembling.jl +++ b/test/patterns/ensembling.jl @@ -160,6 +160,13 @@ LearnAPI.strip(model::EnsembleFitted) = EnsembleFitted( ) ) +# convenience method: +LearnAPI.fit(algorithm::Ensemble, X, y, extras...; kwargs...) = + fit(algorithm, (X, y, extras...); kwargs...) +LearnAPI.update(algorithm::Ensemble, X, y, extras...; kwargs...) = + update(algorithm, (X, y, extras...); kwargs...) + + # synthetic test data: N = 10 # number of observations train = 1:6 diff --git a/test/patterns/gradient_descent.jl b/test/patterns/gradient_descent.jl index ff41782b..19f0d363 100644 --- a/test/patterns/gradient_descent.jl +++ b/test/patterns/gradient_descent.jl @@ -227,9 +227,9 @@ function LearnAPI.update_observations( ) # unpack data: - X = observations.X - y_hot = observations.y_hot - classes = observations.classes + X = observations_new.X + y_hot = observations_new.y_hot + classes = observations_new.classes nclasses = length(classes) classes == model.classes || error("New training target has incompatible classes.") @@ -328,6 +328,16 @@ LearnAPI.training_losses(model::PerceptronClassifierFitted) = model.losses ) +# ### Convenience methods + +LearnAPI.fit(algorithm::PerceptronClassifier, X, y; kwargs...) = + fit(algorithm, (X, y); kwargs...) +LearnAPI.update_observations(algorithm::PerceptronClassifier, X, y; kwargs...) = + update_observations(algorithm, (X, y); kwargs...) +LearnAPI.update(algorithm::PerceptronClassifier, X, y; kwargs...) = + update(algorithm, (X, y); kwargs...) + + # ## Tests # synthetic test data: diff --git a/test/patterns/regression.jl b/test/patterns/regression.jl index 4bcc9fe1..35376519 100644 --- a/test/patterns/regression.jl +++ b/test/patterns/regression.jl @@ -10,6 +10,9 @@ import DataFrames # We overload `obs` to expose internal representation of data. See later for a simpler # variation using the `obs` fallback. + +# ## Implementation + # no docstring here - that goes with the constructor struct Ridge lambda::Float64 @@ -117,6 +120,13 @@ LearnAPI.strip(model::RidgeFitted) = ) ) +# convenience method: +LearnAPI.fit(algorithm::Ridge, X, y; kwargs...) = + fit(algorithm, (X, y); kwargs...) + + +# ## Tests + # synthetic test data: n = 30 # number of observations train = 1:6 @@ -190,6 +200,9 @@ struct BabyRidge lambda::Float64 end + +# ## Implementation + """ BabyRidge(; lambda=0.1) @@ -250,6 +263,13 @@ LearnAPI.strip(model::BabyRidgeFitted) = ) ) +# convenience method: +LearnAPI.fit(algorithm::BabyRidge, X, y; kwargs...) = + fit(algorithm, (X, y); kwargs...) + + +# ## Tests + @testset "test a variation which does not overload LearnAPI.obs" begin algorithm = BabyRidge(lambda=0.5) diff --git a/test/predict_transform.jl b/test/predict_transform.jl deleted file mode 100644 index 7a496115..00000000 --- a/test/predict_transform.jl +++ /dev/null @@ -1,19 +0,0 @@ -using Test -using LearnAPI - -struct Goose end - -LearnAPI.fit(algorithm::Goose) = Ref(algorithm) -LearnAPI.algorithm(::Base.RefValue{Goose}) = Goose() -LearnAPI.predict(::Base.RefValue{Goose}, ::Point, data) = sum(data) -LearnAPI.transform(::Base.RefValue{Goose}, data) = prod(data) -@trait Goose kinds_of_proxy = (Point(),) - -@testset "predict and transform argument slurping" begin - model = fit(Goose()) - @test predict(model, Point(), 2, 3, 4) == 9 - @test predict(model, 2, 3, 4) == 9 - @test transform(model, 2, 3, 4) == 24 -end - -true diff --git a/test/runtests.jl b/test/runtests.jl index 9ef643f8..f6210235 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -4,9 +4,7 @@ test_files = [ "tools.jl", "traits.jl", "clone.jl", - "fit_update.jl", "accessor_functions.jl", - "predict_transform.jl", "patterns/regression.jl", "patterns/static_algorithms.jl", "patterns/ensembling.jl", From c71298e8bde2ab244c427f242751f77af0955e58 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Mon, 14 Oct 2024 11:40:49 +1300 Subject: [PATCH 4/5] fix some mistakes --- src/fit_update.jl | 14 +++++--------- src/predict_transform.jl | 4 ++-- test/patterns/ensembling.jl | 2 +- test/patterns/static_algorithms.jl | 1 - 4 files changed, 8 insertions(+), 13 deletions(-) diff --git a/src/fit_update.jl b/src/fit_update.jl index 75ae5d6e..de022826 100644 --- a/src/fit_update.jl +++ b/src/fit_update.jl @@ -44,10 +44,8 @@ LearnAPI.jl does not guarantee such signatures are actually implemented. $(DOC_DATA_INTERFACE(:fit)) """ -fit(algorithm, data; kwargs...) = - fit(algorithm; kwargs...) -fit(algorithm, data1, datas...; kwargs...) = - fit(algorithm, (data1, datas...); kwargs...) +function fit end + # # UPDATE AND COUSINS @@ -88,7 +86,7 @@ Implementation is optional. The signature must include See also [`LearnAPI.clone`](@ref) """ -update(model, data1, datas...; kwargs...) = update(model, (data1, datas...); kwargs...) +function update end """ update_observations(model, new_data; verbosity=1, parameter_replacements...) @@ -124,8 +122,7 @@ Implementation is optional. The signature must include See also [`LearnAPI.clone`](@ref). """ -update_observations(algorithm, data1, datas...; kwargs...) = - update_observations(algorithm, (data1, datas...); kwargs...) +function update_observations end """ update_features(model, new_data; verbosity=1, parameter_replacements...) @@ -151,5 +148,4 @@ Implementation is optional. The signature must include See also [`LearnAPI.clone`](@ref). """ -update_features(algorithm, data1, datas...; kwargs...) = - update_features(algorithm, (data1, datas...); kwargs...) +function update_features end diff --git a/src/predict_transform.jl b/src/predict_transform.jl index 16c6cabf..9e773192 100644 --- a/src/predict_transform.jl +++ b/src/predict_transform.jl @@ -166,8 +166,8 @@ $(DOC_MUTATION(:transform)) $(DOC_DATA_INTERFACE(:transform)) """ -transform(model, data1, data2, datas...; kwargs...) = - transform(model, (data1, data2, datas...); kwargs...) # automatic slurping +function transform end + """ inverse_transform(model, data) diff --git a/test/patterns/ensembling.jl b/test/patterns/ensembling.jl index 8571818a..ad348e4a 100644 --- a/test/patterns/ensembling.jl +++ b/test/patterns/ensembling.jl @@ -163,7 +163,7 @@ LearnAPI.strip(model::EnsembleFitted) = EnsembleFitted( # convenience method: LearnAPI.fit(algorithm::Ensemble, X, y, extras...; kwargs...) = fit(algorithm, (X, y, extras...); kwargs...) -LearnAPI.update(algorithm::Ensemble, X, y, extras...; kwargs...) = +LearnAPI.update(algorithm::EnsembleFitted, X, y, extras...; kwargs...) = update(algorithm, (X, y, extras...); kwargs...) diff --git a/test/patterns/static_algorithms.jl b/test/patterns/static_algorithms.jl index 21b43738..5a4c277f 100644 --- a/test/patterns/static_algorithms.jl +++ b/test/patterns/static_algorithms.jl @@ -56,7 +56,6 @@ end X = DataFrames.DataFrame(rand(3, 4), [:x, :y, :z, :w]) model = fit(algorithm) # no data arguments! # if provided, data is ignored: - @test fit(algorithm, "junk")[] == model[] @test LearnAPI.algorithm(model) == algorithm W = transform(model, X) @test W == DataFrames.DataFrame(Tables.matrix(X)[:,[1,4]], [:x, :w]) From 6242737794086e96d5f6d005d11fe0b9666f838d Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Mon, 14 Oct 2024 11:43:48 +1300 Subject: [PATCH 5/5] typo --- docs/src/anatomy_of_an_implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index 28de88a3..319e98ed 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -485,7 +485,7 @@ interfaces](@ref data_interfaces) for details. As above, we add a signature which plays no role vis-à-vis LearnAPI.jl. -```@exammple anatomy2 +```@example anatomy2 LearnAPI.fit(algorithm::Ridge, X, y; kwargs...) = fit(algorithm, (X, y); kwargs...) ```