wording

Author: josephsdavid

src/MLJMultivariateStatsInterface.jl

@@ -463,7 +463,7 @@ Where
 
 # Operations
 
-- `transform(mach, Xnew)`: Return predictions of the target given new
+- `transform(mach, Xnew)`: Return lower dimensional projection of the target given new
   features `Xnew` having the same Scitype as `X` above.
 
 # Fitted parameters
@@ -509,9 +509,8 @@ PCA
 """
 $(MMI.doc_header(KernelPCA))
 
-`KernelPCA` Principal component analysis. Learns a linear transformation to
-project the data  on a lower dimensional space while preserving most of the initial
-variance.
+`KernelPCA` Kernel principal component analysis. Using a kernel, the linear
+operations of PCA are performed in a [reproducing Hilbert space](https://en.wikipedia.org/wiki/Reproducing_kernel_Hilbert_space).
 
 # Training data
 
@@ -587,9 +586,9 @@ KernelPCA
 """
 $(MMI.doc_header(ICA))
 
-`ICA` Principal component analysis. Learns a linear transformation to
-project the data  on a lower dimensional space while preserving most of the initial
-variance.
+`ICA` is a computational technique for separating a multivariate signal into
+additive subcomponents, with the assumption that the subcomponents are
+non-Gaussian and independent from each other.
 
 # Training data
 
@@ -603,56 +602,51 @@ Where
 
 # Hyper-parameters
 
-- `maxoutdim=0`: The maximum number of output dimensions. If not set, defaults to
-  0, where all components are kept (e.g., the number of components/output dimensions
-  is equal to the size of the smallest dimension of the training matrix).
-- `kernel::Function=(x,y)->x'y`: The kernel function, takes in 2 vector arguments
-   x and y, returns a scalar value. Defaults to the dot product of X and Y.
-- `solver::Symbol=:auto`: solver to use for the eigenvalues, one of `:eig`(default),
-  `:eigs`.
-- `inverse::Bool=true`: perform calculations needed for inverse transform
-- `beta::Real=1.0`: strength of the ridge regression that learns the inverse transform
-  when inverse is true.
-- `tol::Real=0.0`: Convergence tolerance for eigs solver.
-- `maxiter::Int=300`: maximum number of iterations for eigs solver.
+- `k::Int=0`: The number of independent components to recover, set automatically if `0`.
+- `alg::Symbol=:fastica`: The algorithm to use (only `:fastica` is supported at the moment).
+- `fun::Symbol=:tanh`: The approximate neg-entropy function, one of `:tanh`, `:gaus`.
+- `do_whiten::Bool=true`: Whether or not to perform pre-whitening.
+- `maxiter::Int=100`: The maximum number of iterations.
+- `tol::Real=1e-6`: The convergence tolerance for change in matrix W.
+- `mean::Union{Nothing, Real, Vector{Float64}}=nothing`: mean to use, if nothing (default)
+   centering is computed and applied, if zero, no centering, a vector of means can
+   be passed.
+- `winit::Union{Nothing,Matrix{<:Real}}=nothing`: Initial guess for matrix `W` either
+   an empty matrix (random initilization of `W`), a matrix of size `k × k` (if `do_whiten`
+   is true), a matrix of size `m × k` otherwise. If unspecified i.e `nothing` an empty
+   `Matrix{<:Real}` is used.
 
 # Operations
 
-- `transform(mach, Xnew)`: Return predictions of the target given new
+- `transform(mach, Xnew)`: Return lower dimensional projection of the target given new
   features `Xnew` having the same Scitype as `X` above.
 
 # Fitted parameters
 
 The fields of `fitted_params(mach)` are:
 
-- `projection`: Returns the projection matrix (of size `(d, p)`).
-  Each column of the projection matrix corresponds to a principal component.
-  The principal components are arranged in descending order of
-  the corresponding variances.
+ BUG: Does not have a projection class. It would also be cool to see the whitened
+matrix in fitted_params, to show how the covariance is the identity
 
 # Report
 
 The fields of `report(mach)` are:
 
 - `indim`: Dimensions of the provided data.
 - `outdim`: Dimensions of the transformed result.
-- `principalvars`: The variance of the principal components.
+- `mean`: The mean vector.
 
 # Examples
 
 ```
 using MLJ
 using LinearAlgebra
 
-KPCA = @load KernelPCA pkg=MultivariateStats
+ICA = @load ICA pkg=MultivariateStats
 
 X, y = @load_iris
 
-function rbf_kernel(length_scale)
-    return (x,y) -> norm(x-y)^2 / ((2 * length_scale)^2)
-end
-
-model = KPCA(maxoutdim=2, kernel = rbf_kernel(1))
+model = ICA(k = 2, tol=0.1)
 mach = machine(model, X) |> fit!
 
 projection = transform(mach, X)
@@ -662,6 +656,88 @@ See also
 TODO: ADD REFERENCES
 """
 ICA
+"""
+$(MMI.doc_header(LDA))
+
+`LDA`: Multiclass linear discriminant analysis. The algorithm learns a
+projection matrix `P` that projects a feature matrix `Xtrain` onto a lower dimensional
+space of dimension `out_dim` such that the trace of the transformed between-class
+scatter matrix(`Pᵀ*Sb*P`) is maximized relative to the trace of the transformed
+within-class scatter matrix (`Pᵀ*Sw*P`).The projection matrix is scaled such that
+`Pᵀ*Sw*P=I` or `Pᵀ*Σw*P=I`(where `Σw` is the within-class covariance matrix) .
+Predicted class posterior probability for feature matrix `Xtest` are derived by
+applying a softmax transformationto a matrix `Pr`, such that  rowᵢ of `Pr` contains
+computed distances(based on a distance metric) in the transformed space of rowᵢ in
+`Xtest` to the centroid of each class.
+
+# Training data
+
+In MLJ or MLJBase, bind an instance `model` to data with
+    mach = machine(model, X)
+
+Where
+
+- `X`: is any table of input features (eg, a `DataFrame`) whose columns
+  are of scitype `Continuous`; check the scitype with `schema(X)`
+
+# Hyper-parameters
+
+- `method::Symbol=:gevd`: The solver, one of `:gevd` or `:whiten` methods.
+- `cov_w::CovarianceEstimator`=SimpleCovariance: An estimator for the within-class
+    covariance (used in computing within-class scatter matrix, Sw), by default set
+    to the standard `MultivariateStats.SimpleCovariance()` but
+    could be set to any robust estimator from `CovarianceEstimation.jl`..
+- `cov_b::CovarianceEstimator`=SimpleCovariance: The same as `cov_w` but for the between-class
+    covariance (used in computing between-class scatter matrix, Sb).
+- `out_dim::Int=0`: The output dimension, i.e dimension of the transformed space,
+    automatically set if 0 is given (default).
+- `regcoef::Float64=1e-6`: The regularization coefficient (default value 1e-6). A positive
+    value `regcoef * eigmax(Sw)` where `Sw` is the within-class scatter matrix, is added
+    to the diagonal of Sw to improve numerical stability. This can be useful if using
+    the standard covariance estimator.
+- `dist::SemiMetric=SqEuclidean`: The distance metric to use when performing classification
+    (to compare the distance between a new point and centroids in the transformed space),
+    an alternative choice can be the `CosineDist`.Defaults to `SqEuclidean`.
+
+# Operations
+
+- `transform(mach, Xnew)`: Return lower dimensional projection of the target given new
+  features `Xnew` having the same Scitype as `X` above.
+
+# Fitted parameters
+
+The fields of `fitted_params(mach)` are:
+
+ BUG: Does not have a projection class. It would also be cool to see the whitened
+matrix in fitted_params, to show how the covariance is the identity
+
+# Report
+
+The fields of `report(mach)` are:
+
+- `indim`: Dimensions of the provided data.
+- `outdim`: Dimensions of the transformed result.
+- `mean`: The mean vector.
+
+# Examples
+
+```
+using MLJ
+using LinearAlgebra
+
+LA = @load LDA pkg=MultivariateStats
+
+X, y = @load_iris
+
+model = ICA(k = 2, tol=0.1)
+mach = machine(model, X) |> fit!
+
+projection = transform(mach, X)
+```
+
+See also
+TODO: ADD REFERENCES
+"""
 LDA
 BayesianLDA
 SubspaceLDA