Add parameter transformation module with new bijective transforms and Jacobian computation (#64)

Author: chaoming0625

changelog.md

@@ -1,6 +1,45 @@
 # Release Notes
 
 
+## Version 0.1.5
+
+### New Features
+
+#### Parameter Transformation Module (`braintools.param`)
+- **7 new bijective transforms** for constrained optimization and probabilistic modeling:
+  - **Positive**: Constrains parameters to (0, +∞) using exponential transformation
+  - **Negative**: Constrains parameters to (-∞, 0) using negative softplus
+  - **ScaledSigmoid**: Sigmoid with adjustable sharpness/temperature parameter (beta)
+  - **Power**: Box-Cox family power transformation for variance stabilization
+  - **Ordered**: Ensures monotonically increasing output vectors (useful for cutpoints in ordinal regression)
+  - **Simplex**: Stick-breaking transformation for probability vectors summing to 1
+  - **UnitVector**: Projects vectors onto the unit sphere (L2 norm = 1)
+- **Jacobian computation**: Added `log_abs_det_jacobian()` method to Transform base class and implementations for probabilistic modeling
+  - Implemented for: Identity, Sigmoid, Softplus, Log, Exp, Affine, Chain, Positive
+
+#### Surrogate Gradient Enhancements (`braintools.surrogate`)
+
+- Gradient computation of hyperparameters of surrogate gradient functions.
+- Fix batching issue in surrogate gradient functions
+
+
+### Improvements
+
+#### API Enhancements
+- **`__repr__` methods**: Added string representations to all Transform classes and Param class for better debugging
+- **Enhanced documentation**: Updated `docs/apis/param.rst` with comprehensive API reference
+  - Organized sections: Base Classes, Parameter Wrapper, Bounded Transforms, Positive/Negative Transforms, Advanced Transforms, Composition Transforms
+  - Descriptive explanations for each transform's use case
+
+#### Code Quality
+- **Comprehensive test coverage**: Added 28 new tests for param module (45 total tests passing)
+  - Tests for all new transforms: roundtrip, constraints, repr methods
+  - Tests for `log_abs_det_jacobian` correctness
+  - Tests for edge cases and numerical stability
+
+
+
+
 ## Version 0.1.4
 
 ### New Features

docs/apis/braintools.rst

@@ -50,30 +50,3 @@ combining encoder outputs.
     spike_bitwise_ixor
     spike_bitwise
 
-
-
-Parameter Transformations
--------------------------
-
-Smooth and invertible transforms used to keep optimization parameters within
-valid domains during training.
-
-.. autosummary::
-    :toctree: generated/
-    :nosignatures:
-    :template: classtemplate.rst
-
-    Transform
-    IdentityTransform
-    SigmoidTransform
-    SoftplusTransform
-    NegSoftplusTransform
-    LogTransform
-    ExpTransform
-    TanhTransform
-    SoftsignTransform
-    AffineTransform
-    ChainTransform
-    MaskedTransform
-    CustomTransform
-    ClippedTransform

docs/apis/param.rst

@@ -0,0 +1,138 @@
+``braintools.param`` module
+===========================
+
+.. currentmodule:: braintools.param
+.. automodule:: braintools.param
+
+Parameter transformation module for constrained optimization and probabilistic modeling.
+Provides bijective transforms that map between unconstrained and constrained parameter spaces.
+
+Overview
+--------
+
+The ``braintools.param`` module provides:
+
+- **Bijective transformations** for mapping parameters between constrained and unconstrained domains
+- **Parameter wrapper** (``Param``) that automatically applies transforms during optimization
+- **Jacobian computation** via ``log_abs_det_jacobian`` for probabilistic modeling
+- **Composition utilities** for building complex transforms from simple ones
+
+Base Classes
+------------
+
+These classes provide the foundational architecture for all transforms in the module.
+The ``Transform`` class defines the common interface with ``forward``, ``inverse``,
+and ``log_abs_det_jacobian`` methods.
+
+.. autosummary::
+   :toctree: generated/
+   :nosignatures:
+   :template: classtemplate.rst
+
+   Transform
+   Identity
+
+Parameter Wrapper
+-----------------
+
+The ``Param`` class wraps parameter values and automatically applies bijective
+transformations. It stores values in unconstrained space internally while exposing
+constrained values via the ``.data`` property.
+
+.. autosummary::
+   :toctree: generated/
+   :nosignatures:
+   :template: classtemplate.rst
+
+   Param
+
+Bounded Transforms
+------------------
+
+These transforms map unbounded values to bounded intervals. They are useful for
+parameters that must lie within specific ranges, such as probabilities, correlation
+coefficients, or physical quantities with known bounds.
+
+.. autosummary::
+   :toctree: generated/
+   :nosignatures:
+   :template: classtemplate.rst
+
+   Sigmoid
+   ScaledSigmoid
+   Tanh
+   Softsign
+   Clip
+
+- **Sigmoid**: Maps ℝ → [lower, upper] using the logistic sigmoid function
+- **ScaledSigmoid**: Sigmoid with adjustable sharpness/temperature parameter
+- **Tanh**: Maps ℝ → (lower, upper) using hyperbolic tangent
+- **Softsign**: Maps ℝ → (lower, upper) using softsign function
+- **Clip**: Hard clipping to bounds (non-bijective)
+
+Positive/Negative Transforms
+----------------------------
+
+These transforms constrain parameters to be strictly positive or negative.
+They are commonly used for variance parameters, rate constants, and other
+quantities that must maintain a specific sign.
+
+.. autosummary::
+   :toctree: generated/
+   :nosignatures:
+   :template: classtemplate.rst
+
+   Softplus
+   NegSoftplus
+   Log
+   Exp
+   Positive
+   Negative
+
+- **Softplus**: Maps ℝ → [lower, ∞) using log(1 + exp(x))
+- **NegSoftplus**: Maps ℝ → (-∞, upper] using negative softplus
+- **Log/Exp**: Maps ℝ → (lower, ∞) using exponential transformation
+- **Positive**: Convenience class for (0, ∞) constraint
+- **Negative**: Convenience class for (-∞, 0) constraint
+
+Advanced Transforms
+-------------------
+
+These transforms implement sophisticated constraints for specialized applications
+in probabilistic modeling, ordinal regression, and directional statistics.
+
+.. autosummary::
+   :toctree: generated/
+   :nosignatures:
+   :template: classtemplate.rst
+
+   Power
+   Ordered
+   Simplex
+   UnitVector
+
+- **Power**: Box-Cox family power transformation for variance stabilization
+- **Ordered**: Ensures monotonically increasing output vectors (useful for cutpoints)
+- **Simplex**: Stick-breaking transformation for probability vectors summing to 1
+- **UnitVector**: Projects vectors onto the unit sphere (L2 norm = 1)
+
+Composition Transforms
+----------------------
+
+These transforms allow building complex transformations from simpler ones.
+They support chaining, masking, and custom user-defined transformations.
+
+.. autosummary::
+   :toctree: generated/
+   :nosignatures:
+   :template: classtemplate.rst
+
+   Affine
+   Chain
+   Masked
+   Custom
+
+- **Affine**: Linear transformation y = ax + b
+- **Chain**: Composes multiple transforms sequentially
+- **Masked**: Applies transform selectively based on boolean mask
+- **Custom**: User-defined transformation with custom forward/inverse functions