add tutorials

Author: statmlben

doc/source/autoapi/rehline/index.rst

@@ -19,6 +19,14 @@ Overview
      - Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty.
 
 
+.. list-table:: Function
+   :header-rows: 0
+   :widths: auto
+   :class: summarytable
+
+   * - :py:obj:`ReHLine_solver <rehline.ReHLine_solver>`\ (X, U, V, Tau, S, T, A, b, max_iter, tol, shrink, verbose, trace_freq)
+     - \-
+
 
 
 Classes
@@ -101,7 +109,7 @@ Classes
    >>> U = -(C*y).reshape(1,-1)
    >>> L = U.shape[0]
    >>> V = (C*np.array(np.ones(n))).reshape(1,-1)
-   >>> clf = ReHLine(loss={'name': 'svm'}, C=C)
+   >>> clf = ReHLine(C=C)
    >>> clf.U, clf.V = U, V
    >>> clf.fit(X=X)
    >>> print('sol privided by rehline: %s' %clf.coef_)
@@ -320,6 +328,10 @@ Classes
 
 
 
+Functions
+---------
+.. py:function:: ReHLine_solver(X, U, V, Tau=np.empty(shape=(0, 0)), S=np.empty(shape=(0, 0)), T=np.empty(shape=(0, 0)), A=np.empty(shape=(0, 0)), b=np.empty(shape=0), max_iter=1000, tol=0.0001, shrink=1, verbose=1, trace_freq=100)
+
 
 
 

doc/source/example.rst

@@ -5,7 +5,7 @@ Example Gallery
 ---------------
 
 .. nblinkgallery::
-   :caption: A few links
+   :caption: Emprical Risk Minimization
    :name: rst-link-gallery
 
    examples/QR.ipynb

doc/source/index.rst

@@ -31,7 +31,7 @@
 
 - Homepage: `https://rehline.github.io/ <https://rehline.github.io/>`_
 - GitHub repo: `https://github.com/softmin/ReHLine-python <https://github.com/softmin/ReHLine-python>`_
-- Documentation: `https://rehline.readthedocs.io <https://rehline.readthedocs.io/en/latest/>`_
+- Documentation: `https://rehline-python.readthedocs.io <https://rehline-python.readthedocs.io/en/latest/>`_
 - PyPi: `https://pypi.org/project/rehline <https://pypi.org/project/rehline>`_
 - Paper: `NeurIPS | 2023 <https://openreview.net/pdf?id=3pEBW2UPAD>`_
 .. - Open Source: `MIT license <https://opensource.org/licenses/MIT>`_
@@ -76,5 +76,6 @@ If you use this code please star 🌟 the repository and cite the following pape
    :hidden:
 
    getting_started
+   tutorials
    example
    benchmark

doc/source/tutorials.rst

@@ -1,7 +1,7 @@
 Tutorials
-=========
+---------
 
-`ReHLine` is designed to address the empirical regularized ReLU-ReHU minimization problem, named *ReHLine optimization*, of the following form:
+`ReHLine` is designed to address the regularized ReLU-ReHU minimization problem, named *ReHLine optimization*, of the following form:
 
 .. math::
 
@@ -17,87 +17,43 @@ Some popular examples include SVMs with fairness constraints (FairSVM),
 elastic net regularized quantile regression (ElasticQR), 
 and ridge regularized Huber minimization (RidgeHuber).
 
-.. image:: ./figs/tab.png
+See `Manual ReHLine Formulation`_ documentation for more details and examples on converting your problem to ReHLine formulation.
 
-Solving PLQ ERMs
-----------------
 
-Loss
-****
+Moreover, the following specific classes of formulations can be directly solved by `ReHLine`.
 
-.. code:: python
-   
-   # name (str): name of the custom loss function
-   # loss_kwargs: more keys and values for loss parameters
-   loss = {'name': <loss_name>, <**loss_kwargs>}
+- **Empirical Risk Minimization** (ERM) with various loss functions, see `ReHLine: Empirical Risk Minimization`_.
+- **Matrix Factorization** (MF) with with various loss functions, see `ReHLine: Matrix Factorization`_.
 
-.. list-table::
-
- * - **SVM**
-   - | ``loss_name``: 'hinge' / 'svm' / 'SVM'
-     |
-     | *Example:* ``loss = {'name': 'SVM'}``
-
- * - **Quantile Reg**
-   - | ``loss_name``: 'check' / 'quantile' / 'quantile regression' / 'QR'
-     | ``qt`` (*float*): qt
-     |
-     | *Example:* ``loss = {'name': 'QR', 'qt': 0.25}``
-
- * - **Smooth SVM**
-   - | ``loss_name``: 'sSVM' / 'smooth SVM' / 'smooth hinge'
-     |
-     | *Example:* ``loss = {'name': 'sSVM'}``
-
- * - **Huber**
-   - | ``loss_name``: 'huber' / 'Huber'
-     |
-     | *Example:* ``loss = {'name': 'huber'}``
-
- * - **SVR**
-   - | ``loss_name``: 'SVR' / 'svr'
-     | ``epsilon`` (*float*): 0.1
-     |
-     | *Example:* ``loss = {'name': 'svr', 'epsilon': 0.1}``
-
-constraint
-**********
-
-.. code:: python
-   
-   # list of 
-   # name (str): name of the custom loss function
-   # loss_kwargs: more keys and values for loss parameters
-   constraint = [{'name': <loss_name>, <**loss_kwargs>}, ...]
+List of Tutorials
+=================
 
 .. list-table::
+ :align: left
+ :widths: 10 10 20
+ :header-rows: 1
+
+ * - tutorials
+   - | API
+   - | description
+ * - `Manual ReHLine Formulation <./tutorials/ReHLine_manual.rst>`_
+   - | `ReHLine <./autoapi/rehline/index.html#rehline.ReHLine>`_
+   - | ReHLine minimization with manual parameter settings.
+
+ * - `ReHLine: Empirical Risk Minimization <./tutorials/ReHLine_ERM.rst>`_
+   - | `plqERM_Ridge <./autoapi/rehline/index.html#rehline.plqERM_Ridge>`_
+   - | Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty.
+
+ * - `ReHLine: Matrix Factorization <./tutorials/ReHLine_MF.rst>`_
+   - | `plqMF_Ridge <./autoapi/rehline/index.html#rehline.plqERM_Ridge>`_
+   - | Matrix Factorization (MF) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty.
+
+.. toctree::
+   :maxdepth: 2
+   :hidden:
+
+   ./tutorials/ReHLine_manual
+   ./tutorials/ReHLine_ERM
+   ./tutorials/loss
+   ./tutorials/constraint
 
- * - **SVM**
-   - | ``loss_name``: 'hinge' / 'svm' / 'SVM'
-     |
-     | *Example:* ``loss = {'name': 'SVM'}``
-
- * - **Quantile Reg**
-   - | ``loss_name``: 'check' / 'quantile' / 'quantile regression' / 'QR'
-     | ``qt`` (*list*): [q1, q2, ... qK]
-     |
-     | *Example:* ``loss = {'name': 'QR', 'qt': [0.25, 0.75]}``
-
- * - **Smooth SVM**
-   - | ``loss_name``: 'sSVM' / 'smooth SVM' / 'smooth hinge'
-     |
-     | *Example:* ``loss = {'name': 'sSVM'}``
-
- * - **Huber**
-   - | ``loss_name``: 'huber' / 'Huber'
-     |
-     | *Example:* ``loss = {'name': 'huber'}``
-
- * - **SVR**
-   - | ``loss_name``: 'SVR' / 'svr'
-     | ``epsilon`` (*float*): 0.1
-     |
-     | *Example:* ``loss = {'name': 'svr', 'epsilon': 0.1}``
-
-manual ReHLine
---------------

doc/source/tutorials/ReHLine_ERM.rst

@@ -0,0 +1,26 @@
+ReHLine: Empirical Risk Minimization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The objective function is given by the following PLQ formulation, where :math:`\phi` is a convex piecewise linear function and :math:`\lambda` is a positive regularization parameter.
+
+.. math::
+
+    \min_{\pmb{\beta} \in \mathbb{R}^d} \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \pmb{\beta}) + \frac{1}{2} \| \pmb{\beta} \|_2^2, \ \text{ s.t. } \ 
+    \mathbf{A} \pmb{\beta} + \mathbf{b} \geq \mathbf{0},
+
+where :math:`\text{PLQ}(\cdot, \cdot)` is a convex piecewise linear quadratic function, see `Loss <./loss.rst>`_ for build-in loss functions, and :math:`\mathbf{A}` is a :math:`K \times d` matrix, and :math:`\mathbf{b}` is a :math:`K`-dimensional vector for linear constraints, see `Constraints <./constraint.rst>`_ for more details.
+
+For example, it supports the following loss functions and constraints.
+
+.. image:: ../figs/tab.png
+
+Example
+-------
+
+.. nblinkgallery::
+   :caption: Emprical Risk Minimization
+   :name: rst-link-gallery
+
+   ../examples/QR.ipynb
+   ../examples/SVM.ipynb
+   ../examples/FairSVM.ipynb

doc/source/tutorials/ReHLine_MF.rst

@@ -0,0 +1,2 @@
+ReHLine: Matrix Factorization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~