Skip to content

Commit 311a3d5

Browse files
statmlbenstatmlben
committed
update readme
1 parent 5597494 commit 311a3d5

File tree

1 file changed

+175
-33
lines changed

1 file changed

+175
-33
lines changed

‎README.md‎

Lines changed: 175 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,114 @@
1-
# ReHLine <a href="https://github.com/softmin/ReHLine"><img src="doc/source/figs/logo.png" align="right" height="138" /></a>
1+
# ReHLine-Python: Efficient Solver for ERM with PLQ Loss and Linear Constraints <a href="https://github.com/softmin/ReHLine"><img src="doc/source/figs/logo.png" align="right" height="138" /></a>
22

3-
**ReHLine** is designed to be a computationally efficient and practically useful software package for large-scale empirical risk minimization (ERM) problems.
4-
5-
- Documentation: [https://rehline-python.readthedocs.io](https://rehline-python.readthedocs.io)
6-
- Project homepage: [https://rehline.github.io](https://rehline.github.io)
7-
- GitHub repo: [https://github.com/softmin/ReHLine-python](https://github.com/softmin/ReHLine-python)
8-
- PyPi: [https://pypi.org/project/rehline](https://pypi.org/project/rehline)
9-
- Paper: [NeurIPS | 2023](https://openreview.net/pdf?id=3pEBW2UPAD)
10-
<!-- - Open Source: [MIT license](https://opensource.org/licenses/MIT) -->
3+
[![PyPI version](https://badge.fury.io/py/rehline.svg)](https://badge.fury.io/py/rehline)
4+
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
5+
[![Documentation](https://img.shields.io/badge/docs-latest-blue.svg)](https://rehline-python.readthedocs.io)
6+
[![Paper](https://img.shields.io/badge/NeurIPS-2023-red.svg)](https://openreview.net/pdf?id=3pEBW2UPAD)
7+
[![Downloads](https://pepy.tech/badge/rehline)](https://pepy.tech/project/rehline)
118

12-
The **ReHLine** solver has four appealing
13-
"linear properties":
9+
> **Fast, scalable, and scikit-learn compatible optimization for machine learning**
1410
15-
- It applies to any convex piecewise linear-quadratic loss function, including the hinge loss, the check loss, the Huber loss, etc.
16-
- In addition, it supports linear equality and inequality constraints on the parameter vector.
17-
- The optimization algorithm has a provable linear convergence rate.
18-
- The per-iteration computational complexity is linear in the sample size.
11+
**ReHLine-Python** is the official Python implementation of ReHLine, a powerful solver for large-scale **empirical risk minimization (ERM) problems** with **convex piecewise linear-quadratic (PLQ) loss functions** and **linear constraints**. Built with high-performance C++ core and seamless Python integration, ReHLine delivers exceptional speed while maintaining ease of use.
1912

13+
See more details in the [ReHLine documentation](https://rehline-python.readthedocs.io).
2014

21-
## ✨ New Features: Scikit-Learn Compatible Estimators
15+
## ✨ Key Features
2216

23-
We are excited to introduce full scikit-learn compatibility! `ReHLine` now provides `plq_Ridge_Classifier` and `plq_Ridge_Regressor` estimators that integrate seamlessly with the entire scikit-learn ecosystem.
17+
- **🚀 Blazing Fast**: Linear computational complexity per iteration, scales to millions of samples
18+
- **🎯 Versatile**: Supports any convex PLQ loss (hinge, check, Huber, and more)
19+
- **🔒 Constrained Optimization**: Handle linear equality and inequality constraints
20+
- **📊 Scikit-Learn Compatible**: Drop-in replacement with `GridSearchCV`, `Pipeline` support
21+
- **🐍 Pythonic API**: Both low-level and high-level interfaces for flexibility
2422

25-
This means you can:
26-
- Drop `ReHLine` estimators directly into your existing scikit-learn `Pipeline`.
27-
- Perform robust hyperparameter tuning using `GridSearchCV`.
28-
- Use standard scikit-learn evaluation metrics and cross-validation tools.
23+
24+
## 📦 Installation
25+
26+
### Quick Install
27+
28+
```bash
29+
pip install rehline
30+
```
31+
32+
## 🚀 Quick Start
33+
34+
### Scikit-Learn Style API (Recommended)
35+
36+
ReHLine provides `plq_Ridge_Classifier` and `plq_Ridge_Regressor` that work seamlessly with scikit-learn:
37+
38+
```python
39+
from rehline import plq_Ridge_Classifier
40+
from sklearn.datasets import make_classification
41+
from sklearn.model_selection import train_test_split, GridSearchCV
42+
from sklearn.pipeline import Pipeline
43+
from sklearn.preprocessing import StandardScaler
44+
45+
# Generate dataset
46+
X, y = make_classification(n_samples=1000, n_features=20, random_state=42)
47+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
48+
49+
# Simple usage
50+
clf = plq_Ridge_Classifier(loss={'name': 'svm'}, C=1.0)
51+
clf.fit(X_train, y_train)
52+
print(f"Accuracy: {clf.score(X_test, y_test):.3f}")
53+
54+
# Use in Pipeline
55+
pipeline = Pipeline([
56+
('scaler', StandardScaler()),
57+
('classifier', plq_Ridge_Classifier(loss={'name': 'svm'}))
58+
])
59+
pipeline.fit(X_train, y_train)
60+
61+
# Hyperparameter tuning with GridSearchCV
62+
param_grid = {
63+
'C': [0.1, 1.0, 10.0],
64+
'loss': [{'name': 'svm'}, {'name': 'sSVM'}]
65+
}
66+
grid_search = GridSearchCV(plq_Ridge_Classifier(loss={"name": "svm"}), param_grid, cv=5)
67+
grid_search.fit(X_train, y_train)
68+
print(f"Best params: {grid_search.best_params_}")
69+
```
70+
71+
> See more details in [ReHLine with Scikit-Learn](https://rehline-python.readthedocs.io/en/latest/tutorials/ReHLine_sklearn.html).
72+
73+
### Low-Level API for Custom Problems
74+
75+
```python
76+
from rehline import ReHLine
77+
import numpy as np
78+
79+
# Define custom PLQ loss parameters
80+
clf = ReHLine()
81+
# Set custom U, V matrices for ReLU loss
82+
# and S, T, tau for ReHU loss
83+
## U
84+
clf.U = -(C*y).reshape(1,-1)
85+
## V
86+
clf.V = (C*np.array(np.ones(n))).reshape(1,-1)
87+
88+
# Set custom linear constraints A*beta + b >= 0
89+
X_sen = X[:,0]
90+
tol_sen = 0.1
91+
clf.A = np.repeat([X_sen @ X], repeats=[2], axis=0) / n
92+
clf.A[1] = -clf.A[1]
93+
94+
clf.fit(X)
95+
```
96+
97+
> See more detailed in [Manual ReHLine Formulation](https://rehline-python.readthedocs.io/en/latest/tutorials/ReHLine_manual.html).
98+
99+
100+
## 🎯 Use Cases
101+
102+
ReHLine excels at solving a wide range of machine learning problems:
103+
104+
| **Problem** | **Description** | **Key Benefits** |
105+
|------------|-----------------|------------------|
106+
| **Support Vector Machines** | Binary and multi-class classification | 100-400× faster than CVXPY solvers |
107+
| **Fair Machine Learning** | Classification with fairness constraints | Handles demographic parity efficiently |
108+
| **Quantile Regression** | Robust conditional quantile estimation | 2800× faster than general solvers |
109+
| **Huber Regression** | Outlier-resistant regression | Superior to specialized solvers |
110+
| **Sparse Learning** | Feature selection with L1 regularization | Scales to high dimensions |
111+
| **Custom Optimization** | Any PLQ loss with linear constraints | Flexible framework for research |
29112

30113
<!--
31114
## 📝 Formulation
@@ -52,17 +135,76 @@ This formulation has a wide range of applications spanning various fields, inclu
52135
53136
![](./figs/tab.png) -->
54137

55-
## ⌛ Benchmark (powered by benchopt)
138+
## ⚡ Performance Benchmarks
139+
140+
ReHLine delivers **exceptional speed** compared to state-of-the-art solvers. Here are speed-up factors on real-world datasets:
141+
142+
### Speed Comparison vs. Popular Solvers
143+
144+
| **Task** | **vs. ECOS** | **vs. MOSEK** | **vs. SCS** | **vs. Specialized Solvers** |
145+
|----------|--------------|---------------|-------------|----------------------------|
146+
| **SVM** | **415×** faster | **∞** (failed) | **340×** faster | **4.5×** vs. LIBLINEAR |
147+
| **Fair SVM** | **273×** faster | **100×** faster | **252×** faster | **∞** vs. DCCP (failed) |
148+
| **Quantile Regression** | **2843×** faster | **∞** (failed) | **∞** (failed) | — |
149+
| **Huber Regression** | **∞** (failed) | **452×** faster | **∞** (failed) | **2.4×** vs. hqreg |
150+
| **Smoothed SVM** | — | — | — | **1.6-2.3×** vs. SAGA/SAG/SDCA/SVRG |
151+
152+
> **Note**: "∞" indicates the competing solver failed to produce a valid solution or exceeded time limits. Results from [NeurIPS 2023 paper](https://openreview.net/pdf?id=3pEBW2UPAD).
153+
154+
### Reproducible Benchmarks (powered by benchopt)
155+
156+
All benchmarks are reproducible via [benchopt](https://github.com/benchopt/benchopt) at our [ReHLine-benchmark](https://github.com/softmin/ReHLine-benchmark) repository.
157+
158+
| **Problem** | **Benchmark Code** | **Interactive Results** |
159+
|------------|-------------------|------------------------|
160+
| SVM | [Code](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_SVM) | [📊 View](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_SVM.html) |
161+
| Smoothed SVM | [Code](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_sSVM) | [📊 View](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_sSVM.html) |
162+
| Fair SVM | [Code](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_FairSVM) | [📊 View](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_FairSVM.html) |
163+
| Quantile Regression | [Code](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_QR) | [📊 View](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_QR.html) |
164+
| Huber Regression | [Code](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_Huber) | [📊 View](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_Huber.html) |
165+
166+
## 🤝 Contributing
167+
168+
We welcome contributions! Whether it's bug reports, feature requests, or code contributions:
169+
170+
- 🐛 [Open an issue](https://github.com/softmin/ReHLine-python/issues)
171+
- 💬 [Start a discussion](https://github.com/softmin/ReHLine-python/discussions)
172+
- 🔀 Submit a pull request
173+
174+
## 📚 Citation
175+
176+
If you use ReHLine in your research, please cite our NeurIPS 2023 paper:
177+
178+
```bibtex
179+
@inproceedings{dai2023rehline,
180+
title={ReHLine: Regularized Composite ReLU-ReHU Loss Minimization with Linear Computation and Linear Convergence},
181+
author={Dai, Ben and Qiu, Yixuan},
182+
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
183+
year={2023}
184+
}
185+
```
186+
187+
## 🔗 ReHLine Ecosystem
188+
189+
<table>
190+
<tr>
191+
<td width="50%">
192+
193+
### 🏠 Core Projects
194+
- **[ReHLine](https://github.com/softmin/ReHLine)** - Main repository and documentation
195+
- **[ReHLine-python](https://github.com/softmin/ReHLine-python)** - Python interface (this repo)
196+
- **[ReHLine-r](https://github.com/softmin/ReHLine-r)** - R interface
197+
- **[ReHLine-cpp](https://github.com/softmin/ReHLine-cpp)** - High-performance C++ core
198+
199+
</td>
200+
<td width="50%">
56201

57-
Some existing problems of recent interest in statistics and machine
58-
learning can be solved by **ReHLine**, and we provide reproducible
59-
benchmark code and results at the
60-
[ReHLine-benchmark](https://github.com/softmin/ReHLine-benchmark) repository.
202+
### 📊 Resources
203+
- **[ReHLine-benchmark](https://github.com/softmin/ReHLine-benchmark)** - Reproducible benchmarks
204+
- **[Project Homepage](https://rehline.github.io)** - Official website
205+
- **[Documentation](https://rehline-python.readthedocs.io)** - Full Python docs
206+
- **[NeurIPS 2023 Paper](https://openreview.net/pdf?id=3pEBW2UPAD)** - Research paper
61207

62-
| Problem | Results |
63-
|---------- |:-----------------:|
64-
|[FairSVM](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_FairSVM) | [Result](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_FairSVM.html)|
65-
|[ElasticQR](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_QR) | [Result](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_QR.html)|
66-
|[RidgeHuber](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_Huber) | [Result](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_Huber.html)|
67-
|[SVM](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_SVM) | [Result](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_SVM.html)|
68-
|[Smoothed SVM](https://github.com/softmin/ReHLine-benchmark/tree/main/benchmark_sSVM) | [Result](https://rehline-python.readthedocs.io/en/latest/_static/benchmark/benchmark_sSVM.html)|
208+
</td>
209+
</tr>
210+
</table>

0 commit comments

Comments
 (0)