Skip to content

Commit 1a64074

Browse files
committed
update readme
1 parent 1526e17 commit 1a64074

File tree

1 file changed

+91
-172
lines changed

1 file changed

+91
-172
lines changed

README.md

Lines changed: 91 additions & 172 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,31 @@
55
<br>
66

77
<H2 align="center">
8-
A collection and visualization of single objective black-box functions for optimization benchmarking
8+
A collection of single-objective black-box functions for optimization benchmarking
99
</H2>
1010

11+
<br>
12+
13+
## Introduction
14+
15+
**Surfaces** is a Python library designed for researchers and practitioners in optimization and machine learning. It provides a comprehensive collection of benchmark functions to evaluate and compare optimization algorithms under standardized conditions.
16+
17+
### Key Features
18+
19+
- **Mathematical Test Functions**: 24 test functions covering 1D, 2D, and N-dimensional optimization landscapes
20+
- **Machine Learning Test Functions**: Real-world ML benchmarks using scikit-learn models with hyperparameter optimization
21+
- **Standardized Interface**: Consistent API across all functions for easy algorithm evaluation
22+
- **Flexible Configuration**: Support for different metrics (loss/score), dimensionalities, and evaluation modes
23+
- **Optimization Research**: Ideal for testing metaheuristics, gradient-free methods, and hyperparameter optimization algorithms
24+
25+
### Use Cases
26+
27+
- **Algorithm Development**: Test new optimization algorithms against established benchmarks
28+
- **Performance Comparison**: Compare different optimizers on standardized problem sets
29+
- **Research Publications**: Use well-known test functions with consistent implementations
30+
- **Educational Purposes**: Learn optimization concepts with visual and mathematical examples
31+
- **Hyperparameter Tuning**: Benchmark autoML and hyperparameter optimization methods
32+
1133

1234
<br>
1335

@@ -63,54 +85,6 @@
6385
</tr>
6486
</table>
6587

66-
### Machine Learning Functions
67-
68-
Surfaces also provides powerful visualizations for machine learning test functions, showing hyperparameter interactions and dataset-specific performance patterns.
69-
70-
<table style="width:100%">
71-
<tr>
72-
<th> <b>ML Function</b> </th>
73-
<th> <b>Hyperparameter Interactions</b> </th>
74-
<th> <b>Dataset Performance Analysis</b> </th>
75-
</tr>
76-
<tr>
77-
<th> <ins>K-Neighbors Classifier</ins> <br><br>
78-
<small>Shows how n_neighbors and algorithm<br>
79-
interact to affect classification<br>
80-
performance across different datasets</small>
81-
</th>
82-
<td>
83-
<img src="./doc/images/ml_functions/k_neighbors_classifier_n_neighbors_vs_algorithm_heatmap.jpg" width="90%">
84-
<br><small>Darker regions indicate better performance combinations</small>
85-
</td>
86-
<td>
87-
<img src="./doc/images/ml_functions/k_neighbors_classifier_dataset_vs_n_neighbors_analysis.jpg" width="100%">
88-
<br><small>Shows optimal n_neighbors varies by dataset</small>
89-
</td>
90-
</tr>
91-
<tr>
92-
<th> <ins>Gradient Boosting Regressor</ins> <br><br>
93-
<small>Demonstrates the interaction between<br>
94-
n_estimators and max_depth parameters<br>
95-
for regression tasks</small>
96-
</th>
97-
<td>
98-
<img src="./doc/images/ml_functions/gradient_boosting_regressor_n_estimators_vs_max_depth_heatmap.jpg" width="90%">
99-
<br><small>Reveals sweet spots in hyperparameter space</small>
100-
</td>
101-
<td>
102-
<img src="./doc/images/ml_functions/gradient_boosting_regressor_dataset_vs_max_depth_analysis.jpg" width="100%">
103-
<br><small>Shows max_depth sensitivity across datasets</small>
104-
</td>
105-
</tr>
106-
</table>
107-
108-
#### ML Visualization Features
109-
110-
- ** Hyperparameter Interactions**: Visualize how different hyperparameters interact and affect model performance
111-
- ** Dataset Sensitivity**: Understand how hyperparameters perform across different datasets
112-
- ** Optimal Regions**: Identify sweet spots in hyperparameter space
113-
- ** Performance Patterns**: Discover dataset-specific optimization strategies
11488

11589
<br>
11690

@@ -122,161 +96,106 @@ The most recent version of Surfaces is available on PyPi:
12296
pip install surfaces
12397
```
12498

125-
## Example
126-
```python
127-
import numpy as np
99+
## Public API
128100

129-
from surfaces.test_functions.mathematical import SphereFunction, AckleyFunction
130-
from surfaces.visualize import plotly_surface
101+
### Test Functions
131102

103+
Surfaces provides two main categories of test functions for optimization benchmarking:
132104

133-
sphere_function = SphereFunction(n_dim=2, metric="score")
134-
ackley_function = AckleyFunction(metric="loss")
105+
#### Mathematical Functions
135106

107+
Import from `surfaces.test_functions.mathematical`:
136108

137-
step_ = 0.05
138-
min_ = 10
139-
max_ = 10
140-
search_space = {
141-
"x0": np.arange(-min_, max_, step_),
142-
"x1": np.arange(-min_, max_, step_),
143-
}
109+
**1D Functions:**
110+
- `GramacyAndLeeFunction`
144111

145-
plotly_surface(sphere_function.objective_function, search_space).show()
146-
plotly_surface(ackley_function.objective_function, search_space).show()
147-
```
112+
**2D Functions:**
113+
- `AckleyFunction`, `BealeFunction`, `BoothFunction`, `BukinFunctionN6`
114+
- `CrossInTrayFunction`, `DropWaveFunction`, `EasomFunction`, `EggholderFunction`
115+
- `GoldsteinPriceFunction`, `HimmelblausFunction`, `HölderTableFunction`
116+
- `LangermannFunction`, `LeviFunctionN13`, `MatyasFunction`, `McCormickFunction`
117+
- `SchafferFunctionN2`, `SimionescuFunction`, `ThreeHumpCamelFunction`
148118

149-
### ML Function Visualization Example
119+
**N-Dimensional Functions:**
120+
- `GriewankFunction`, `RastriginFunction`, `RosenbrockFunction`
121+
- `SphereFunction`, `StyblinskiTangFunction`
150122

151-
```python
152-
from surfaces.test_functions.machine_learning.tabular.classification.test_functions import KNeighborsClassifierFunction
153-
from surfaces.visualize import plotly_ml_hyperparameter_heatmap, plotly_dataset_hyperparameter_analysis
123+
#### Machine Learning Functions
154124

155-
# Create ML function
156-
knn_func = KNeighborsClassifierFunction(metric="accuracy")
125+
Import from `surfaces.test_functions.machine_learning`:
157126

158-
# 1. Hyperparameter interaction analysis
159-
fig1 = plotly_ml_hyperparameter_heatmap(
160-
knn_func,
161-
'n_neighbors',
162-
'algorithm',
163-
title="KNN: Hyperparameter Interactions"
164-
)
165-
fig1.show()
166-
167-
# 2. Dataset sensitivity analysis
168-
fig2 = plotly_dataset_hyperparameter_analysis(
169-
knn_func,
170-
'n_neighbors',
171-
title="KNN: Performance Across Datasets"
172-
)
173-
fig2.show()
174-
```
127+
- `KNeighborsClassifierFunction` - K-nearest neighbors classification
128+
- `KNeighborsRegressorFunction` - K-nearest neighbors regression
129+
- `GradientBoostingRegressorFunction` - Gradient boosting regression
175130

176-
## Search Data Collection for Machine Learning Functions
131+
### Common Interface
177132

178-
Surfaces includes a powerful search data collection system for machine learning test functions that enables fast evaluation and benchmarking of optimization algorithms. This feature pre-computes and stores evaluation results across parameter grids, providing instant lookups instead of expensive ML model training.
133+
All test functions provide:
179134

180-
### Key Benefits
135+
- `objective_function(parameters)` - Evaluate the function
136+
- `search_space()` - Get parameter bounds/ranges
137+
- Constructor parameters:
138+
- `metric="loss"` or `"score"` - Optimization direction
139+
- `sleep=0` - Add artificial delays for benchmarking
140+
- `n_dim=N` (for N-dimensional functions) - Set dimensionality
181141

182-
- ** Fast evaluations**: 100-1000x speedup for ML function evaluations
183-
- ** Timing data included**: Original evaluation times preserved for realistic benchmarking
184-
- ** SQLite storage**: Lightweight, dependency-free database storage
185-
- ** Easy to use**: Simple API integrated with existing ML test functions
142+
## Usage Examples
186143

187-
### Quick Start
144+
### Basic Mathematical Function
188145

189146
```python
190-
from surfaces.test_functions.machine_learning.tabular.regression.test_functions import GradientBoostingRegressorFunction
191-
192-
# Create ML function
193-
ml_func = GradientBoostingRegressorFunction()
147+
from surfaces.test_functions.mathematical import SphereFunction, AckleyFunction
194148

195-
# Collect search data (one-time setup)
196-
ml_func.collect_search_data()
149+
# Create functions
150+
sphere_function = SphereFunction(n_dim=2, metric="score")
151+
ackley_function = AckleyFunction(metric="loss")
197152

198-
# Now use fast evaluation mode
199-
fast_func = GradientBoostingRegressorFunction(evaluate_from_data=True)
153+
# Get search space
154+
search_space = sphere_function.search_space()
155+
print(search_space) # {'x0': array([...]), 'x1': array([...])}
200156

201-
# Lightning-fast evaluation (uses stored data)
202-
params = {'n_estimators': 50, 'max_depth': 5, 'cv': 3, 'dataset': ml_func.dataset_default[0]}
203-
result = fast_func.evaluate(params) # ~1000x faster than normal evaluation
157+
# Evaluate function
158+
params = {'x0': 0.5, 'x1': -0.3}
159+
result = sphere_function.objective_function(params)
160+
print(f"Sphere function result: {result}")
204161
```
205162

206-
### Benchmarking with Timing Data
207-
208-
Perfect for testing optimization algorithms within time budgets:
163+
### Machine Learning Function
209164

210165
```python
211-
# Get timing statistics for realistic benchmarking
212-
timing_stats = ml_func.get_timing_statistics()
213-
print(f"Average evaluation time: {timing_stats['average_time']:.4f}s")
214-
print(f"Total time budget: {timing_stats['total_time']:.2f}s")
215-
216-
# Simulate algorithm testing within time constraints
217-
budget_seconds = 60.0
218-
evaluations_possible = int(budget_seconds / timing_stats['average_time'])
219-
print(f"Can test ~{evaluations_possible} parameter combinations in {budget_seconds}s")
220-
```
221-
222-
### Command-Line Usage
223-
224-
Use the included command-line utility for batch data collection:
225-
226-
```bash
227-
# Collect data for all ML functions
228-
python collect_ml_search_data.py --all
166+
from surfaces.test_functions.machine_learning import KNeighborsClassifierFunction
229167

230-
# Collect for specific function
231-
python collect_ml_search_data.py gradient_boosting_regressor
168+
# Create ML function
169+
knn_func = KNeighborsClassifierFunction(metric="accuracy")
232170

233-
# Check collection status
234-
python collect_ml_search_data.py --list
171+
# Get search space
172+
search_space = knn_func.search_space()
173+
print(search_space.keys()) # dict_keys(['n_neighbors', 'algorithm', 'cv', 'dataset'])
235174

236-
# Use custom batch size for memory efficiency
237-
python collect_ml_search_data.py --batch-size 50 --all
175+
# Evaluate function
176+
params = {
177+
'n_neighbors': 5,
178+
'algorithm': 'auto',
179+
'cv': 3,
180+
'dataset': knn_func.search_space()['dataset'][0]
181+
}
182+
accuracy = knn_func.objective_function(params)
183+
print(f"KNN accuracy: {accuracy}")
238184
```
239185

240-
### Advanced Usage
186+
### N-Dimensional Functions
241187

242-
**Custom Search Spaces:**
243188
```python
244-
# Define custom parameter ranges
245-
custom_space = {
246-
'n_estimators': [50, 100, 150],
247-
'max_depth': [3, 5, 7, 10],
248-
'cv': [3, 5],
249-
'dataset': ml_func.dataset_default[:2] # Use subset of datasets
250-
}
189+
from surfaces.test_functions.mathematical import SphereFunction
251190

252-
ml_func.collect_search_data(search_space=custom_space)
253-
```
254-
255-
**Collection Status and Management:**
256-
```python
257-
# Check collection status
258-
status = ml_func.get_search_data_status()
259-
print(f"Coverage: {status['completion_percentage']:.1f}%")
260-
print(f"Stored: {status['stored_evaluations']}/{status['total_combinations']}")
191+
# Create functions with different dimensionalities
192+
sphere_1d = SphereFunction(n_dim=1)
193+
sphere_3d = SphereFunction(n_dim=3)
194+
sphere_10d = SphereFunction(n_dim=10)
261195

262-
# Clear stored data if needed
263-
ml_func.clear_search_data()
196+
# Each has appropriate search space
197+
print(sphere_1d.search_space().keys()) # dict_keys(['x0'])
198+
print(sphere_3d.search_space().keys()) # dict_keys(['x0', 'x1', 'x2'])
199+
print(sphere_10d.search_space().keys()) # dict_keys(['x0', 'x1', ..., 'x9'])
264200
```
265201

266-
### Available ML Functions
267-
268-
- `GradientBoostingRegressorFunction` - Gradient boosting for regression
269-
- `KNeighborsRegressorFunction` - K-nearest neighbors regression
270-
- `KNeighborsClassifierFunction` - K-nearest neighbors classification
271-
272-
### How It Works
273-
274-
1. **Data Collection Phase**: ML function is evaluated across all parameter combinations in its search space
275-
2. **Storage**: Results stored in SQLite database with timing information
276-
3. **Fast Evaluation**: Lookups return pre-computed results instantly
277-
4. **Benchmarking**: Original timing data enables realistic algorithm testing
278-
279-
The system automatically handles:
280-
- Parameter grid generation
281-
- Database indexing for fast lookups
282-
- Function object serialization

0 commit comments

Comments
 (0)