You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title = {EvoRBF: A Nature-inspired Algorithmic Framework for Evolving Radial Basis Function Networks},
51
+
month = may,
52
+
year = 2024,
53
+
publisher = {Zenodo},
54
+
doi = {10.5281/zenodo.11136007},
55
+
url = {https://doi.org/10.5281/zenodo.11136007}
56
+
}
57
+
58
+
@article{van2023mealpy,
59
+
title={MEALPY: An open-source library for latest meta-heuristic algorithms in Python},
60
+
author={Van Thieu, Nguyen and Mirjalili, Seyedali},
61
+
journal={Journal of Systems Architecture},
62
+
year={2023},
63
+
publisher={Elsevier},
64
+
doi={10.1016/j.sysarc.2023.102871}
65
+
}
66
+
```
40
67
41
68
42
69
# Theory
43
-
You can read several papers by using Google scholar search. Here we will walk through some basic concepts and parameters that matter to this network.
70
+
71
+
You can read several papers by using Google scholar search. There are many ways we can use Nature-inspired Algorithms
72
+
to optimize Radial Basis Function network, for example, you can read [this paper](https://doi.org/10.1016/B978-0-443-18764-3.00015-1).
73
+
Here we will walk through some basic concepts and parameters that matter to this network.
74
+
44
75
45
76
## Structure
46
77
@@ -50,56 +81,157 @@ You can read several papers by using Google scholar search. Here we will walk th
50
81
51
82
52
83
## Training algorithm
53
-
1. RBF needs to train the centers, and widths of Gaussian activation function. (This is 1st phase)
54
-
2. RBF usually use KMeans to find centers ==> Increase the complexity and time.
55
-
+ In that case, user need to define widths ==> Can use 1 single width or each hidden with different width.
56
-
+ Or RBF use random to find centers ==> Not good to split samples to different clusters.
57
-
3. RBF needs to train the output weights. (This is 2nd phase)
58
-
4. RBF do not use Gradient descent to calculate output weights, it used Moore–Penrose inverse (matrix multiplication, least square method) ==> so it is faster than MLP network.
59
-
5. Moore-Penrose inverse can find the exact solution ==> why you want to use Gradient or Metaheuristics here ==> Hell no.
60
-
6. In case of overfitting, what can we do with this network ==> We add Regularization method.
61
-
7. If you have large-scale dataset ==> Set more hidden nodes ==> Then increase the Regularization parameter.
62
84
85
+
### Traditional RBF models
63
86
64
-
So, we can see that what we need to do.
65
-
1. Use Moore-Penrose inverse to find the output weights (May be we let user use Regularization method, the parameter will be automatic tuned)
66
-
2. We need remove KMeans method, but we don't want to use Random method ==> We use metaheuristics here?
67
-
+ May be not a good idea, since metaheuristics can be more complex than KMeans.
68
-
+ But if we use metaheuristics to find everything (centers, width, )
87
+
In case of traditional RBF model. There are a few parameters need to identify to get the best model.
88
+
```code
89
+
1. The number of hidden nodes in hidden layer
90
+
2. The centers and widths (sigmas) of Gaussian function
91
+
3. The output weights
92
+
4. The regularization factor (lambda) L2
93
+
```
69
94
70
-
## Implementation version
95
+
To train their parameters,
96
+
```code
97
+
1. Using hyper-parameter tuning model such as GridSearchCV or RandomizedSearchCV to get the best hidden nodes
98
+
2. The centers can be calculated by Random or KMeans or unsupervised learning algorithms
99
+
3. The widths (sigmas) can be computed by hyper-parameter tuning process.
100
+
+ Width can be a single value that represent all hidden nodes has the same curve of Gaussian function
101
+
+ Width can be multiple values that each hidden node has a different value.
102
+
4. The output weights can be calculated by Moore-Penrose inverse (Matrix multiplication). Do not use Gradient Descent.
103
+
5. When setting regularization L2. lambda can be computed by hyper-parameter tuning process.
104
+
```
71
105
72
-
1. Rbf
106
+
Example,
107
+
```python
108
+
from evorbf import RbfRegressor, RbfClassifier
73
109
110
+
model = RbfClassifier(size_hidden=10, center_finder="kmeans", sigmas=2.0, reg_lambda=0.1, seed=None)
111
+
model = RbfRegressor(size_hidden=4, center_finder="random", sigmas=(1.5, 2, 2, 2.5), reg_lambda=0, seed=42)
74
112
113
+
model.fit(X=X_train, y=y_train)
114
+
y_pred = model.predict(X_test)
115
+
y_pred_prob = model.predict_proba(X_test)
116
+
```
75
117
76
118
77
-
#Citation Request
119
+
### Advanced RBF models
78
120
79
-
If you want to understand how Nature-inspired Algorithms is applied to Radial Basis Function Network, you
80
-
need to read the paper titled "Application of artificial intelligence in estimating mining capital expenditure using radial basis function neural network optimized by metaheuristic algorithms".
81
-
The paper can be accessed at the following [this link](https://doi.org/10.1016/B978-0-443-18764-3.00015-1)
121
+
In case of advanced RBF model. User can have so many different options.
122
+
```code
123
+
1. Choice different RBF kernel function such as Multiquadric (MQ), Inverse Multiquadric (IMQ), Thin Plate Spline (TPS), Exponential, Power,...
124
+
2. Choice different unsupervised learning algorithms to calculate the centers, and may be the number of hidden nodes.
125
+
+ For example, KMeans, or random algorithms, you need to set up the number of hidden nodes.
126
+
+ But, for MeanShift or DBSCAN algorithms, you don't need to set that value. They can automatically identify the number of cluters (number of hidden nodes).
127
+
3. This version may have the bias in output layer.
128
+
```
82
129
130
+
Examples,
131
+
```python
132
+
from evorbf import AdvancedRbfClassifier, AdvancedRbfRegressor
83
133
84
-
```bibtex
85
-
@software{thieu_2024_11136008,
86
-
author = {Nguyen Van Thieu},
87
-
title = {EvoRBF: A Nature-inspired Algorithmic Framework for Evolving Radial Basis Function Networks},
88
-
month = may,
89
-
year = 2024,
90
-
publisher = {Zenodo},
91
-
doi = {10.5281/zenodo.11136007},
92
-
url = {https://doi.org/10.5281/zenodo.11136007}
93
-
}
134
+
model = AdvancedRbfClassifier(center_finder="random", finder_params={"n_centers": 15},
1. RBF needs to train the centers, and widths of Gaussian activation function. (This is 1st phase)
216
+
2. RBF usually use KMeans to find centers ==> Increase the complexity and time.
217
+
+ In that case, user need to define widths ==> Can use 1 single width or each hidden with different width.
218
+
+ Or RBF use random to find centers ==> Not good to split samples to different clusters.
219
+
3. RBF needs to train the output weights. (This is 2nd phase)
220
+
4. RBF do not use Gradient descent to calculate output weights, it used Moore–Penrose inverse (matrix multiplication, least square method) ==> so it is faster than MLP network.
221
+
5. Moore-Penrose inverse can find the exact solution ==> why you want to use Gradient or Metaheuristics here ==> Hell no.
222
+
6. In case of overfitting, what can we do with this network ==> We add Regularization method.
223
+
7. If you have large-scale dataset ==> Set more hidden nodes ==> Then increase the Regularization parameter.
224
+
225
+
```code
226
+
1. RbfRegressor, RbfClassifier: You need to set up 4 types of hyper-parameters.
227
+
2. AdvancedRbfRegressor, AdvancedRbfClassifier: You need to set up 6 types of hyper-parameters.
228
+
But you have many option to choice, and you can design your own RBF models, a new one that nobody has used it before.
229
+
For example, RBF that has bias in output layer or RBF that use DBSCAN and Exponential kernel function.
230
+
3. NiaRbfRegressor, NiaRbfClassifier: You need to set up the hidden size. However, these are best classes in this library.
231
+
+ The sigmas are automatically calculated for each hidden nodes.
232
+
+ The reguarlization factor is also automatically tuned to fine the best one.
233
+
4. NiaRbfTuner. This class also extremely useful for traditional RBF models, it can tune hidden size, however,
234
+
there is only 1 sigma value will be presented all hidden nodes.
103
235
```
104
236
105
237
@@ -118,8 +250,10 @@ $ python
118
250
>>> evorbf.__version__
119
251
```
120
252
121
-
In this example below, we will use Whale Optimization Algorithm to optimize the `sigmas` (in non-linear Gaussian
122
-
kernel) and `weights` (of hidden-output layer) in RBF network (WOA-RBF model) for Diabetes prediction problem.
253
+
We have provided above several ways to import and call the proposed classes. If you need more details how to
254
+
use each of them, please check out the folder [examples](/examples). In this short demonstration, we will use
255
+
Whale Optimization Algorithm to optimize the `sigmas` (in non-linear Gaussian kernel) and `reg_lambda` of
256
+
L2 regularization in RBF network (WOA-RBF model) for Diabetes prediction problem.
0 commit comments