Skip to content

Commit 1c87d4a

Browse files
wilsonrljrwilsonrljr
authored
Merge pull request #194 from wilsonrljr/v0.6.0
V0.6.0
2 parents 3873d2a + 2b1d5a4 commit 1c87d4a

37 files changed

+390
-483
lines changed

‎CHANGELOG.md‎

Lines changed: 73 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,77 @@
11
# Changes in SysIdentPy
22

3+
## v0.6.0
4+
5+
### CONTRIBUTORS
6+
7+
- wilsonrljr
8+
- oliveira-mark
9+
10+
### CHANGES
11+
12+
This release introduces significant improvements focused on improving code organization, readability, and compliance with PEP8 standards. It also includes a new foundational class for Error Reduction Ratio (ERR) based algorithms, expanded testing, and the discontinuation of Python 3.7 support.
13+
14+
15+
- **New Features:**
16+
- Introduced the `OFRBase` class, encapsulating common methods essential for ERR-based algorithms.
17+
- Refactored the `FROLS` class to inherit from `OFRBase`, focusing it solely on the ERR algorithm.
18+
- Implemented the Ultra Orthogonal Forward Regression (UOFR) algorithm, also inheriting from `OFRBase`.
19+
20+
- **API Changes:**
21+
- **BREAKING CHANGE**: Fixed a typo in Bernstein Basis Function. Previously it was defined as Bersntein.
22+
- **Refactoring and Modularization:**
23+
- Extracted the `InformationMatrix` class from `narmax_base` into a new module: `utils.information_matrix`.
24+
- Moved specific methods to newly created modules: `utils.lags` and `utils.simulation`, promoting better separation of concerns.
25+
- Add deprecation message for RidgeRegression `solver` argument.
26+
27+
- **Code Quality Improvements:**
28+
- Renamed variables and methods for better readability and PEP8 compliance, including changing uppercase variable names to lowercase.
29+
- Updated imports to use new utility modules, reducing redundancy and improving maintainability.
30+
- Removed unused imports and redundant parentheses, streamlining the codebase.
31+
- Change python version in deploy action.
32+
33+
- **Testing Enhancements:**
34+
- Added comprehensive tests for basis functions: Bernstein, Bilinear, Hermite (normalized), Laguerre, and Legendre.
35+
- Introduced tests for utility methods, including `narmax_tools`, `save_load`, and the new simulation utilities.
36+
- Increased test coverage to **92%**, ensuring robustness and reliability.
37+
38+
- **Validation and Error Handling:**
39+
- Implemented a validation check for `train_percentage`, raising an error for values exceeding 100%.
40+
- Adapted methods and tests following the removal of the `InformationMatrix` class to ensure consistency across the codebase.
41+
42+
- **Documentation Updates:**
43+
- Launched a redesigned frontend page featuring a modern UI and improved responsiveness.
44+
- Restructured multiple sections for better organization and clarity.
45+
- Overhauled key guides, including `quick_start`, `developer_guide`, and `user_guide`.
46+
- Added new examples, including the Lorenz System and Chaotic Map.
47+
- Enhanced grammar and readability across documentation.
48+
- Updated dependencies related to `mkdocs` for better performance and compatibility.
49+
- Improved Google Analytics integration.
50+
- Fixed broken links for the Nubank and Estatidados blogs, and refined link formatting in the book.
51+
- Updated class docstrings to align with the latest changes.
52+
- Standardized docstrings and method signatures to use lowercase variable names, following PEP8 guidelines.
53+
- Revised contribution examples to reflect the latest `sysidentpy` version.
54+
- Integrated book examples into traditional documentation, with direct links to the book section.
55+
- Adjusted structure, titles, and links across various docs and examples for better navigation.
56+
- Removed dataset files; datasets are now hosted in a dedicated repository, `sysidentpy-data`.
57+
- Acknowledged JetBrains support and collaboration in the README and sponsor page.
58+
- Fix edit uri when clicking to edit doc page in doc website.
59+
- Now every example loads the data from `sysidentpy-data` repository.
60+
61+
- **Python Version Support Update:**
62+
- **Support for Python 3.7 has been discontinued.** This aligns with the official end of support for Python 3.7 and resolves compatibility issues with newer dependencies.
63+
- Certain parameter estimation algorithms, such as Bounded Variable Least Squares, require newer versions of SciPy that no longer support Python 3.7.
64+
- Users can still run SysIdentPy on Python 3.7, but some features, including some parameter estimation functionalities, will be unavailable.
65+
66+
### IMPACT
67+
68+
These changes improve the modularity, readability, and maintainability of the codebase. The introduction of the `OFRBase` class simplifies the implementation of ERR-based algorithms, facilitating future extensions. Comprehensive testing ensures the reliability of both new and existing functionalities.
69+
70+
### TESTING
71+
72+
All new and existing tests were executed, achieving **92% test coverage**.
73+
74+
375
## v0.5.3
476

577
### CONTRIBUTORS
@@ -8,7 +80,7 @@
880

981
### CHANGES
1082

11-
IMPORTANT! This update addresses a bug related to the Bilinear basis function for models with more the 2 inputs. This release keep providing crucial groundwork for the future development of SysIdentPy, making easier to add new features and improve the code, setting the stage for a robust and feature-complete 1.0.0 release in the feature.
83+
IMPORTANT! This update addresses a bug related to the Bilinear basis function for models with more the 2 inputs. This release keep providing crucial groundwork for the future development of SysIdentPy, making easier to add new features and improve the code, setting the stage for a robust and feature-complete 1.0.0 release in the future.
1284

1385
- **API Changes:**
1486
- Fix Bilinear basis function issue for models with more than 2 inputs. This fix the `get_max_xlag` method in `basis_function_base` and also fix how `combination_xlag` is created.

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
<h1 align="center"> NARMAX Methods For System Identification and TimeSeries Forecasting </h1>
1717
<h3 align="center">From Classical Approaches to Neural Networks</h3>
1818

19-
**SysIdentPy** offers State of the Art techniques to build your NARMAX models, including its variants `NARX`, `NARMA`, `NAR`, `NFIR`, `ARMAX`, `ARX`, `ARMA` and others. It also includes tons of interesting examples to help you build nonlinear forecasting models using SysIdentPy.
19+
**SysIdentPy** offers State-of-the-Art techniques to build your NARMAX models, including its variants `NARX`, `NARMA`, `NAR`, `NFIR`, `ARMAX`, `ARX`, `ARMA` and others. It also includes tons of interesting examples to help you build nonlinear forecasting models using SysIdentPy.
2020

2121
</div>
2222

@@ -65,11 +65,11 @@ pip install sysidentpy
6565

6666
`SysIdentPy` requires:
6767

68-
- Python (>= 3.7)
68+
- Python (>= 3.8) (3.7 works for everything except for 3 parameter estimation methods)
6969
- NumPy (>= 1.9.2) for numerical algorithms
7070
- Matplotlib >= 3.3.2 for static plotting and visualizations
7171
- Pytorch (>=1.7.1) for building NARX neural networks
72-
- scipy (>= 1.7.0) for numerical and optimization algorithms
72+
- scipy (>= 1.8.0) for numerical and optimization algorithms
7373

7474
The library is compatible with Linux, Windows, and macOS. Some examples may also require additional packages like pandas.
7575

‎docs/changelog/changelog.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,77 @@ template: overrides/main.html
55

66
# Changes in SysIdentPy
77

8+
## v0.6.0
9+
10+
### CONTRIBUTORS
11+
12+
- wilsonrljr
13+
- oliveira-mark
14+
15+
### CHANGES
16+
17+
This release introduces significant improvements focused on improving code organization, readability, and compliance with PEP8 standards. It also includes a new foundational class for Error Reduction Ratio (ERR) based algorithms, expanded testing, and the discontinuation of Python 3.7 support.
18+
19+
20+
- **New Features:**
21+
- Introduced the `OFRBase` class, encapsulating common methods essential for ERR-based algorithms.
22+
- Refactored the `FROLS` class to inherit from `OFRBase`, focusing it solely on the ERR algorithm.
23+
- Implemented the Ultra Orthogonal Forward Regression (UOFR) algorithm, also inheriting from `OFRBase`.
24+
25+
- **API Changes:**
26+
- **BREAKING CHANGE**: Fixed a typo in Bernstein Basis Function. Previously it was defined as Bersntein.
27+
- **Refactoring and Modularization:**
28+
- Extracted the `InformationMatrix` class from `narmax_base` into a new module: `utils.information_matrix`.
29+
- Moved specific methods to newly created modules: `utils.lags` and `utils.simulation`, promoting better separation of concerns.
30+
- Add deprecation message for RidgeRegression `solver` argument.
31+
32+
- **Code Quality Improvements:**
33+
- Renamed variables and methods for better readability and PEP8 compliance, including changing uppercase variable names to lowercase.
34+
- Updated imports to use new utility modules, reducing redundancy and improving maintainability.
35+
- Removed unused imports and redundant parentheses, streamlining the codebase.
36+
- Change python version in deploy action.
37+
38+
- **Testing Enhancements:**
39+
- Added comprehensive tests for basis functions: Bernstein, Bilinear, Hermite (normalized), Laguerre, and Legendre.
40+
- Introduced tests for utility methods, including `narmax_tools`, `save_load`, and the new simulation utilities.
41+
- Increased test coverage to **92%**, ensuring robustness and reliability.
42+
43+
- **Validation and Error Handling:**
44+
- Implemented a validation check for `train_percentage`, raising an error for values exceeding 100%.
45+
- Adapted methods and tests following the removal of the `InformationMatrix` class to ensure consistency across the codebase.
46+
47+
- **Documentation Updates:**
48+
- Launched a redesigned frontend page featuring a modern UI and improved responsiveness.
49+
- Restructured multiple sections for better organization and clarity.
50+
- Overhauled key guides, including `quick_start`, `developer_guide`, and `user_guide`.
51+
- Added new examples, including the Lorenz System and Chaotic Map.
52+
- Enhanced grammar and readability across documentation.
53+
- Updated dependencies related to `mkdocs` for better performance and compatibility.
54+
- Improved Google Analytics integration.
55+
- Fixed broken links for the Nubank and Estatidados blogs, and refined link formatting in the book.
56+
- Updated class docstrings to align with the latest changes.
57+
- Standardized docstrings and method signatures to use lowercase variable names, following PEP8 guidelines.
58+
- Revised contribution examples to reflect the latest `sysidentpy` version.
59+
- Integrated book examples into traditional documentation, with direct links to the book section.
60+
- Adjusted structure, titles, and links across various docs and examples for better navigation.
61+
- Removed dataset files; datasets are now hosted in a dedicated repository, `sysidentpy-data`.
62+
- Acknowledged JetBrains support and collaboration in the README and sponsor page.
63+
- Fix edit uri when clicking to edit doc page in doc website.
64+
- Now every example loads the data from `sysidentpy-data` repository.
65+
66+
- **Python Version Support Update:**
67+
- **Support for Python 3.7 has been discontinued.** This aligns with the official end of support for Python 3.7 and resolves compatibility issues with newer dependencies.
68+
- Certain parameter estimation algorithms, such as Bounded Variable Least Squares, require newer versions of SciPy that no longer support Python 3.7.
69+
- Users can still run SysIdentPy on Python 3.7, but some features, including some parameter estimation functionalities, will be unavailable.
70+
71+
### IMPACT
72+
73+
These changes improve the modularity, readability, and maintainability of the codebase. The introduction of the `OFRBase` class simplifies the implementation of ERR-based algorithms, facilitating future extensions. Comprehensive testing ensures the reliability of both new and existing functionalities.
74+
75+
### TESTING
76+
77+
All new and existing tests were executed, achieving **92% test coverage**.
78+
879
## v0.5.3
980

1081
### CONTRIBUTORS

‎docs/user-guide/tutorials/PV-forecasting-benchmark.ipynb‎

Lines changed: 28 additions & 98 deletions
Large diffs are not rendered by default.

‎docs/user-guide/tutorials/basis-function-overview.ipynb‎

Lines changed: 13 additions & 13 deletions
Large diffs are not rendered by default.

‎docs/user-guide/tutorials/electromechanical-system-identification-entropic-regression.ipynb‎

Lines changed: 10 additions & 8 deletions
Large diffs are not rendered by default.

‎docs/user-guide/tutorials/f-16-aircraft-n-steps-ahead-prediction.ipynb‎

Lines changed: 12 additions & 12 deletions
Large diffs are not rendered by default.

‎docs/user-guide/tutorials/f-16-aircraft.ipynb‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,12 +72,14 @@
7272
},
7373
{
7474
"cell_type": "code",
75-
"execution_count": 2,
75+
"execution_count": null,
7676
"metadata": {},
7777
"outputs": [],
7878
"source": [
7979
"f_16 = pd.read_csv(\n",
80-
" r\"../examples/datasets/f-16.txt\", header=None, names=[\"x1\", \"x2\", \"y\"]\n",
80+
" r\"https://raw.githubusercontent.com/wilsonrljr/sysidentpy-data/refs/heads/main/datasets/f_16_vibration_test/f-16.txt\",\n",
81+
" header=None,\n",
82+
" names=[\"x1\", \"x2\", \"y\"],\n",
8183
")"
8284
]
8385
},

‎docs/user-guide/tutorials/load-forecasting-benchmark.ipynb‎

Lines changed: 28 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,13 @@
5757
},
5858
{
5959
"cell_type": "code",
60-
"execution_count": 2,
61-
"metadata": {},
60+
"execution_count": null,
61+
"metadata": {
62+
"ExecuteTime": {
63+
"end_time": "2025-03-27T22:52:48.732533Z",
64+
"start_time": "2025-03-27T22:52:47.228193Z"
65+
}
66+
},
6267
"outputs": [],
6368
"source": [
6469
"from warnings import simplefilter\n",
@@ -81,9 +86,7 @@
8186
"\n",
8287
"%matplotlib inline\n",
8388
"\n",
84-
"loss = mean_squared_error\n",
85-
"\n",
86-
"data_location = r\".\\datasets\""
89+
"loss = mean_squared_error"
8790
]
8891
},
8992
{
@@ -95,14 +98,14 @@
9598
},
9699
{
97100
"cell_type": "code",
98-
"execution_count": 3,
101+
"execution_count": null,
99102
"metadata": {},
100103
"outputs": [
101104
{
102105
"name": "stderr",
103106
"output_type": "stream",
104107
"text": [
105-
"C:\\Users\\wilso\\Desktop\\projects\\GitHub\\sysidentpy\\sysidentpy\\model_structure_selection\\forward_regression_orthogonal_least_squares.py:618: UserWarning: n_info_values is greater than the maximum number of all regressors space considering the chosen y_lag, u_lag, and non_degree. We set as 5\n",
108+
"c:\\Users\\wilso\\Desktop\\projects\\GitHub\\sysidentpy\\sysidentpy\\model_structure_selection\\ofr_base.py:537: UserWarning: n_info_values is greater than the maximum number of all regressors space considering the chosen y_lag, u_lag, and non_degree. We set as 5\n",
106109
" self.info_values = self.information_criterion(reg_matrix, y)\n"
107110
]
108111
},
@@ -125,8 +128,9 @@
125128
}
126129
],
127130
"source": [
128-
"files = [\"\\SanFrancisco_Hospital.csv\"]\n",
129-
"raw = pd.read_csv(data_location + files[0])\n",
131+
"raw = pd.read_csv(\n",
132+
" \"https://raw.githubusercontent.com/wilsonrljr/sysidentpy-data/refs/heads/main/datasets/san_francisco_hospital/SanFrancisco_Hospital.csv\"\n",
133+
")\n",
130134
"df = pd.DataFrame()\n",
131135
"df[\"ds\"] = pd.date_range(\"1/1/2015 1:00:00\", freq=str(60) + \"Min\", periods=8760)\n",
132136
"df[\"y\"] = raw.iloc[:, 0].values\n",
@@ -193,8 +197,9 @@
193197
}
194198
],
195199
"source": [
196-
"files = [\"\\SanFrancisco_Hospital.csv\"]\n",
197-
"raw = pd.read_csv(data_location + files[0])\n",
200+
"raw = pd.read_csv(\n",
201+
" \"https://raw.githubusercontent.com/wilsonrljr/sysidentpy-data/refs/heads/main/datasets/san_francisco_hospital/SanFrancisco_Hospital.csv\"\n",
202+
")\n",
198203
"df = pd.DataFrame()\n",
199204
"df[\"ds\"] = pd.date_range(\"1/1/2015 1:00:00\", freq=str(60) + \"Min\", periods=8760)\n",
200205
"df[\"y\"] = raw.iloc[:, 0].values\n",
@@ -265,8 +270,9 @@
265270
],
266271
"source": [
267272
"set_random_seed(42)\n",
268-
"files = [\"\\SanFrancisco_Hospital.csv\"]\n",
269-
"raw = pd.read_csv(data_location + files[0])\n",
273+
"raw = pd.read_csv(\n",
274+
" \"https://raw.githubusercontent.com/wilsonrljr/sysidentpy-data/refs/heads/main/datasets/san_francisco_hospital/SanFrancisco_Hospital.csv\"\n",
275+
")\n",
270276
"df = pd.DataFrame()\n",
271277
"df[\"ds\"] = pd.date_range(\"1/1/2015 1:00:00\", freq=str(60) + \"Min\", periods=8760)\n",
272278
"df[\"y\"] = raw.iloc[:, 0].values\n",
@@ -315,60 +321,14 @@
315321
"WARNING: nprophet - fit: Parts of code may break if using other than daily data.\n"
316322
]
317323
},
318-
{
319-
"name": "stdout",
320-
"output_type": "stream",
321-
"text": [
322-
"04-26 19:54:06 - WARNING - Parts of code may break if using other than daily data.\n"
323-
]
324-
},
325324
{
326325
"name": "stderr",
327326
"output_type": "stream",
328327
"text": [
329-
"INFO: nprophet.utils - set_auto_seasonalities: Disabling yearly seasonality. Run NeuralProphet with yearly_seasonality=True to override this.\n"
330-
]
331-
},
332-
{
333-
"name": "stdout",
334-
"output_type": "stream",
335-
"text": [
336-
"04-26 19:54:06 - INFO - Disabling yearly seasonality. Run NeuralProphet with yearly_seasonality=True to override this.\n"
337-
]
338-
},
339-
{
340-
"name": "stderr",
341-
"output_type": "stream",
342-
"text": [
343-
"INFO: nprophet.config - set_auto_batch_epoch: Auto-set batch_size to 32\n"
344-
]
345-
},
346-
{
347-
"name": "stdout",
348-
"output_type": "stream",
349-
"text": [
350-
"04-26 19:54:06 - INFO - Auto-set batch_size to 32\n"
351-
]
352-
},
353-
{
354-
"name": "stderr",
355-
"output_type": "stream",
356-
"text": [
357-
"INFO: nprophet.config - set_auto_batch_epoch: Auto-set epochs to 7\n"
358-
]
359-
},
360-
{
361-
"name": "stdout",
362-
"output_type": "stream",
363-
"text": [
364-
"04-26 19:54:06 - INFO - Auto-set epochs to 7\n"
365-
]
366-
},
367-
{
368-
"name": "stderr",
369-
"output_type": "stream",
370-
"text": [
371-
"Epoch[7/7]: 100%|██████████| 7/7 [00:03<00:00, 1.82it/s, SmoothL1Loss=0.0102, MAE=81.6, RegLoss=0.011] \n",
328+
"INFO: nprophet.utils - set_auto_seasonalities: Disabling yearly seasonality. Run NeuralProphet with yearly_seasonality=True to override this.\n",
329+
"INFO: nprophet.config - set_auto_batch_epoch: Auto-set batch_size to 32\n",
330+
"INFO: nprophet.config - set_auto_batch_epoch: Auto-set epochs to 7\n",
331+
"Epoch[7/7]: 100%|██████████| 7/7 [00:04<00:00, 1.74it/s, SmoothL1Loss=0.0102, MAE=81.6, RegLoss=0.011] \n",
372332
"INFO: nprophet - _evaluate: Validation metrics: SmoothL1Loss MAE\n",
373333
"1 0.011 84.733\n"
374334
]
@@ -377,18 +337,16 @@
377337
"name": "stdout",
378338
"output_type": "stream",
379339
"text": [
380-
"04-26 19:54:10 - INFO - Validation metrics: SmoothL1Loss MAE\n",
381-
"1 0.011 84.733\n",
382340
"11397.103026422525\n"
383341
]
384342
}
385343
],
386344
"source": [
387345
"set_random_seed(42)\n",
388346
"\n",
389-
"# set_log_level(\"ERROR\")\n",
390-
"files = [\"\\SanFrancisco_Hospital.csv\"]\n",
391-
"raw = pd.read_csv(data_location + files[0])\n",
347+
"raw = pd.read_csv(\n",
348+
" \"https://raw.githubusercontent.com/wilsonrljr/sysidentpy-data/refs/heads/main/datasets/san_francisco_hospital/SanFrancisco_Hospital.csv\"\n",
349+
")\n",
392350
"df = pd.DataFrame()\n",
393351
"df[\"ds\"] = pd.date_range(\"1/1/2015 1:00:00\", freq=str(60) + \"Min\", periods=8760)\n",
394352
"df[\"y\"] = raw.iloc[:, 0].values\n",
@@ -403,7 +361,6 @@
403361
"\n",
404362
"future = m.make_future_dataframe(df_val, n_historic_predictions=True)\n",
405363
"forecast = m.predict(future)\n",
406-
"# fig = m.plot(forecast)\n",
407364
"print(loss(forecast[\"y\"][24:-1], forecast[\"yhat1\"][24:-1]))\n",
408365
"\n",
409366
"neuralprophet_loss = loss(forecast[\"y\"][24:-1], forecast[\"yhat1\"][24:-1])"
@@ -473,11 +430,9 @@
473430
}
474431
],
475432
"metadata": {
476-
"interpreter": {
477-
"hash": "d637594906b92e2064a8978fcd7d47bb213aa9d5bb48152798bc86e8ea48bb1d"
478-
},
479433
"kernelspec": {
480-
"display_name": "Python 3.8.11 64-bit ('neural_prophet': conda)",
434+
"display_name": "neural_prophet",
435+
"language": "python",
481436
"name": "python3"
482437
},
483438
"language_info": {

0 commit comments

Comments
 (0)