Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 18 additions & 18 deletions docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -146,41 +146,41 @@ functionality of the modules.
- Default Value
- Description
* - mode
- ``univariate``
- TODO
- ``univariate`` (options: ``multivariate``, ``ensemble``)
- Mode of operation for the data inspector.
* - ensemble.model
- ``WeightEnsemble``
- TODO
- Model to use when inspector mode is ``ensemble``.
* - ensemble.module
- ``streamad.process``
- TODO
- Python module for the ensemble model.
* - ensemble.model_args
-
- TODO
- Additional Arguments for the ensemble model.
* - models.model
- ``ZScoreDetector``
- TODO
- Model to use for data inspection
* - models.module
- ``streamad.model``
- TODO
- Base python module for inspection models
* - models.model_args
-
- TODO
- Additional arguments for the model
* - models.model_args.is_global
- ``false``
- TODO
-
* - anomaly_threshold
- ``0.01``
- TODO
- Threshold for classifying an observation as an anomaly.
* - score_threshold
- ``0.5``
- TODO
- Threshold for the anomaly score.
* - time_type
- ``ms``
- TODO
- Unit of time used in time range calculations.
* - time_range
- ``20``
- TODO
- Time window for data inspection

``pipeline.data_analysis``
^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand All @@ -193,17 +193,17 @@ functionality of the modules.
- Default Value
- Description
* - model
- ``rf``
- TODO
- ``rf`` option: ``XGBoost``
- Model to use for the detector
* - checksum
- Not given here
- TODO
- Checksum for the model file to ensure integrity
* - base_url
- https://heibox.uni-heidelberg.de/d/0d5cbcbe16cd46a58021/
- TODO
- Base URL for downloading the model if not present locally
* - threshold
- ``0.5``
- TODO
- Threshold for the detector's classification.

Environment Configuration
.........................
Expand Down
94 changes: 92 additions & 2 deletions docs/training.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,95 @@ For hyperparameter optimisation we use ``optuna``.
It offers GPU support to get the best parameters.


TODO: add configuration parameters for the training, test and explanation process
------------------------------------------------------------------------------------
Training Parameters
-------------------

.. list-table::
:header-rows: 1

* - Parameter
- Type
- Default
- Description
* - ``--dataset``
- ``[combine|dgarchive|cic|dgta]``
- ``combine``
- Data set to train model, choose between all available datasets.
* - ``--dataset_path``
- ``Path``
-
- Dataset path, follow folder structure.
* - ``--dataset_max_rows``
- ``int``
- ``-1``
- Maximum rows to load from each dataset.
* - ``--model``
- ``[xg|rf|gbm]``
-
- Model to train, choose between XGBoost, RandomForest, or GBM.
* - ``--model_output_path``
- ``Path``
- ``./results/model``
- Path to store model. Output is ``{MODEL}_{SHA256}.pickle``.

Testing Parameters
------------------

.. list-table::
:header-rows: 1

* - Parameter
- Type
- Default
- Description
* - ``--dataset``
- ``[combine|dgarchive|cic|dgta]``
- ``combine``
- Data set to test model.
* - ``--dataset_path``
- ``Path``
-
- Dataset path, follow folder structure.
* - ``--dataset_max_rows``
- ``int``
- ``-1``
- Maximum rows to load from each dataset.
* - ``--model``
- ``[xg|rf|gbm]``
-
- Model architecture to test.
* - ``--model_path``
- ``Path``
-
- Path to trained model.

Explanation Parameters
----------------------

.. list-table::
:header-rows: 1

* - Parameter
- Type
- Default
- Description
* - ``--dataset``
- ``[combine|dgarchive|cic|dgta]``
- ``combine``
- Data set to explain model predictions.
* - ``--dataset_path``
- ``Path``
-
- Dataset path, follow folder structure.
* - ``--dataset_max_rows``
- ``int``
- ``-1``
- Maximum rows to load from each dataset.
* - ``--model``
- ``[xg|rf|gbm]``
-
- Model architecture to explain.
* - ``--model_path``
- ``Path``
-
- Path to trained model.
14 changes: 14 additions & 0 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,20 @@ Now, you can start each module, e.g. the `Inspector`:
(.venv) $ python src/inspector/main.py


Commit Hook
------------

Contributing to the project you might be noting failed pipeline runs.
This can be due to the pre.commit hook finding errors in the formatting. Therefore, we suggest you run

.. code-block:: console

(.venv) pre-commit run --show-diff-on-failure --color=always --all-files

before committing your changes to GitHub.
This reformates the code accordingly, preventing errors in the pipeline.


Configuration
-------------

Expand Down
Loading