Release of version 1.2.0, 1.2.1 and SPPCheck 1.0.0

Release of SPPCheck version 1.0.0
Note that this release does not include code documentation yet.
It must still be transferred from the masters thesis into code documentation later on.

## [1.2.1] - 2022-08-21

### Added

* Release of SPPCheck v1.0.0
  * Its changes will be logged within this file
  * SPPCheck function and internal structure
  * Added Grafana Dashboard for SPPCheck

### Changed

* Upgraded CodeQL to v3
* Extended `SppUtils.mk_logger_file` method by an logger_dir argument to allow different log files for SPPMon and SPPCheck
  * Added "sppcheckLogs" to the gitignore file.
* Moved all cSpell settings from the workspace file to the settings file
* Removed/Moved total count of exceptions during execution
* Moved verbose setting of sub-modules into init method to avoid issues and clarify program structure
* Error-Messages now have the prefix "ERROR: "
* Added InfluxClient function "get_list_rp" to query all retention policies, moving it out of existing functionality

### Fixed

* Code scanning alert #2 and #3: Printing passwords into logger due to an faulty if-expression.
* CheckPID file now uses the class-verbose setting and no longer ARGS-Verbose, making it independent of actual args.
* Fixed PID file checks under windows
* Fixed PID file entries not being deleted.
* Removed unnecessary and confusing check when creating a SelectionQuery

### Known Issues

* SPPCheck lacks a lot of documentation, which is to be delivered on a later date.

## [1.2.0] - 2022-06-17

### Added

* Adds FullLogs and LoadedSystem information to the Grafana SPPMon Runtime Duration panel.
* Adds clarification that all timestamps are changed into second precision on insert.
* Adds support for batch insert to insert into a different retention policy
* Selection queries now also support an alternative retention policy to query from
* CreateRP-Method in the influxClient to allow creating non-lasting retention policies
* Adds pandas-stubs, openpyxl and pyxlsb to requirements file
* Prints total count of errors during the execution if there are any - instead of "script finished" output
* Added type spelling dictionary to the settings file
* Added linting settings to the settings file
* Added two new tables to the definitions.py file which are only executed if SPPCheck is executed.

### Changed

* Moved pid files-functions and other functions from SPPMon to helper functions.
  * Adjusted log messages to make their message generic if required.
  * Added arguments to replace self-access.
* Changed default log and pid-file locations from `home/sppmonLogs/FILE` to `spectrum-protect-sppmon/sppmonLogs/FILE`
* Added init declarations in the SPPMon `__init__` function to allow overview of all self-vars
* Changed SelectionQueries to only take a single table instead of an List of such, as it isn't required and complicates the code
* Predefined retention policies are no longer protected in the definitions module.
* Sending a select query now raises a error when it fails.
* Allows SelectionQueries from table to be another inner-selection query
  * Changed all calls of the constructor
  * Made sure to only allow inner queries when using the selection-keyword.
* Changed the severity of unknown type-annotation due to submodules not being typed from error to warning.
* Minor printing edits when generifing Dashboard
* Removes deprecated functions

### Fixed

* Fixed typo in exception_utils.py file, renaming it and all references on it.
* Lots of typos in the inline documentation
* SelectionQuery and associated methods: Introduces Optional annotation to fix linter error

### Known Issues

* In newer Grafana versions the dashboard import might corrupt the datasource name. A fix is unavailable yet, though the error is only visual.
* Some typos in the table definitions (commited / uncommited) cannot be fixed. This would not be backward-compatible and break the database.

Author: NielsKorschinsky

.gitignore

@@ -7,6 +7,11 @@ config_files/*
 Grafana/customer_dashboards/*
 !Grafana/customer_dashboards/.gitkeep
 
+# ### Log files ###
+
+sppmonLogs/*
+sppcheckLogs/*
+
 # `python` and `scripts are used as workspace dirs
 # ignore if vs code is opened up on top level
 .vs/*

CHANGELOG.md

@@ -5,20 +5,106 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
+## SPPMon [Unreleased] - yyyy-mm-dd
 
-## [Unreleased] - yyyy-mm-dd
+### Added
+
+### Changed
+
+### Fixed
+
+### Known Issues
+
+## SPPCheck [Unreleased] - yyyy-mm-dd
+
+### Added
+
+### Changed
+
+### Fixed
+
+### Known Issues
+
+## [1.2.1] - 2022-08-21
+
+### Added
+
+* Release of SPPCheck v1.0.0
+  * Its changes will be logged within this file
+  * SPPCheck function and internal structure
+  * Added Grafana Dashboard for SPPCheck
+
+### Changed
+
+* Upgraded CodeQL to v3
+* Extended `SppUtils.mk_logger_file` method by an logger_dir argument to allow different log files for SPPMon and SPPCheck
+  * Added "sppcheckLogs" to the gitignore file.
+* Moved all cSpell settings from the workspace file to the settings file
+* Removed/Moved total count of exceptions during execution
+* Moved verbose setting of sub-modules into init method to avoid issues and clarify program structure
+* Error-Messages now have the prefix "ERROR: "
+* Added InfluxClient function "get_list_rp" to query all retention policies, moving it out of existing functionality
+
+### Fixed
+
+* Code scanning alert #2 and #3: Printing passwords into logger due to an faulty if-expression.
+* CheckPID file now uses the class-verbose setting and no longer ARGS-Verbose, making it independent of actual args.
+* Fixed PID file checks under windows
+* Fixed PID file entries not being deleted.
+* Removed unnecessary and confusing check when creating a SelectionQuery
+
+### Known Issues
+
+* SPPCheck lacks a lot of documentation, which is to be delivered on a later date.
+
+## [1.2.0] - 2022-06-17
 
 ### Added
 
+* Adds FullLogs and LoadedSystem information to the Grafana SPPMon Runtime Duration panel.
+* Adds clarification that all timestamps are changed into second precision on insert.
+* Adds support for batch insert to insert into a different retention policy
+* Selection queries now also support an alternative retention policy to query from
+* CreateRP-Method in the influxClient to allow creating non-lasting retention policies
+* Adds pandas-stubs, openpyxl and pyxlsb to requirements file
+* Prints total count of errors during the execution if there are any - instead of "script finished" output
+* Added type spelling dictionary to the settings file
+* Added linting settings to the settings file
+* Added two new tables to the definitions.py file which are only executed if SPPCheck is executed.
+
 ### Changed
 
+* Moved pid files-functions and other functions from SPPMon to helper functions.
+  * Adjusted log messages to make their message generic if required.
+  * Added arguments to replace self-access.
+* Changed default log and pid-file locations from `home/sppmonLogs/FILE` to `spectrum-protect-sppmon/sppmonLogs/FILE`
+* Added init declarations in the SPPMon `__init__` function to allow overview of all self-vars
+* Changed SelectionQueries to only take a single table instead of an List of such, as it isn't required and complicates the code
+* Predefined retention policies are no longer protected in the definitions module.
+* Sending a select query now raises a error when it fails.
+* Allows SelectionQueries from table to be another inner-selection query
+  * Changed all calls of the constructor
+  * Made sure to only allow inner queries when using the selection-keyword.
+* Changed the severity of unknown type-annotation due to submodules not being typed from error to warning.
+* Minor printing edits when generifing Dashboard
+* Removes deprecated functions
+
 ### Fixed
 
+* Fixed typo in exception_utils.py file, renaming it and all references on it.
+* Lots of typos in the inline documentation
+* SelectionQuery and associated methods: Introduces Optional annotation to fix linter error
+
+### Known Issues
+
+* In newer Grafana versions the dashboard import might corrupt the datasource name. A fix is unavailable yet, though the error is only visual.
+* Some typos in the table definitions (commited / uncommited) cannot be fixed. This would not be backward-compatible and break the database.
+
 ## [1.1.1] - 2022-02-22
 
 ### Added
 
-* Adds FullLogs and LoadedSystem information to the Grafana SPPMon Runtime Duration panel. 
+* Adds FullLogs and LoadedSystem information to the Grafana SPPMon Runtime Duration panel.
 * Adds license information into each SPPMon code file.
 * Specifies the encoding and reading permission when opening config files.
 
@@ -121,7 +207,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 * Corrupted config-file path no longer breaks SPPMon but prints an error message
 * Partly fixes #40, but some IDs still remain missing due to other issues. The fix is described within the issue.
 * JobLogs: The type-filter is no longer ignored on regular/loaded execution - requesting way fewer logs.
-* Fixed an error when importing individual job log statistics, and `ressourceType` was missing
+* Fixed an error when importing individual job log statistics, and `resourceType` was missing
 
 #### Changes
 
@@ -133,7 +219,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 * Reworks REST-API POST-Requests to be merged with GET-Requests
   * This includes the repeated tries if a request fails
   * Deprecates `url_set_param`, using functionality of `requests`-package instead
-  * Using Python-Structs for URL-params instead of cryptical encoded params
+  * Using Python-Structs for URL-params instead of cryptic encoded params
 * Reworks REST-API requests to use parameters more efficiently and consistently, making the code hopefully more readable.
   * Changes get_url_params to gain all parameters from URL-Encoding
   * Introduces set_url_params to set all params into URL encoding
@@ -142,7 +228,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
   * Changed to query so-far unknown endpoint, using count and group aggregate to query all data with a single API-request
   > This brings the SLA request in line with the other API requests.
 * Reworked/Commented the job-log request and prepared a filter via individual JobLog-ID's
-* Labeled python argument ` -create_dashboard` and associated args as deprecated, to be removed in v1.1 (See Grafana-Changes)
+* Labeled python argument `-create_dashboard` and associated args as deprecated, to be removed in v1.1 (See Grafana-Changes)
 
 ### Influx
 
@@ -184,7 +270,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 * Changed dashboard to be exported for `external use`:
   * You may change the data source on importing
   * Both UID and Dashboard names will be variable generated based on the data source chosen
-  * Labeled Python argument ` -create_dashboard` and associated args as deprecated, to be removed in v1.1
+  * Labeled Python argument `-create_dashboard` and associated args as deprecated, to be removed in v1.1
     > Note: Listed here only for completeness, see Python changes
   * Created a Python stand-alone script for generifying dashboard
     > Note: Listed here only for completeness, see script changes
@@ -263,10 +349,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 ### Changed
 
-* Reminder: `--minimumLogs` depricated, to be removed in V1.0. Use `--loadedSystem` instead
-* `--processStats` depricated, integrated into `--ssh`
-* `transfer_data` removed. Use `copy_datase` instead.
-  * removed `--old_database`, integraded into `copy_database` CFG file.
+* Reminder: `--minimumLogs` deprecated, to be removed in V1.0. Use `--loadedSystem` instead
+* `--processStats` deprecated, integrated into `--ssh`
+* `transfer_data` removed. Use `copy_database` instead.
+  * removed `--old_database`, integrated into `copy_database` CFG file.
 * `--test` implemented
 
 InfluxDB tables:
@@ -275,8 +361,8 @@ InfluxDB tables:
 * `jobs_statistics`: New
 * `jobLogs`: Renaming of arguments, adding `jobExecutionTime`.
 * `sppmon_metrics`: Added `influxdb_version` and new arguments
-* `vmReplicateSummary` and `vmReplicateStats`: `removed tag `messageId`
+* `vmReplicateSummary` and `vmReplicateStats`: `removed tag` messageId
 * `vadps`: Moved 3 tags to fields, adjusted CQ to run on distinct ID's
 * `ProcessStats`: Removed 2 fields and 3 tags due change `top` to `ps` command.
 * `office365stats` and `office365transfBytes` new
-* `df_ssh`: Renamed `avail` to `available`, bugfixing a tag
+* `df_ssh`: Renamed `avail` to `available`, bugfixing a tag