New benchmarking suite (#42)

* init benchmark

* fix details

* fix lint

* add benchmark tests

* paper benchmark results (#41)

* Enable Unrestricted LLM Output with Parsing + Add One-Shot Anomaly Detection Support (#39)

* Core Changed to get normal behavior in pipeline

* Transformation changed

* anomalies.py changed

* Hugginface.py changed : no restrictions token, and also has normal as input if 1-shot

* Timeseries preprocessing.py

* jsons files added for primitives

* jsons files added for primitives

* pipelines 0shot and 1shot added

* add boolean for restrict_tokens in HF

* good messages.json for prompt

* Added load_normal in sigllm.data

* Fixed load_normal in sigllm.data

* Fixed lint format

* Fixed lint format Ruff

* Fixed from review Sarah

* Fixed lint format after working on Sarah's reviews

* Dataset prompter parameters

* .jons removed from input names in 1_shot pipeline.json

* .jons removed from input names in 1_shot pipeline.json

* fix PR issues & add unittests

* add unittests for parse_anomaly_response

* remove unused functions

* add new functionality tests

* update ubuntu image

* change normal->single

* fix lint

* swap normal -> single

---------

Co-authored-by: Salim Cherkaoui <[email protected]>
Co-authored-by: Sarah Alnegheimish <[email protected]>

* support load_normal

---------

Co-authored-by: scherkao31 <[email protected]>
Co-authored-by: Salim Cherkaoui <[email protected]>

Author: 3 people

pyproject.toml

@@ -204,4 +204,11 @@ convention = "google"
 
 [tool.ruff.lint.pycodestyle]
 max-doc-length = 99
-max-line-length = 99
+max-line-length = 99
+
+[tool.coverage.report]
+exclude_also = [
+    "def __repr__",
+    "def main",
+    "if __name__ == .__main__.:"
+]