Remove config to use cache

.github/workflows/schema.yml

@@ -27,6 +27,7 @@ jobs:
 
           - name: Validate JSON files
             run: |
-                check-jsonschema --schemafile ./json-schema/schema_sigmahq_taxonomy.json ./tools/sigmahq_taxonomy.json  
-                check-jsonschema --schemafile ./json-schema/schema_sigmahq_filename.json ./tools/sigmahq_filename.json 
-                check-jsonschema --schemafile ./json-schema/schema_sigmahq_windows_validator.json ./tools/sigmahq_windows_validator.json
+                check-jsonschema --schemafile ./json-schema/schema_sigmahq_filename.json ./tools/sigmahq_filename.json
+                check-jsonschema --schemafile ./json-schema/schema_sigmahq_taxonomy.json ./tools/sigmahq_taxonomy.json
+                check-jsonschema --schemafile ./json-schema/schema_sigmahq_windows_eventid.json ./tools/sigmahq_windows_eventid.json
+                check-jsonschema --schemafile ./json-schema/schema_sigmahq_windows_provider.json ./tools/sigmahq_windows_provider.json

CHANGELOG.md

@@ -0,0 +1,131 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+### Added
+- Initial comprehensive validator set for SigmaHQ rules (49 validators)
+- Configurable parameters for flexible validation rules
+- Taxonomy-based field validation using sigmahq_taxonomy.json
+
+### Changed
+- Updated README to reflect current implementation status
+- Removed references to non-existent configuration files
+- Clarified programmatic usage in documentation
+
+## [v0.20.1] - 2023-12-06
+### Fixed
+- **Validator improvements**: Enhanced validation logic for title formatting and field name checks
+- **Bug fixes**: Resolved issues with status validation and correlation rule detection
+- **Performance**: Optimized validation execution for large rule sets
+
+## [v0.20.0] - 2023-11-26
+### Added
+- **New validators**:
+  - `SigmahqFieldUserValidator`: Enhanced User field name localization checks
+  - `SigmahqInvalidHashKvValidator`: Improved Sysmon Hash key-value search validation
+- **Configuration support**: Added configurable word lists for falsepositive validation
+
+## [v0.12.2] - 2023-11-11
+### Fixed
+- **Validation accuracy**:
+  - Improved detection of redundant modified fields
+  - Enhanced status field validation with folder checks
+- **Edge cases**: Better handling of empty or malformed rule files
+
+## [v0.12.1] - 2023-10-31
+### Security
+- **Dependency updates**:
+  - Updated sigma dependency to v4.0.0 with security patches
+  - Fixed potential vulnerability in regex pattern validation
+- **Validation hardening**: Added input sanitization for rule parsing
+
+## [v0.12.0] - 2023-10-31
+### Added
+- **Correlation rules support**:
+  - Enhanced `SigmahqCorrelationRulesMinimumValidator`
+  - Improved group-by field validation
+- **Metadata validation**: Stronger checks for level and logsource fields
+
+## [v0.11.0] - 2023-10-23
+### Changed
+- **Breaking changes**:
+  - Updated validator naming conventions to be more consistent
+  - Modified severity levels for certain validation cases
+- **Improvements**:
+  - Better error messages and issue descriptions
+  - Enhanced test coverage for edge cases
+
+## [v0.10.2] - 2023-09-23
+### Fixed
+- **Bug fixes**:
+  - Resolved false positives in title validation
+  - Fixed issues with field existence checks
+- **Performance**: Optimized validator execution order
+
+## [v0.10.1] - 2023-09-22
+### Documentation
+- **Improved documentation**:
+  - Enhanced README with detailed configuration table
+  - Added validation severity levels explanation
+  - Better examples in contributing guidelines
+
+## [v0.10.0] - 2023-07-29
+### Added
+- **New validators**:
+  - `SigmahqTagsTechniquesWithoutTacticsValidator`
+  - `SigmahqInvalidAllModifierValidator`
+- **Configuration system**: Initial support for configurable parameters
+
+## [v0.9.6] - 2023-05-29
+### Fixed
+- **Stability improvements**:
+  - Better handling of malformed YAML files
+  - Improved error recovery during validation
+- **Validation accuracy**: Enhanced detection of invalid field names
+
+## [v0.1.0] - Initial Development (YYYY-MM-DD)
+### Added
+- **Core infrastructure**:
+  - Validator base classes and patterns
+  - Comprehensive test framework
+- **Initial validators**:
+  - Author, date, description validation
+  - Basic condition and field validation
+- **Taxonomy support**: Integrated sigmahq_taxonomy.json for field validation
+
+## Types of changes
+```markdown
+[Unreleased]
+### Added
+- A new feature or validator
+
+### Changed
+- A breaking change or significant improvement
+
+### Deprecated
+- A previously used feature or validator
+
+### Removed
+- A feature or validator that was removed
+
+### Fixed
+- A bug fix or improvement to existing functionality
+
+### Security
+- Vulnerability patches and security-related changes
+```
+
+## How to contribute
+1. **Check guidelines**: Review [CONTRIBUTING.md](CONTRIBUTING.md)
+2. **Discuss changes**: Open an issue for major changes
+3. **Submit PRs**: Follow code style and documentation standards
+4. **Testing**: Ensure comprehensive test coverage (1:1 with validators)
+
+## Support
+For issues or questions, please:
+- Check existing GitHub issues before opening new ones
+- Contact maintainers via GitHub discussions

CONTRIBUTING.md

@@ -0,0 +1,26 @@
+# 🤝 Contributing
+
+Contributions are welcome! Please follow these guidelines:
+
+1. **Adding Validators**:
+   - Follow the existing `Sigmahq[Category]Validator` naming pattern
+   - Implement a clear docstring explaining the validation purpose
+   - Add comprehensive test cases in `/tests/sigmahq/[category]`
+
+2. **Development Setup**:
+   ```bash
+   git clone https://github.com/SigmaHQ/pySigma-validators-sigmaHQ.git
+   cd pySigma-validators-sigmaHQ
+   poetry install
+   ```
+
+3. **Testing**:
+   Run the full test suite with coverage:
+   ```bash
+   poetry pytest
+   ```
+
+4. **Code Style**:
+   - Follow existing Python style (PEP 8)
+   - Use type hints where appropriate
+   - Maintain consistent docstring formatting

README.md

@@ -4,23 +4,172 @@
 ![Coverage Badge](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/frack113/b27ee1cbe964fb1a299cc20c3403f8c8/raw/pySigma-validators-sigmaHQ.json)
 ![Status](https://img.shields.io/badge/Status-pre--release-orange)
 
+
 ## 🌟 Purpose
 
-Create all validators specific to the requirements of the SigmaHQ rules repository
+Create all validators specific to the requirements of the SigmaHQ rules repository.
+
+This package provides comprehensive validation for Sigma rules, ensuring they meet SigmaHQ's quality standards and best practices through:
+
+- **Structural validation**: Checks required fields and proper formatting
+- **Content validation**: Ensures descriptions, titles, and tags follow conventions
+- **Technical validation**: Validates detection logic, field usage, and modifiers
+- **Metadata validation**: Verifies status, dates, authorship, and references
+
+## 📦 Installation
+
+Install the package using pip:
+
+```bash
+pip install pySigma-validators-sigmaHQ
+```
 
 ## 🏗️ Validators
 
-TBD
+### Core Rule Structure
+- **Author**:
+  - `SigmahqAuthorExistenceValidator`: Ensures author field is present
+
+- **Date & Modification**:
+  - `SigmahqDateExistenceValidator`: Validates required date field
+  - `SigmahqModifiedDateOrderValidator`: Checks modified date ordering
+  - `SigmahqModifiedWithoutDateValidator`: Prevents modified without date
+  - `SigmahqRedundantModifiedValidator`: Detects unnecessary modified fields
+
+- **Description**:
+  - `SigmahqDescriptionExistenceValidator`: Ensures description exists
+  - `SigmahqDescriptionLengthValidator`: Validates minimum description length (configurable)
+  - `SigmahqLinkInDescriptionValidator`: Checks for proper hyperlink references
+
+### Detection Logic
+- **Condition Validation**:
+  - `SigmahqOfthemConditionValidator`: Validates 'of them' usage patterns
+  - `SigmahqOfselectionConditionValidator`: Ensures correct 'All/X of' format
+  - `SigmahqMissingAsteriskConditionValidator`: Checks for required asterisks
+
+- **Field Validation**:
+  - `SigmahqCategoryEventIdValidator`: Validates EventID field usage by category
+  - `SigmahqCategoryWindowsProviderNameValidator`: Ensures proper Provider_Name usage
+  - `SigmahqUnsupportedRegexGroupConstructValidator`: Detects unsupported regex patterns
+
+### Correlation Rules
+- **Correlation Validation**:
+  - `SigmahqCorrelationRulesMinimumValidator`: Enforces minimum rule count for correlations
+  - `SigmahqCorrelationGroupByExistenceValidator`: Validates required group-by fields
+
+### Filename Conventions
+- `SigmahqFilenameConventionValidator`: Ensures proper filename patterns
+- `SigmahqCorrelationFilenamePrefixValidator`: Validates correlation filenames start with 'correlation_'
+- `SigmahqFilenamePrefixValidator`: Checks for correct prefix usage
+
+### Metadata Fields
+- **Level**:
+  - `SigmahqLevelExistenceValidator`: Validates required level field
+
+- **Logsource**:
+  - `SigmahqLogsourceUnknownValidator`: Detects unknown log sources
+  - `SigmahqSysmonMissingEventidValidator`: Ensures Sysmon rules have EventID
+
+- **Status**:
+  - `SigmahqStatusExistenceValidator`: Checks for required status field
+  - `SigmahqStatusValidator`: Validates Deprecated/Unsupported statuses with folder checks
+  - `SigmahqStatusToHighValidator`: Prevents premature status elevation (configurable min_days)
+
+### Tagging System
+- **Tag Validation**:
+  - `SigmahqTagsUniqueDetectionValidator`: Ensures unique detection tags
+  - `SigmahqTagsDetectionValidator`: Validates required detection tags by folder
+  - `SigmahqTagsUniqueTlpValidator`: Checks TLP tag uniqueness
+  - `SigmahqTagsTlpValidator`: Validates authorized TLP tags (configurable word list)
+  - `SigmahqTagsTechniquesWithoutTacticsValidator`: Ensures technique-tactic pairing
+
+### Title Formatting
+- **Title Validation**:
+  - `SigmahqTitleLengthValidator`: Enforces maximum title length (default: 120 chars, configurable)
+  - `SigmahqTitleStartValidator`: Prevents titles starting with 'Detect'/'Detects'
+  - `SigmahqTitleEndValidator`: Ensures proper title ending (no trailing dots)
+  - `SigmahqTitleCaseValidator`: Validates article/preposition casing
 
-## 🧬 Data
+### References & Links
+- `SigmahqGithubLinkValidator`: Validates GitHub link presence in rules
+- `SigmahqMitreLinkValidator`: Checks for proper MITRE tag usage instead of links
 
-All the data value are in the config.py
+### False Positives
+- **Falsepositive Validation**:
+  - `SigmahqFalsepositivesCapitalValidator`: Ensures proper capitalization
+  - `SigmahqFalsepositivesBannedWordValidator`: Detects banned words (configurable word list)
+  - `SigmahqFalsepositivesTypoWordValidator`: Checks for common typos
 
-To use a local json version, you need to put them in a `validator_json` folder visible from the launch directory.
+### Field-Specific Validation (Detection Items)
+- **Field Name**:
+  - `SigmahqSpaceFieldNameValidator`: Prevents space in field names
+  - `SigmahqFieldnameCastValidator`: Detects type casting errors
+  - `SigmahqInvalidFieldnameValidator`: Validates field existence in logsource
+  - `SigmahqFieldUserValidator`: Checks for localized User field names
+  - `SigmahqInvalidHashKvValidator`: Validates Sysmon Hash key-value searches
+  - `SigmahqRedundantFieldValidator`: Detects redundant field usage
 
-## 📜 Maintainer
+- **Fields**:
+  - `SigmahqFieldsExistenceValidator`: Validates deprecated fields usage
+  - `SigmahqUnknownFieldValidator`: Detects unknown field references
 
-This pipeline is currently maintained by:
+- **Modifier**:
+  - `SigmahqFieldDuplicateValueValidator`: Checks for duplicate values in field lists
+  - `SigmahqInvalidAllModifierValidator`: Prevents invalid All modifier usage
+
+## 🧬 Configuration
+
+### Current Implementation
+All validation logic is implemented directly in validator classes within `sigma/validators/sigmahq`.
+No external configuration files are currently required.
+
+### Configurable Parameters
+
+| Validator | Parameter | Type | Default Value | Description |
+|-----------|----------|------|---------------|-------------|
+| Title Length | `max_length` | int | 120 | Maximum allowed title length in characters |
+| Status Elevation | `min_days` | int | 60 | Minimum days before status can be elevated beyond EXPERIMENTAL |
+| Falsepositive Words | `word_list` | Tuple[str] | See source | List of banned words for falsepositive validation |
+
+### Future Configuration
+
+The architecture supports external configuration through **Class-level parameters** 
+For local customizations create a `validator_json` folder in your project root
+
+## 📂 File Structure
+
+```
+sigma/
+  validators/
+    sigmahq/
+      [validator_files].py  # All validation logic here (49 validators)
+tests/
+  sigmahq/                  # Comprehensive test cases (1:1 with validators)
+tools/
+  sigmahq_taxonomy.json     # Taxonomy data used by field validators
+```
+
+## 🛠️ Validation Severity
+
+Validators use the following severity levels:
+
+| Severity | Description | Example Use Cases |
+|----------|-------------|-------------------|
+| **LOW** | Minor style issues that don't affect functionality | Title casing, redundant fields |
+| **MEDIUM** | Potential quality issues that should be addressed | Long titles, missing descriptions |
+| **HIGH** | Critical issues that may break rule functionality | Missing required fields, invalid statuses |
+
+All validators currently use MEDIUM severity unless they detect critical issues.
+
+## 📜 Maintainers
+
+This project is maintained by:
 
 * [François Hubaut (@frack113)](https://twitter.com/frack113)
 * [Christian Burkard (@phantinuss)](https://twitter.com/phantinuss)
+
+
+## 🔍 License
+
+GNU Lesser General Public License v2.1 (LGPL-2.1)
+See [LICENSE](https://github.com/SigmaHQ/pySigma-validators-sigmaHQ/blob/main/LICENSE) for details.