Skip to content

Remove config to use cache#69

Open
frack113 wants to merge 12 commits intoSigmaHQ:v0.30.0from
frack113:Refractor
Open

Remove config to use cache#69
frack113 wants to merge 12 commits intoSigmaHQ:v0.30.0from
frack113:Refractor

Conversation

@frack113
Copy link
Copy Markdown
Member

@frack113 frack113 commented Dec 7, 2025

Breack logic to use diskcache in v0.30.0

  • Use diskcache create:
    • data_filename
    • data_taxonomy
    • data_windows_eventid
    • data_windows_provider
  • Split sigmahq_windows_validator.json into
    • sigmahq_windows_eventid.json
    • sigmahq_windows_provider.json
  • Update json schema
  • rename field.py to fieldname.py
  • Remove dead code
    • config.py
    • update_ref.py
    • test_config.py
    • sigmahq_data.py

@frack113 frack113 marked this pull request as draft December 7, 2025 13:01
@frack113 frack113 marked this pull request as ready for review December 7, 2025 13:30
@frack113 frack113 changed the base branch from main to v0.30.0 December 10, 2025 05:30
@frack113 frack113 moved this to In review in Version 0.30.0 Dec 12, 2025
@nasbench nasbench requested a review from Copilot December 24, 2025 15:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the SigmaHQ validator to use diskcache for dynamic data loading instead of static generated Python files. The changes improve the architecture by:

  • Implementing disk caching for remote JSON data from the SigmaHQ repository
  • Splitting Windows validator data into separate eventid and provider JSON files
  • Removing dead code including the config system and data generation scripts
  • Updating validators to use the new data loading modules

Reviewed changes

Copilot reviewed 30 out of 31 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
sigma/validators/sigmahq/data/data_taxonomy.py New module for loading and caching taxonomy data with disk cache
sigma/validators/sigmahq/data/data_filename.py New module for loading and caching filename pattern data
sigma/validators/sigmahq/data/data_windows_eventid.py New module for Windows EventID validation data
sigma/validators/sigmahq/data/data_windows_provider.py New module for Windows provider validation data
sigma/validators/sigmahq/fieldname.py Updated to use new data_taxonomy module instead of config
sigma/validators/sigmahq/logsource.py Updated to use new data_taxonomy module
sigma/validators/sigmahq/filename.py Updated to use new data_filename module
sigma/validators/sigmahq/detection.py Updated to use new Windows data modules
sigma/validators/sigmahq/modifier.py New module containing field duplicate and all modifier validators
tools/sigmahq_windows_eventid.json New JSON file separating EventID validation data
tools/sigmahq_windows_provider.json Updated JSON file with only provider data
json-schema/schema_sigmahq_windows_eventid.json New JSON schema for EventID validation
json-schema/schema_sigmahq_windows_provider.json Updated schema removing eventid field
tests/* Updated test imports from field to fieldname/modifier modules

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@frack113
Copy link
Copy Markdown
Member Author

Test will not pass as the json are not in the main branch by now 😞
Wait for a new repo to put them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

2 participants