Skip to content

feat: add -X, --remove-duplicate-detections option to eid-metrics and logon-summary#1552

Merged
YamatoSecurity merged 2 commits intomainfrom
1478-add-X-metrics-command
Jan 24, 2025
Merged

feat: add -X, --remove-duplicate-detections option to eid-metrics and logon-summary#1552
YamatoSecurity merged 2 commits intomainfrom
1478-add-X-metrics-command

Conversation

@fukusuket
Copy link
Collaborator

@fukusuket fukusuket commented Jan 23, 2025 •
edited
Loading

What Changed

Evidence

Integration-Test

I would appreciate it if you could check it out when you have time🙏

@fukusuket fukusuket added the enhancement New feature or request label Jan 23, 2025
@fukusuket fukusuket added this to the 3.1 (2025/2/22 Ninja Day) milestone Jan 23, 2025
@fukusuket fukusuket self-assigned this Jan 23, 2025
@fukusuket
Copy link
Collaborator Author

fukusuket commented Jan 23, 2025 •
edited
Loading

eid-metrics

Test data

fukusuke@fukusukenoMacBook-Air hayabusa-3.0.1-mac-aarch64 % find ../rdp_test
../rdp_test
../rdp_test/Microsoft-Windows-TerminalServices-LocalSessionManager%4Operational.evtx
../rdp_test/dup
../rdp_test/dup/Microsoft-Windows-TerminalServices-LocalSessionManager%4Operational.evtx

without -X

fukusuke@fukusukenoMacBook-Air hayabusa-3.0.1-mac-aarch64 % ./hayabusa eid-metrics -d ../rdp_test -q
Generating Event ID Metrics

Start time: 2025/01/23 20:16
Total event log files: 2
Total file size: 139.3 KB

Currently scanning for event ID metrics. Please wait.

[00:00:00] 2 / 2   [========================================] 100%

Scanning finished.


Total Event Records:  152

First Timestamp:  2024-11-14 12:22:50.756 +09:00
Last Timestamp:  2024-11-23 07:50:25.865 +09:00

╭───────┬───────┬─────────┬────┬───────────────────────────────────╮
│ Total ┆   %   ┆ Channel ┆ ID ┆               Event               │
╞═══════╪═══════╪═════════╪════╪═══════════════════════════════════╡
│ 60    ┆ 39.5% ┆ RDS-LSM ┆ 34 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 16    ┆ 10.5% ┆ RDS-LSM ┆ 40 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 12    ┆ 7.9%  ┆ RDS-LSM ┆ 24 ┆ Session disconnected              │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 10    ┆ 6.6%  ┆ RDS-LSM ┆ 41 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 10    ┆ 6.6%  ┆ RDS-LSM ┆ 22 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 10    ┆ 6.6%  ┆ RDS-LSM ┆ 42 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 10    ┆ 6.6%  ┆ RDS-LSM ┆ 21 ┆ Shell start notification received │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 8     ┆ 5.3%  ┆ RDS-LSM ┆ 23 ┆ Session logoff                    │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 6     ┆ 3.9%  ┆ RDS-LSM ┆ 32 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 6     ┆ 3.9%  ┆ RDS-LSM ┆ 54 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 4     ┆ 2.6%  ┆ RDS-LSM ┆ 25 ┆ Unknown                           │
╰───────┴───────┴─────────┴────┴───────────────────────────────────╯
Elapsed time: 00:00:00.007

with -X

fukusuke@fukusukenoMacBook-Air hayabusa-3.0.1-mac-aarch64 % ./hayabusa eid-metrics -d ../rdp_test -q -X
Generating Event ID Metrics

Start time: 2025/01/23 20:16
Total event log files: 2
Total file size: 139.3 KB

Currently scanning for event ID metrics. Please wait.

[00:00:00] 2 / 2   [========================================] 100%

Scanning finished.


Total Event Records:  152

First Timestamp:  2024-11-14 12:22:50.756 +09:00
Last Timestamp:  2024-11-23 07:50:25.865 +09:00

╭───────┬───────┬─────────┬────┬───────────────────────────────────╮
│ Total ┆   %   ┆ Channel ┆ ID ┆               Event               │
╞═══════╪═══════╪═════════╪════╪═══════════════════════════════════╡
│ 30    ┆ 19.7% ┆ RDS-LSM ┆ 34 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 8     ┆ 5.3%  ┆ RDS-LSM ┆ 40 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 6     ┆ 3.9%  ┆ RDS-LSM ┆ 24 ┆ Session disconnected              │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 5     ┆ 3.3%  ┆ RDS-LSM ┆ 22 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 5     ┆ 3.3%  ┆ RDS-LSM ┆ 42 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 5     ┆ 3.3%  ┆ RDS-LSM ┆ 21 ┆ Shell start notification received │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 5     ┆ 3.3%  ┆ RDS-LSM ┆ 41 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 4     ┆ 2.6%  ┆ RDS-LSM ┆ 23 ┆ Session logoff                    │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 3     ┆ 2.0%  ┆ RDS-LSM ┆ 54 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 3     ┆ 2.0%  ┆ RDS-LSM ┆ 32 ┆ Unknown                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 2     ┆ 1.3%  ┆ RDS-LSM ┆ 25 ┆ Unknown                           │
╰───────┴───────┴─────────┴────┴───────────────────────────────────╯
Elapsed time: 00:00:00.006

@fukusuket
Copy link
Collaborator Author

fukusuket commented Jan 23, 2025 •
edited
Loading

logon-summary

Test data

fukusuke@fukusukenoMacBook-Air hayabusa-3.0.1-mac-aarch64 % find ../rdp_test
../rdp_test
../rdp_test/Microsoft-Windows-TerminalServices-LocalSessionManager%4Operational.evtx
../rdp_test/dup
../rdp_test/dup/Microsoft-Windows-TerminalServices-LocalSessionManager%4Operational.evtx

without -X

fukusuke@fukusukenoMacBook-Air hayabusa-3.0.1-mac-aarch64 % ./hayabusa logon-summary -d ../rdp_test -q
Generating Logon Summary

Start time: 2025/01/23 20:17
Total event log files: 2
Total file size: 139.3 KB

Currently scanning for the logon summary. Please wait.

[00:00:00] 2 / 2   [========================================] 100%

Scanning finished.


Total Event Records:  152

First Timestamp:  2024-11-14 12:22:50.756 +09:00
Last Timestamp:  2024-11-23 07:50:25.865 +09:00

Logon Summary:
Successful Logons:
╭────────────┬────────────┬────────────────┬─────────────────┬─────────────────┬───────────────────╮
│ Successful ┆ Event      ┆ Target Account ┆ Target Computer ┆ Source Computer ┆ Source IP Address │
╞════════════╪════════════╪════════════════╪═════════════════╪═════════════════╪═══════════════════╡
│ 6          ┆ RDS-LSM 21 ┆ Administrator  ┆ EC2AMAZ-3NFFVNI ┆ -               ┆ 219.100.37.234    │
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 4          ┆ RDS-LSM 21 ┆ samurai        ┆ EC2AMAZ-3NFFVNI ┆ -               ┆ 219.100.37.234    │
╰────────────┴────────────┴────────────────┴─────────────────┴─────────────────┴───────────────────╯



Failed Logons:
 No logon failed events were detected.

Elapsed time: 00:00:00.009

with -X

fukusuke@fukusukenoMacBook-Air hayabusa-3.0.1-mac-aarch64 % ./hayabusa logon-summary -d ../rdp_test -q -X
Generating Logon Summary

Start time: 2025/01/23 20:17
Total event log files: 2
Total file size: 139.3 KB

Currently scanning for the logon summary. Please wait.

[00:00:00] 2 / 2   [========================================] 100%

Scanning finished.


Total Event Records:  152

First Timestamp:  2024-11-14 12:22:50.756 +09:00
Last Timestamp:  2024-11-23 07:50:25.865 +09:00

Logon Summary:
Successful Logons:
╭────────────┬────────────┬────────────────┬─────────────────┬─────────────────┬───────────────────╮
│ Successful ┆ Event      ┆ Target Account ┆ Target Computer ┆ Source Computer ┆ Source IP Address │
╞════════════╪════════════╪════════════════╪═════════════════╪═════════════════╪═══════════════════╡
│ 3          ┆ RDS-LSM 21 ┆ Administrator  ┆ EC2AMAZ-3NFFVNI ┆ -               ┆ 219.100.37.234    │
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 2          ┆ RDS-LSM 21 ┆ samurai        ┆ EC2AMAZ-3NFFVNI ┆ -               ┆ 219.100.37.234    │
╰────────────┴────────────┴────────────────┴─────────────────┴─────────────────┴───────────────────╯



Failed Logons:
 No logon failed events were detected.

Elapsed time: 00:00:00.009

@fukusuket fukusuket marked this pull request as ready for review January 23, 2025 11:20
@fukusuket fukusuket changed the title feat: add -X, --remove-duplicate-detections option to 'eid-metrics' and logon-summary feat: add -X, --remove-duplicate-detections option to eid-metrics and logon-summary Jan 23, 2025
YamatoSecurity
YamatoSecurity approved these changes Jan 24, 2025
View reviewed changes
Copy link
Collaborator

@YamatoSecurity YamatoSecurity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fukusuket LGTM! Thanks so much!

@YamatoSecurity YamatoSecurity merged commit 6a14a63 into main Jan 24, 2025
@YamatoSecurity YamatoSecurity deleted the 1478-add-X-metrics-command branch January 24, 2025 08:41
@fukusuket fukusuket mentioned this pull request Jan 24, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@YamatoSecurity YamatoSecurity YamatoSecurity approved these changes

Assignees

@fukusuket fukusuket

Labels

enhancement New feature or request

Projects

None yet

Milestone

3.1 (2025/2/22 Ninja Day)

Development

Successfully merging this pull request may close these issues.

Invalid: -x option in logon-summary and eid-metrics will result in duplicate results

2 participants

@fukusuket @YamatoSecurity