[Rule Tuning] December-January AWS Rule Tuning

[terrancedejesus]

Pull Request

Issue link(s):

• https://github.com/elastic/ia-trade-team/issues/272
• https://github.com/elastic/ia-trade-team/issues/517

Summary - What I changed

This pull request tunes several AWS noisy rules based on telemetry from December 2024 --> January 2025.

For peer review - details regarding the tunings and context is available in the monthly AWS meta. However, I have summarized the rules and their respective tunings below.

AWS STS Temporary Credentials via AssumeRole

• Reason for Tuning: Shifted to a list of inclusions for aws.cloudtrail.user_identity.invoked_by to inherently exclude AWS services inaccessible to adversaries, significantly reducing false positives from global alerts (from 2.6M to ~2.4K alerts).

AWS Access Secret in Secrets Manager

• Reason for Tuning: Noise primarily from clusters accessing different secret keys for the same user caused frequent false positives. Changed New Terms to user.id but noted global tuning limitations, requiring customers to manage exceptions.

AWS SSM SendCommand Execution by Rare User

• Reason for Tuning: Reduced false positives by focusing on user in New Terms and excluding AWS service domains. Alerts decreased from 1.4M to ~312 globally in 60 days, ensuring key behavior is still captured.

AWS Systems Manager SecureString Parameter Request with Decryption Flag

• Reason for Tuning: Aggregated noisy alerts (~907K) caused by a specific cluster. Updated New Terms to aws.cloudtrail.user_identity.arn to reduce false positives and added investigation guidance for analyzing user.id activity.

AWS IAM Assume Role Policy Update

• Reason for Tuning: Transitioned to a New Terms rule based on user identity ARN and target role name, eliminating many false positives (~292K). Updated investigation notes and created a PwnCloud module to better simulate and validate activity.

AWS EC2 User Data Retrieval for EC2 Instance

• Reason for Tuning: Focused on user_identity ARN and instanceId to detect anomalous activity while excluding AWS internal calls (aws.cloudtrail.user_identity.invoked_by). Reduced false positives (~177K alerts globally to ~45K), with further reduction expected.

AWS EC2 Route Table Modified or Deleted

• Reason for Tuning: Updated to a New Terms rule targeting user identity ARN, excluding AWS internal services (CloudFormation, Service Catalog, FSX). This change reduced noise (~84K alerts) related to IaaC activities like Terraform without relying on user-agent filtering. Updated the investigation guide and description.

QoL Changes

• Added investigation guides where missing
• Updated description and rule naming (including file naming) if necessary to improve clarity about the threat

How To Test

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[github-actions]

Rule: Tuning - Guidelines

These guidelines serve as a reminder set of considerations when tuning an existing rule.

Documentation and Context

• Detailed description of the suggested changes.
• Provide example JSON data or screenshots.
• Provide evidence of reducing benign events mistakenly identified as threats (False Positives).
• Provide evidence of enhancing detection of true threats that were previously missed (False Negatives).
• Provide evidence of optimizing resource consumption and execution time of detection rules (Performance).
• Provide evidence of specific environment factors influencing customized rule tuning (Contextual Tuning).
• Provide evidence of improvements made by modifying sensitivity by changing alert triggering thresholds (Threshold Adjustments).
• Provide evidence of refining rules to better detect deviations from typical behavior (Behavioral Tuning).
• Provide evidence of improvements of adjusting rules based on time-based patterns (Temporal Tuning).
• Provide reasoning of adjusting priority or severity levels of alerts (Severity Tuning).
• Provide evidence of improving quality integrity of our data used by detection rules (Data Quality).
• Ensure the tuning includes necessary updates to the release documentation and versioning.

Rule Metadata Checks

• updated_date matches the date of tuning PR merged.
• min_stack_version should support the widest stack versions.
• name and description should be descriptive and not include typos.
• query should be inclusive, not overly exclusive. Review to ensure the original intent of the rule is maintained.

Testing and Validation

• Validate that the tuned rule's performance is satisfactory and does not negatively impact the stack.
• Ensure that the tuned rule has a low false positive rate.

[terrancedejesus]

Note that while this is a relatively small change, we note why we are unable to tune the rule at this time in the tuning meta. As a result, we want to encourage user's to add exceptions on user.id to reduce FPs.

[terrancedejesus]

This is not a new rule, but renamed. logic adjusted and moved out of BBR.

[terrancedejesus]

Not a new rule, name and file name adjusted, investigation guide added and logic adjusted.

[eric-forte-elastic]

🟢 Manual review, looks good to me! 👍