You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Rule Tuning] Rapid Secret Retrieval Attempts from AWS SecretsManager (#5291)
This rule is quite loud in telemetry. However over 80% of alerts come from a single cluster, which seems to have a lot of role usage and operation-type IAM Users which assume other roles constantly. This makes me think the majority of these alerts may be from multiple role sessions retrieving secrets, rather than a single user retrieving multiple secrets rapidly. I've looked at the prod telemetry data we have access to and found a few instances where a single secret is accessed multiple times or certain AWS services retrieve multple secrets rapidly. All of these instances have been accounted for in the tunings here.
- query has been changed to remove `GetBatchSecretValue` from the query since this API call actually calls the `GetSecretValue` for each of the secrets it's retrieving. So we only need to look for `GetSecretValue`
- I've added the AWS services found in prod data that contribute to rapid secret retrieval
- Changed the threshold parameters to look for a single `user.id` retrieving more than 20 unique secret values (`aws.cloudtrail.request_parameters`)
- updated the description and investigation guide
- reduced execution window
Copy file name to clipboardExpand all lines: rules/integrations/aws/credential_access_rapid_secret_retrieval_attempts_from_secretsmanager.toml
+80-44Lines changed: 80 additions & 44 deletions
Original file line number
Diff line number
Diff line change
@@ -2,78 +2,105 @@
2
2
creation_date = "2024/04/11"
3
3
integration = ["aws"]
4
4
maturity = "production"
5
-
updated_date = "2025/02/03"
5
+
updated_date = "2025/11/07"
6
6
7
7
[rule]
8
8
author = ["Elastic"]
9
9
description = """
10
-
This rule attempts to identify rapid secret retrieval attempts from AWS SecretsManager. Adversaries may attempt to
11
-
retrieve secrets from the Secrets Manager programmatically using the `GetSecretValue` or `BatchGetSecretValue` API
12
-
actions.
10
+
Identifies rapid secret retrieval activity from AWS Secrets Manager using the GetSecretValue or BatchGetSecretValue API
11
+
actions. Adversaries who compromise an IAM user, instance role, or temporary credentials may attempt to enumerate or
12
+
exfiltrate secrets in bulk to escalate privileges, move laterally, or gain persistence. This rule detects 20 or more
13
+
unique secret retrievals by the same user identity within a short time window, which may indicate credential compromise
14
+
or automated secret harvesting.
13
15
"""
14
16
false_positives = [
15
17
"""
16
-
Verify whether the user identity, user agent, and/or hostname should be using GetSecretString or BatchGetSecretValue
18
+
Verify whether the user identity, user agent, and/or hostname should be using GetSecretValue or BatchGetSecretValue
17
19
APIs for the specified SecretId. If known behavior is causing false positives, it can be exempted from the rule.
18
20
""",
19
21
]
20
-
from = "now-9m"
22
+
from = "now-6m"
21
23
index = ["filebeat-*", "logs-aws.cloudtrail*"]
22
24
language = "kuery"
23
25
license = "Elastic License v2"
24
-
name = "Rapid Secret Retrieval Attempts from AWS SecretsManager"
26
+
name = "AWS Secrets Manager Rapid Secrets Retrieval"
25
27
note = """## Triage and analysis
26
28
27
-
### Investigating Rapid Secret Retrieval Attempts from AWS SecretsManager
29
+
> **Disclaimer**:
30
+
> This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance.
31
+
> While every effort has been made to ensure its quality, validate and adapt it for your operational needs.
28
32
29
-
AWS Secrets Manager is a service that enables the replacement of hardcoded credentials in code, including passwords, with an API call to Secrets Manager to retrieve the secret programmatically.
This rule looks for the rapid retrieval of credentials using `GetSecretValue` or `BatchGetSecretValue` actions in Secrets Manager programmatically. This is a [Threshold](https://www.elastic.co/guide/en/security/current/rules-ui-create.html#create-threshold-rule) rule indicating 20 or more successful attempts to retrieve a secret value from Secrets Manager by the same user identity within a short timespan.
35
+
AWS Secrets Manager stores sensitive credentials such as database passwords, API keys, OAuth tokens, and service
36
+
configuration values. In credential compromise scenarios, attackers frequently attempt to retrieve as many secrets as
37
+
possible in a short timeframe to escalate privileges or move laterally across the environment.
38
+
39
+
This threshold rule triggers when a single user identity successfully retrieves 20 or more unique secrets using
40
+
`GetSecretValue` or `BatchGetSecretValue` within a short timespan. Retrieval of many different secrets in rapid succession is highly unusual and strongly associated with reconnaissance, secret harvesting, or compromised automation.
41
+
42
+
Note: `BatchGetSecretValue` API calls the `GetSecretValue` API for each secret value; this alert only captures the `GetSecretValue` calls rather than the `BatchGetSecretValue` call itself.
32
43
33
44
#### Possible investigation steps
34
45
35
-
- Identify the account and its role in the environment, and inspect the related policy.
36
-
- Identify the applications that should use this account.
37
-
- Investigate other alerts associated with the user account during the past 48 hours.
38
-
- Investigate abnormal values in the `user_agent.original` field by comparing them with the intended and authorized usage and historical data. Suspicious user agent values include non-SDK, AWS CLI, custom user agents, etc.
39
-
- Assess whether this behavior is prevalent in the environment by looking for similar occurrences involving other users.
40
-
- Contact the account owner and confirm whether they are aware of this activity.
41
-
- Considering the source IP address and geolocation of the user who issued the command:
42
-
- Do they look normal for the calling user?
43
-
- If the source is an EC2 IP address, is it associated with an EC2 instance in one of your accounts or is the source IP from an EC2 instance that's not under your control?
44
-
- If it is an authorized EC2 instance, is the activity associated with normal behavior for the instance role or roles? Are there any other alerts or signs of suspicious activity involving this instance?
45
-
- Review IAM permission policies for the user identity and specific secrets accessed.
46
-
- Examine the request parameters. These might indicate the source of the program or the nature of its tasks.
47
-
- If you suspect the account has been compromised, scope potentially compromised assets by tracking servers, services, and data accessed by the account in the last 24 hours.
46
+
- **Identify the user or role**
47
+
- Review `aws.cloudtrail.user_identity.arn`, `user.name`, and `aws.cloudtrail.user_identity.type`.
48
+
- Determine whether the identity normally accesses Secrets Manager or is tied to a known automation workload.
49
+
50
+
- **Analyze the set of secrets retrieved**
51
+
- Expand the alert in Timeline and review `aws.cloudtrail.request_parameters` for all `SecretId` values in the grouped threshold event.
52
+
- Identify whether the accessed secrets include:
53
+
- Privileged database credentials
54
+
- IAM user or service account credentials
55
+
- Production application secrets
56
+
- Rarely accessed or high-sensitivity secrets
57
+
58
+
- **Assess the runtime context**
59
+
- Investigate `source.ip`, `source.geo.location`, and `user_agent.original`.
60
+
- Validate whether the calls originated from known internal automation (e.g., ECS task, Lambda runtime, EC2 instance profile)
61
+
or from an unexpected IP or user agent.
62
+
63
+
- **Correlate with other activity from the same identity**
64
+
- Look for related reconnaissance or credential-access events:
65
+
- `ListSecrets`, `DescribeSecret`
66
+
- IAM enumeration (`ListUsers`, `GetCallerIdentity`)
67
+
- Role-chaining or unusual `AssumeRole` flows
68
+
- Check for subsequent use of exposed credentials (RDS login attempts, API activity, abnormal resource access).
69
+
70
+
- **Determine whether unusual automation or deployment activity is occurring**
71
+
- Confirm with application owners whether a new deployment, config reload, or migration might explain the multi-secret access.
48
72
49
73
### False positive analysis
50
74
51
-
- False positives may occur due to the intended usage of the service. Tuning is needed in order to have higher confidence. Consider adding exceptions — preferably with a combination of user agent and IP address conditions.
75
+
- Legitimate application initialization or rollouts may retrieve many secrets once on startup.
76
+
- CI/CD pipelines or configuration management tools may enumerate secrets as part of environment bootstrapping.
77
+
78
+
To reduce noise, consider exceptions based on:
79
+
- Known service roles
80
+
- Expected source IP ranges
81
+
- Specific application identities tied to secret orchestration
52
82
53
83
### Response and remediation
54
84
55
-
- Initiate the incident response process based on the outcome of the triage.
56
-
- Disable or limit the account during the investigation and response.
57
-
- Identify the possible impact of the incident and prioritize accordingly; the following actions can help you gain context:
58
-
- Identify the account role in the cloud environment.
59
-
- Assess the criticality of affected services and servers.
60
-
- Work with your IT team to identify and minimize the impact on users.
61
-
- Identify if the attacker is moving laterally and compromising other accounts, servers, or services.
62
-
- Identify any regulatory or legal ramifications related to this activity.
63
-
- Investigate credential exposure on systems compromised or used by the attacker to ensure all compromised accounts are identified. Rotate secrets or delete API keys as needed to revoke the attacker's access to the environment. Work with your IT teams to minimize the impact on business operations during these actions.
64
-
- Check if unauthorized new users were created, remove unauthorized new accounts, and request password resets for other IAM users.
65
-
- Consider enabling multi-factor authentication for users.
66
-
- Review the permissions assigned to the implicated user to ensure that the least privilege principle is being followed.
67
-
- Implement security best practices [outlined](https://aws.amazon.com/premiumsupport/knowledge-center/security-best-practices/) by AWS.
68
-
- Take the actions needed to return affected systems, data, or services to their normal operational levels.
69
-
- Identify the initial vector abused by the attacker and take action to prevent reinfection via the same vector.
70
-
- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the mean time to respond (MTTR).
85
+
- **Containment**
86
+
- Immediately revoke or disable the IAM user, role session, or instance profile if compromise is suspected.
0 commit comments