Add rules for Azure Activity Logs/GCP Audit ML jobs

[jmcarlock]

Pull Request

Issue link(s):

• https://github.com/elastic/security-team/issues/11909

Summary - What I changed

Add detection rules for the new Azure Activity Logs/GCP Audit logs jobs.

• Add Security ML modules for GCP Audit and Azure Activity Logs kibana#236849

These jobs will be released with 9.3.0.

How To Test

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[github-actions]

Rule: New - Guidelines

These guidelines serve as a reminder set of considerations when proposing a new rule.

Documentation and Context

• Detailed description of the rule.
• List any new fields required in ECS/data sources.
• Link related issues or PRs.
• Include references.

Rule Metadata Checks

• creation_date matches the date of creation PR initially merged.
• min_stack_version should support the widest stack versions.
• name and description should be descriptive and not include typos.
• query should be inclusive, not overly exclusive, considering performance for diverse environments. Non ecs fields should be added to non-ecs-schema.json if not available in an integration.
• min_stack_comments and min_stack_version should be included if the rule is only compatible starting from a specific stack version.
• index pattern should be neither too specific nor too vague, ensuring it accurately matches the relevant data stream (e.g., use logs-endpoint.process-* for process data).
• integration should align with the index. If the integration is newly introduced, ensure the manifest, schemas, and new_rule.yaml template are updated.
• setup should include the necessary steps to configure the integration.
• note should include any additional information (e.g. Triage and analysis investigation guides, timeline templates).
• tags should be relevant to the threat and align/added to the EXPECTED_RULE_TAGS in the definitions.py file.
• threat, techniques, and subtechniques should map to ATT&CK always if possible.

New BBR Rules

• building_block_type should be included if the rule is a building block and the rule should be located in the rules_building_block folder.
• bypass_bbr_timing should be included if adding custom lookback timing to the rule.

Testing and Validation

• Provide evidence of testing and detecting the expected threat.
• Check for existence of coverage to prevent duplication.

[jmcarlock]

Hi @shashank-elastic 👋 I have a question on these, should we adapt the triage guide from the CloudWatch jobs (which these are very similar to), or can you generate new ones with your team's AI process?

[susan-shu-c]

@Mikaayenson question:
For AWS jobs - outside this PR, but we are using them as reference as they are cloud jobs - I noticed that the ML jobs example do not have [rule.treat.technique] mapped but a few others do. - example

Is that something we should be adding in this PR?

[terrancedejesus]

We should definitely have these for Entra ID Sign-In Logs. Happy to talk and discuss more!

[shashank-elastic]

@jmcarlock We can generate the investigation guides using our AI tool.

2 Things for the PR

• If these jobs are applicable for 9.3.0 and above we should keep them min_stack_version=9.3.0 ( Please check PAD rules recently added by @sodhikirti07 )
• I think all ML jobs are maintained under rules/ml/ folder structure irrespective of the integration used upstream. So this should have a changed directory structure.
• I also feel 9.3.0 Is Public Release in Jan 2026. This can be moved to draft or have a comment to merge only when we have 9.3.0 branch prepped for detection-rules.

[jmcarlock]

Thanks @terrancedejesus and @shashank-elastic for the review!

I think all ML jobs are maintained under rules/ml/ folder structure irrespective of the integration used upstream. So this should have a changed directory structure.

@shashank-elastic I am fine making this change, but to be consistent then the AWS CloudTrail jobs (which these are most similar to) should also move, as they are under rules/integrations/aws. Which way would you prefer?

[jmcarlock]

• I also feel 9.3.0 Is Public Release in Jan 2026. This can be moved to draft or have a comment to merge only when we have 9.3.0 branch prepped for detection-rules.

One other question here, the jobs themselves are present in the main branch, which serverless releases weekly. I'm not sure if we will request this, but is there a way to release them earlier for serverless?

[shashank-elastic]

@jmcarlock

• Since there are some ML jobs already under AWS folder and these are similar, I have referred back this to my team so we can decide whats the right folder structure needed. So i will communicate the same once we have reconciled on it.
• Currently we don't have a separate release process for serverless, its the same packages that are applied on both product variants. So If we wanna release this along side 9.2.0 (which is the next major release for us in the week of Aug 20 ) then it will have to be on both product variants.
• Since jobs are available in 9.3.0 ( based on discussions here) its best to min-stack on 9.3.0 else the rules will refer to a non existent job on previous clusters.

[jmcarlock]

Currently we don't have a separate release process for serverless, its the same packages that are applied on both product variants. So If we wanna release this along side 9.2.0 (which is the next major release for us in the week of Aug 20 ) then it will have to be on both product variants.

Thanks @shashank-elastic, I just have a couple more questions. First can you correct the date?

Also, if we merge these rules in your next release, would they get picked up on Serverless if we keep the min_stack_version set to 9.3.0, but be excluded from 9.2.0?

cc @Mikaayenson @pantea-elastic

[shashank-elastic]

Oh @jmcarlock Apologies, we have the release prep and release in week of Oct 20.

I see the next steps as follows.

• When TRADE creates a 9.2.0 branch and makes its main branch for 9.3.0 dev work. We can merge this PR so it will be released along with 9.3.0 packages in January 2026

if we merge these rules in your next release, would they get picked up on Serverless if we keep the min_stack_version set to 9.3.0, but be excluded from 9.2.0?

• To answer this, merging to main does not get the rules released. We do not ship packages from main branch of our repository. We only ship from respective protected branches like 8.18,8.19,9.0,9.1 and so on.
• Since the jobs are available in 9.3.0 in Kibana and we are min stacking this to 9.3.0 , the rules will be available in serverless also only in 9.3.0 packages in January 2026.