[fix] Preserve actions[].params.message field formatting during rule export from the repo

[traut]

Pull Request

Issue link(s):

• [Bug] [DAC] Rule Action Email Strings Incorrectly Handled #5529
• [Meta] Refactor Rule Formatter #3558

Summary - What I changed

Field message is added to the list of fields excluded from string normalization.

How To Test

• unit tests pass
• manual testing confirmed that the line breaks are preserved during TOML->NDJSON export:

$ jq '.actions[].params.message' exports/20260122T103300L.ndjson
"Rule {{context.rule.name}} generated {{state.signals_count}} alerts\n\naaa\n-\nbbb\n-\nccc"

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[github-actions]

Bug - Guidelines

These guidelines serve as a reminder set of considerations when addressing a bug in the code.

Documentation and Context

• Provide detailed documentation (description, screenshots, reproducing the bug, etc.) of the bug if not already documented in an issue.
• Include additional context or details about the problem.
• Ensure the fix includes necessary updates to the release documentation and versioning.

Code Standards and Practices

• Code follows established design patterns within the repo and avoids duplication.
• Ensure that the code is modular and reusable where applicable.

Testing

• New unit tests have been added to cover the bug fix or edge cases.
• Existing unit tests have been updated to reflect the changes.
• Provide evidence of testing and detecting the bug fix (e.g., test logs, screenshots).
• Validate that any rules affected by the bug are correctly updated.
• Ensure that performance is not negatively impacted by the changes.
• Verify that any release artifacts are properly generated and tested.
• Conducted system testing, including fleet, import, and create APIs (e.g., run make test-cli, make test-remote-cli, make test-hunting-cli)

Additional Checks

• Verify that the bug fix works across all relevant environments (e.g., different OS versions).
• Confirm that the proper version label is applied to the PR patch, minor, major.

[eric-forte-elastic]

This fix address the immediate bug and given that multiple users have reported this, we should consider this a priority to fix and prioritize a fix being merged compared to the fix being the best fix. Given this, I approve 🟢 👍

However, for posterity it should be noted that I do not see investigation as to why this is occurring, so we should expect that this may be occurring in other fields as well erroneously.

See testing details comment for more details (since Github no longer allows details in review submission). Screenshot below is a before and after using the test criteria I outlined in the issue write up.

[traut]

@eric-forte-elastic thanks for the review!

The root cause here if our business logic: string values go through cleanup_whitespace if the field is not in a set returned by get_preserved_fmt_fields(), but get_preserved_fmt_fields only considers the fields on the root level of BaseRuleData class. This logic does not accommodate nested fields (like message here) -- settings that field as Markdown, for example, would not fix the behavior.

[eric-forte-elastic]

@eric-forte-elastic thanks for the review!

The root cause here if our business logic: string values go through cleanup_whitespace if the field is not in a set returned by get_preserved_fmt_fields(), but get_preserved_fmt_fields only considers the fields on the root level of BaseRuleData class. This logic does not accommodate nested fields (like message here) -- settings that field as Markdown, for example, would not fix the behavior.

100% that part makes sense and I think your implementation is clean and effective to address that. More of the posterity note is in reference to it being unclear as to why we have cleanup_whitespace and get_preserved_fmt_fields and what those are needed to accomplish, which may or may not be removable tech debt.

[eric-forte-elastic]

Testing details

Details

python -m detection_rules export-rules-from-repo -f rules/test_email.toml -o test_email_action_rule.ndjson

test_email_action_rule.ndjson.txt

test_email.toml.txt