[Automatic Migrations] Add support for Qradar Sequence and toggle for its translation capability

[logeekal]

Summary

This PR adds ability for Rule Migration graph to understand Sequence, DoubleSequence and CauseAndEffectfunction.

Below we can see the natural language description and ESQL translation summary of the Temporal Correlation sequences. As can be clearly seen and after communicating with ESQL Team, we can confirm that Sequences are not supported by ESQL. Because of this, we have also added a toggle function to add/remove support for QRadar function.

Natural description of the Sequence

ESQL Translation

Currently, ESQL responds appropriately that it cannot do Sequence Correlations. This is the expected output.

Comment without Sequence Support

Since we know that Sequences are not supported by ESQL and as it was confirmed with ESQL Team as well, we have add Sequence to unsupported list of functions so that we do not waste Tokens doing the translation. When we have the capability, we can simply enable it.

More Improvements

Better ESQL Query Translation Summary

• Shows how much of the query can/cannot be translated.
• Shows the reasons for which some part of the query cannot be translated.
• Introduces a minimum threshold ( 20% ) below which query is marked Non Translated citing the reasons.

Screenshot

Inclusion Integrations indices field mapping

As can be seen in below screenshot, field mappings are now included in the translation node which results in better query. Screenshot of this trace is included below:

Screenshot

This inevitably results in better query. See before/after screenshots below:

Without Index mappings	With index mappings

Desk Testing Plan

Testing the New Feature

Sequence Rule Migrations without pre-built rule match

• Run Qradar Rule Migrations with this sequences export
• Expectation : Rule should be Not Translated and explanation in the Summary section as to why.

Sequence Rule Migrations with pre-built rule match

• Run Qradar Rule Migrations with this sequences export
• Expectation :
  • Rule should be Not Translated and explanation as to why as shown above.
  • A pre-built rule should be a match.

Regression Testing

• Rule QRadar Rule Migrations export without sequences
• It should run successfully with integrations matches for Netflow or Network packet capture or Oracle with corresponding indices and correct columns.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
• Flaky Test Runner was used on any tests changed
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

• See some risk examples
• ...

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[jonwalstedt]

Suggested change

	#### Key Commands ( including the negate attribute handling):
	#### Key Commands (including the negate attribute handling):

[jonwalstedt]

@logeekal It looks like the links to the xmls for the Qradar rule migrations (the one without pre-built rule match and with pre-built rule match) are the same file, both links point to the same file