WIP: feat(metric alerts): Add tests for differences in workflow-engine-rule-serializers output

[kcons]

This aims to add tests to endpoints that use WorkflowEngineDetectorSerializer to compare to legacy behavior, reporting the diff.
It also includes some attempts to resolve these differences.

[sentry-warden]

DataSourceDetector.DoesNotExist not handled in serializer loop

The .get(Q(detector=detector)) call at line 344 will raise DataSourceDetector.DoesNotExist if a detector exists without a corresponding DataSourceDetector record. This can happen during asynchronous deletion, incomplete migration, or edge cases in detector creation. When this crashes, the entire alert rule list endpoint returns a 500 instead of gracefully handling the missing record.

Verification

Read the full serializer at workflow_engine_detector.py lines 334-364. Confirmed DataSourceDetector.get() is called without DoesNotExist handling. Checked DataSourceDetector model (data_source_detector.py) - it's a lookup table with ForeignKey to Detector but no cascade delete. Confirmed the serializer is used in organization_alert_rule_index.py where detectors are fetched independently from DataSourceDetector records.

Identified by Warden sentry-backend-bugs · XXT-YE6

[github-actions]

Backend Test Failures

Failures on ce3a933 in this run:

tests/sentry/incidents/serializers/test_workflow_engine_detector.py::TestDetectorSerializer::test_sentry_app — log

tests/sentry/incidents/serializers/test_workflow_engine_detector.py:191: in test_sentry_app
    assert self._sort_triggers(serialized_detector) == self._sort_triggers(sentry_app_expected)
E   AssertionError: assert {'aggregate':...'events', ...} == {'aggregate':...'events', ...}
E     
E     Omitting 26 identical items, use -vv to show
E     Differing items:
E     {'triggers': [{'actions': [{'alertRuleTriggerId': '337', 'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzi..., 'alertThreshold': 50, 'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc), ...}]} != {'triggers': [{'actions': [{'alertRuleTriggerId': '337', 'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzi..., 'alertThreshold': 50, 'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc), ...}]}
E     
E     Full diff:
E       {
E           'aggregate': 'count()',
E           'comparisonDelta': None,
E           'createdBy': None,
E           'dataset': 'events',
E           'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E           'dateModified': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E           'description': '',
E           'detectionType': 'static',
E           'environment': None,
E           'extrapolationMode': 'unknown',
E           'id': '353',
E           'name': 'Next Chipmunk',
E           'organizationId': '4555195392327680',
E           'originalAlertRuleId': None,
E           'owner': None,
E           'projects': [
E               'bar',
E           ],
E           'query': 'level:error',
E           'queryType': 0,
E           'resolution': 1.0,
E           'resolveThreshold': 50,
E           'seasonality': None,
E           'sensitivity': None,
E           'status': 0,
E           'thresholdPeriod': 1,
E           'thresholdType': 0,
E           'timeWindow': 10.0,
E           'triggers': [
E               {
E                   'actions': [
E                       {
E                           'alertRuleTriggerId': '337',
E                           'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E                           'desc': 'Send a notification to admin@localhost',
E                           'id': '461',
E     +                     'inputChannelId': None,
E     +                     'integrationId': None,
E     +                     'priority': None,
E     +                     'sentryAppId': None,
... (126 more lines)

tests/sentry/incidents/serializers/test_workflow_engine_detector.py::TestDetectorSerializer::test_simple — log

tests/sentry/incidents/serializers/test_workflow_engine_detector.py:43: in test_simple
    assert self._sort_triggers(serialized_detector) == self._sort_triggers(self.expected)
E   AssertionError: assert {'aggregate':...'events', ...} == {'aggregate':...'events', ...}
E     
E     Omitting 26 identical items, use -vv to show
E     Differing items:
E     {'triggers': [{'actions': [{'alertRuleTriggerId': '350', 'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzi..., 'alertThreshold': 50, 'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc), ...}]} != {'triggers': [{'actions': [{'alertRuleTriggerId': '350', 'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzi..., 'alertThreshold': 50, 'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc), ...}]}
E     
E     Full diff:
E       {
E           'aggregate': 'count()',
E           'comparisonDelta': None,
E           'createdBy': None,
E           'dataset': 'events',
E           'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E           'dateModified': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E           'description': '',
E           'detectionType': 'static',
E           'environment': None,
E           'extrapolationMode': 'unknown',
E           'id': '360',
E           'name': 'Calm Egret',
E           'organizationId': '4555195392327680',
E           'originalAlertRuleId': None,
E           'owner': None,
E           'projects': [
E               'bar',
E           ],
E           'query': 'level:error',
E           'queryType': 0,
E           'resolution': 1.0,
E           'resolveThreshold': 50,
E           'seasonality': None,
E           'sensitivity': None,
E           'status': 0,
E           'thresholdPeriod': 1,
E           'thresholdType': 0,
E           'timeWindow': 10.0,
E           'triggers': [
E               {
E                   'actions': [
E                       {
E                           'alertRuleTriggerId': '350',
E                           'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E                           'desc': 'Send a notification to admin@localhost',
E                           'id': '476',
E                           'inputChannelId': None,
E                           'integrationId': None,
E                           'priority': None,
E                           'sentryAppId': None,
... (52 more lines)

tests/sentry/incidents/serializers/test_workflow_engine_incident.py::TestIncidentSerializer::test_detailed — log

tests/sentry/incidents/serializers/test_workflow_engine_incident.py:68: in test_detailed
    assert self._sort_triggers(serialized_incident) == self._sort_triggers(
E   AssertionError: assert {'activities'...one.utc), ...} == {'activities'...one.utc), ...}
E     
E     Omitting 14 identical items, use -vv to show
E     Differing items:
E     {'alertRule': {'aggregate': 'count()', 'comparisonDelta': None, 'createdBy': None, 'dataset': 'events', ...}} != {'alertRule': {'aggregate': 'count()', 'comparisonDelta': None, 'createdBy': None, 'dataset': 'events', ...}}
E     
E     Full diff:
E       {
E           'activities': None,
E           'alertRule': {
E               'aggregate': 'count()',
E               'comparisonDelta': None,
E               'createdBy': None,
E               'dataset': 'events',
E               'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E               'dateModified': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E               'description': '',
E               'detectionType': 'static',
E               'environment': None,
E               'extrapolationMode': 'unknown',
E               'id': '366',
E               'name': 'Glad Ant',
E               'organizationId': '4555195392327680',
E               'originalAlertRuleId': None,
E               'owner': None,
E               'projects': [
E                   'bar',
E               ],
E               'query': 'level:error',
E               'queryType': 0,
E               'resolution': 1.0,
E               'resolveThreshold': 50,
E               'seasonality': None,
E               'sensitivity': None,
E               'status': 0,
E               'thresholdPeriod': 1,
E               'thresholdType': 0,
E               'timeWindow': 10.0,
E               'triggers': [
E                   {
E                       'actions': [
E                           {
E                               'alertRuleTriggerId': '362',
E                               'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E                               'desc': 'Send a notification to admin@localhost',
E                               'id': '488',
E                               'inputChannelId': None,
E                               'integrationId': None,
... (70 more lines)

tests/sentry/incidents/serializers/test_workflow_engine_incident.py::TestIncidentSerializer::test_no_incident — log

tests/sentry/incidents/serializers/test_workflow_engine_incident.py:119: in test_no_incident
    assert self._sort_triggers(serialized_incident) == self._sort_triggers(
E   AssertionError: assert {'activities'...one.utc), ...} == {'activities'...one.utc), ...}
E     
E     Omitting 13 identical items, use -vv to show
E     Differing items:
E     {'alertRule': {'aggregate': 'count()', 'comparisonDelta': None, 'createdBy': None, 'dataset': 'events', ...}} != {'alertRule': {'aggregate': 'count()', 'comparisonDelta': None, 'createdBy': None, 'dataset': 'events', ...}}
E     
E     Full diff:
E       {
E           'activities': None,
E           'alertRule': {
E               'aggregate': 'count()',
E               'comparisonDelta': None,
E               'createdBy': None,
E               'dataset': 'events',
E               'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E               'dateModified': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E               'description': '',
E               'detectionType': 'static',
E               'environment': None,
E               'extrapolationMode': 'unknown',
E               'id': '10000000843',
E               'name': 'Witty Grouper',
E               'organizationId': '4555195392327680',
E               'originalAlertRuleId': None,
E               'owner': None,
E               'projects': [
E                   'bar',
E               ],
E               'query': 'level:error',
E               'queryType': 0,
E               'resolution': 1.0,
E               'resolveThreshold': 50,
E               'seasonality': None,
E               'sensitivity': None,
E               'status': 0,
E               'thresholdPeriod': 1,
E               'thresholdType': 0,
E               'timeWindow': 10.0,
E               'triggers': [
E                   {
E                       'actions': [
E                           {
E                               'alertRuleTriggerId': '10000001281',
E                               'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E                               'desc': 'Send a notification to admin@localhost',
E                               'id': '10000000552',
E                               'inputChannelId': None,
E                               'integrationId': None,
... (69 more lines)

tests/sentry/incidents/serializers/test_workflow_engine_incident.py::TestIncidentSerializer::test_simple — log

tests/sentry/incidents/serializers/test_workflow_engine_incident.py:59: in test_simple
    assert self._sort_triggers(serialized_incident) == self._sort_triggers(
E   AssertionError: assert {'activities'...one.utc), ...} == {'activities'...one.utc), ...}
E     
E     Omitting 13 identical items, use -vv to show
E     Differing items:
E     {'alertRule': {'aggregate': 'count()', 'comparisonDelta': None, 'createdBy': None, 'dataset': 'events', ...}} != {'alertRule': {'aggregate': 'count()', 'comparisonDelta': None, 'createdBy': None, 'dataset': 'events', ...}}
E     
E     Full diff:
E       {
E           'activities': None,
E           'alertRule': {
E               'aggregate': 'count()',
E               'comparisonDelta': None,
E               'createdBy': None,
E               'dataset': 'events',
E               'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E               'dateModified': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E               'description': '',
E               'detectionType': 'static',
E               'environment': None,
E               'extrapolationMode': 'unknown',
E               'id': '378',
E               'name': 'Robust Chipmunk',
E               'organizationId': '4555195392327680',
E               'originalAlertRuleId': None,
E               'owner': None,
E               'projects': [
E                   'bar',
E               ],
E               'query': 'level:error',
E               'queryType': 0,
E               'resolution': 1.0,
E               'resolveThreshold': 50,
E               'seasonality': None,
E               'sensitivity': None,
E               'status': 0,
E               'thresholdPeriod': 1,
E               'thresholdType': 0,
E               'timeWindow': 10.0,
E               'triggers': [
E                   {
E                       'actions': [
E                           {
E                               'alertRuleTriggerId': '386',
E                               'dateCreated': datetime.datetime(2024, 12, 11, 3, 21, 34, tzinfo=datetime.timezone.utc),
E                               'desc': 'Send a notification to admin@localhost',
E                               'id': '512',
E                               'inputChannelId': None,
E                               'integrationId': None,
... (69 more lines)