Skip to content

Conversation

@peachprosecutor
Copy link

@peachprosecutor peachprosecutor commented Jan 5, 2026

Summary

Fixes an issue where syncing metric-based SLOs would fail due to conflicting fields being sent to the Datadog API.

Problem

The Datadog SLO GET endpoint returns both sli_specification and query fields for metric SLOs. However, the CREATE and UPDATE endpoints only accept one of these fields—not both. When the sync CLI attempted to create or update a metric SLO, the API would reject the request because both fields were present.

Solution

Exclude query from both the diff and outgoing payloads to the API via the excluded_attributes param to avoid conflict errors.

Changes

Aforementioned excluded_attributes update.

Consequences

Users will need the latest version of the datadog-sync-cli tool once the FF is enabled.
Conversely, if the FF is off and the tool is up to date with these changes, then the tool breaks because query will be excluded from payloads and it is a required field.

I tested this by using the datadog-sync-cli locally:

Adding query to excluded attrbitues and running against API w/ FF OFF

- ERROR - [service_level_objectives - c593197d245a56bc825c3efe8040d2fe] - 400 Bad Request - {"errors":["Invalid payload: must specify the query for count types"]}

Running with FF on in the source org but not the destination org:

- 2026-01-06 12:44:55,556 - ERROR - [service_level_objectives - c593197d245a56bc825c3efe8040d2fe] - 400 Bad Request - {"errors":["Invalid payload: Additional properties are not allowed ('count' was unexpected)"]}

Running again with FF on, but with no change to underlying SLO

2026-01-06 12:50:38,051 - INFO - Using local filesystem to store state files
2026-01-06 12:50:38,291 - INFO - clients validated successfully
2026-01-06 12:50:38,539 - INFO - Starting sync...
2026-01-06 12:50:38,539 - INFO - did not import missing dependencies...
2026-01-06 12:50:38,672 - INFO - Global downtime for datadog-sync-cli already exists
2026-01-06 12:50:38,695 - INFO - [service_level_objectives - c593197d245a56bc825c3efe8040d2fe] - skipping resource: Skipping service_level_objectives with id: c593197d245a56bc825c3efe8040d2fe. No differences detected.

Running again with FF on, but with changes to underlying SLO sli_specification:

2026-01-06 12:54:51,575 - INFO - Using local filesystem to store state files
2026-01-06 12:54:51,834 - INFO - clients validated successfully
2026-01-06 12:54:51,834 - WARNING - DDR verification skipped.
2026-01-06 12:54:52,055 - INFO - Starting sync...
2026-01-06 12:54:52,055 - INFO - did not import missing dependencies...
2026-01-06 12:54:52,199 - INFO - Global downtime for datadog-sync-cli already exists
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.44it/s]
2026-01-06 12:54:52,625 - INFO - finished syncing resource items: Successes: 1, Failures: 0, Skipped: 0, Filtered: 0.
2026-01-06 12:54:52,627 - INFO - Finished sync

@peachprosecutor peachprosecutor changed the title fix sync cli for metric SLOs Fix sync cli for metric SLOs Jan 5, 2026
return _id, resp["data"][0]

@staticmethod
def _remove_conflicting_slo_fields(resource: Dict) -> None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only being called for the destination resources. So the source SLO will have both fields and the destination will only have sli_specification meaning diffs will show a difference and try to update it every time this is run, and will make the update call. We can't just ignore the query field like we can other fields either because some SLOs will only have query params right? Or can we just always remove the query from every SLO?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was a bit confused by how this worked initially but I see now how excluded_attributes works (field is excluded from the diff as well as the outgoing destination payload)-- I updated my branch to just add query to the excluded_attributes and tested w/ and w/out the FF enabled, and everything worked as expected.

@peachprosecutor peachprosecutor marked this pull request as ready for review January 6, 2026 18:16
@peachprosecutor peachprosecutor requested a review from a team as a code owner January 6, 2026 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants