Skip to content

Conversation

lucasrenan
Copy link

@lucasrenan lucasrenan commented Aug 22, 2025

Issue

EMR Managed Scaling adjusts Task fleet capacity units as the cluster is running.

Terraform later detects drift and forces them back to the originally declared low target_on_demand_capacity / target_spot_capacity, triggering unintended scale downs during terraform applies.

Root causes

  • EMR Managed Scaling mutates live capacity.
  • Terraform treats initial targets as authoritative desired state.
  • Reconciling causes shrink.
  • lifecycle.ignore_changes must be a static list (cannot be conditionally used).

Workaround

Provide two mutually exclusive aws_emr_instance_fleet resources.
One with lifecycle.ignore_changes on the two capacity attributes selected via a boolean variable. When the ignore variant is active, EMR owns scaling and Terraform stops planning destructive reductions.


This solution duplicates code and isn't elegant. Any other ideas?

@lucasrenan lucasrenan changed the title Add ignore_task_fleet_capacity_drifts to task_instance_fleet feat: Add ignore_task_fleet_capacity_drifts to task_instance_fleet Aug 22, 2025
@lucasrenan lucasrenan requested a review from bryantbiggs August 22, 2025 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants