feat: Supports dynamic blocks in regions_config (#33)

* doc assumption for dynamic blocks in tags and labels

* dynamic_regions_config example

* range for priorities

* allow dynamic block in regions_config

* doc

* update comment

* minimum implementation to have test failing because difference in golden file

* export enclose funcs

* create EncloseNewLines and remove SetAttrExpr

* root replication_specs

* remove priority checks about numerical literal

* reuse getRegionConfig from dynamic block logic

* only sort by priority if all priorities are numerical literals

* remove limitations for priority and electable_nodes

* use config in dynamic blocks from individual

* passing test

* add auto_scaling example

* fix region_configs name replacement

* refactor isDynamicBlock

* go back to unexported tokenNewLine

* add analytics specs

* example in readme

* clarify num_shards limitation

* feedback section

* getDynamicBlockRegionConfigsRegionArray

* refactor fillRegionConfigsDynamicBlock

* EncloseBracketsNewLines

* fillRegionConfigsDynamicBlock doc

* move shards closer to where it's used

* add comment for priority loop

* add dynamic block doc

* small doc adjustment

* rename to fillReplicationSpecsWithDynamicRegionConfigs

* Update README.md

Co-authored-by: Marco Suma <[email protected]>

* link to limitations

* how to handle limitation

---------

Co-authored-by: Marco Suma <[email protected]>

Author: lantoli

CHANGELOG.md

@@ -2,7 +2,7 @@
 
 ENHANCEMENTS:
 
-* Supports `dynamic` block for `tags` and `labels`
+* Supports `dynamic` block for `tags`, `labels` and `regions_config`
 
 ## 1.0.0 (Mar 6, 2025)
 

README.md

@@ -57,7 +57,8 @@ Given the different ways of using dynamic blocks, we recommend reviewing the out
 
 #### Dynamic blocks in tags and labels
 
-You can use `dynamic` blocks for `tags` and `labels`. You can also combine the use of dynamic blocks in `tags` and `labels` with individual blocks in the same cluster definition, e.g.:
+You can use `dynamic` blocks for `tags` and `labels`. The plugin assumes that `for_each` has an expression which is evaluated to a `map` of strings.
+You can also combine the use of dynamic blocks in `tags` and `labels` with individual blocks in the same cluster definition, e.g.:
 ```hcl
 tags {
 	key   = "environment"
@@ -72,12 +73,37 @@ dynamic "tags" {
 }
 ```
 
+#### Dynamic blocks in regions_config
+
+You can use `dynamic` blocks for `regions_config`. The plugin assumes that `for_each` has an expression which is evaluated to a `list` or `set` of objects.
+This is an example of how to use dynamic blocks in `regions_config`:
+```hcl
+  replication_specs {
+    num_shards = var.replication_specs.num_shards
+    zone_name  = var.replication_specs.zone_name # only needed if you're using zones
+    dynamic "regions_config" {
+      for_each = var.replication_specs.regions_config
+      content {
+        priority        = regions_config.value.priority
+        region_name     = regions_config.value.region_name
+        electable_nodes = regions_config.value.electable_nodes
+        read_only_nodes = regions_config.value.read_only_nodes
+      }
+    }
+  }
+```
+Dynamic block and individual blocks for `regions_config` are not supported at the same time. If you need this use case, please send us [feedback](https://github.com/mongodb-labs/atlas-cli-plugin-terraform/issues). There are currently two main approaches to handle this:
+- (Recommended) Remove the individual `regions_config` blocks and add their information to the variable you're using in the `for_each` expression, e.g. using [concat](https://developer.hashicorp.com/terraform/language/functions/concat) if you're using a list or [setunion](https://developer.hashicorp.com/terraform/language/functions/setunion) for sets. In this way, you don't need to change the generated `mongodb_advanced_cluster` configuration.
+- Change the generated `mongodb_advanced_cluster` configuration to join the individual blocks to the code generated for the `dynamic` block. This approach is more error-prone.
+
 ### Limitations
 
-- The plugin doesn't support `regions_config` without `electable_nodes` as there can be some issues with `priority` when they only have `analytics_nodes` and/or `electable_nodes`.
-- [`priority`](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/cluster#priority-1) is required in `regions_config` and must be a numeric [literal expression](https://developer.hashicorp.com/nomad/docs/job-specification/hcl2/expressions#literal-expressions) between 7 and 1, e.g. `var.priority` is not supported. This is to allow reordering them by descending priority as this is expected in `mongodbatlas_advanced_cluster`.
-- [`num_shards`](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/cluster#num_shards-2) in `replication_specs` must be a numeric [literal expression](https://developer.hashicorp.com/nomad/docs/job-specification/hcl2/expressions#literal-expressions), e.g. `var.num_shards` is not supported. This is to allow creating a `replication_specs` element per shard in `mongodbatlas_advanced_cluster`.
-- `dynamic` blocks are currently supported only for `tags` and `labels`. **Coming soon**: support for `replication_specs` and `regions_config`.
+- [`num_shards`](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/cluster#num_shards-2) in `replication_specs` must be a numeric [literal expression](https://developer.hashicorp.com/nomad/docs/job-specification/hcl2/expressions#literal-expressions), e.g. `var.num_shards` is not supported. This is to allow creating a `replication_specs` element per shard in `mongodbatlas_advanced_cluster`. This limitation doesn't apply if you're using `dynamic` blocks in `regions_config` or `replication_specs`.
+- `dynamic` blocks are currently supported only for `tags`, `labels` and `regions_config`. See limitations for `regions_config` support in [its section](#dynamic-blocks-in-regions_config) above. **Coming soon**: support for `replication_specs`.
+
+## Feedback
+
+If you find any issues or have any suggestions, please open an [issue](https://github.com/mongodb-labs/atlas-cli-plugin-terraform/issues) in this repository.
 
 ## Contributing
 

internal/convert/const_names.go

@@ -54,4 +54,5 @@ const (
 	nDynamic                    = "dynamic"
 	nForEach                    = "for_each"
 	nContent                    = "content"
+	nRegion                     = "region"
 )

internal/convert/convert.go

@@ -22,22 +22,22 @@ const (
 	advClusterPlural = "mongodbatlas_advanced_clusters"
 	valClusterType   = "REPLICASET"
 	valMaxPriority   = 7
-	valMinPriority   = 1
-
-	errFreeCluster = "free cluster (because no " + nRepSpecs + ")"
-	errRepSpecs    = "setting " + nRepSpecs
-	errConfigs     = "setting " + nConfig
-	errPriority    = "setting " + nPriority
-	errNumShards   = "setting " + nNumShards
+	valMinPriority   = 0
+	errFreeCluster   = "free cluster (because no " + nRepSpecs + ")"
+	errRepSpecs      = "setting " + nRepSpecs
+	errConfigs       = "setting " + nConfig
+	errPriority      = "setting " + nPriority
+	errNumShards     = "setting " + nNumShards
 
 	commentGeneratedBy       = "Generated by atlas-cli-plugin-terraform."
-	commentConfirmReferences = "Please confirm that all references to this resource are updated."
+	commentConfirmReferences = "Please review the changes and confirm that references to this resource are updated."
 	commentMovedBlock        = "Moved blocks"
 	commentRemovedOld        = "Note: Remember to remove or comment out the old cluster definitions."
+	commentPriorityFor       = "Regions must be sorted by priority in descending order."
 )
 
 var (
-	dynamicBlockAllowList = []string{nTags, nLabels}
+	dynamicBlockAllowList = []string{nTags, nLabels, nConfigSrc}
 )
 
 type attrVals struct {
@@ -129,8 +129,8 @@ func fillMovedBlocks(body *hclwrite.Body, moveLabels []string) {
 	for i, moveLabel := range moveLabels {
 		block := body.AppendNewBlock(nMoved, nil)
 		blockb := block.Body()
-		hcl.SetAttrExpr(blockb, nFrom, fmt.Sprintf("%s.%s", cluster, moveLabel))
-		hcl.SetAttrExpr(blockb, nTo, fmt.Sprintf("%s.%s", advCluster, moveLabel))
+		blockb.SetAttributeRaw(nFrom, hcl.TokensFromExpr(fmt.Sprintf("%s.%s", cluster, moveLabel)))
+		blockb.SetAttributeRaw(nTo, hcl.TokensFromExpr(fmt.Sprintf("%s.%s", advCluster, moveLabel)))
 		if i < len(moveLabels)-1 {
 			body.AppendNewline()
 		}
@@ -202,9 +202,15 @@ func fillReplicationSpecs(resourceb *hclwrite.Body, root attrVals) error {
 			break
 		}
 		specbSrc := specSrc.Body()
-		if err := checkDynamicBlock(specbSrc); err != nil {
+		d, err := fillReplicationSpecsWithDynamicRegionConfigs(specbSrc, root)
+		if err != nil {
 			return err
 		}
+		if d.IsPresent() {
+			resourceb.RemoveBlock(specSrc)
+			resourceb.SetAttributeRaw(nRepSpecs, d.tokens)
+			return nil
+		}
 		// ok to fail as zone_name is optional
 		_ = hcl.MoveAttr(specbSrc, specb, nZoneName, nZoneName, errRepSpecs)
 		shards := specbSrc.GetAttribute(nNumShards)
@@ -251,7 +257,7 @@ func fillTagsLabelsOpt(resourceb *hclwrite.Body, name string) error {
 
 func extractTagsLabelsDynamicBlock(resourceb *hclwrite.Body, name string) (hclwrite.Tokens, error) {
 	d, err := getDynamicBlock(resourceb, name)
-	if err != nil || d.forEach == nil {
+	if err != nil || !d.IsPresent() {
 		return nil, err
 	}
 	key := d.content.Body().GetAttribute(nKey)
@@ -306,14 +312,44 @@ func fillBlockOpt(resourceb *hclwrite.Body, name string) {
 	resourceb.SetAttributeRaw(name, hcl.TokensObject(block.Body()))
 }
 
+// fillReplicationSpecsWithDynamicRegionConfigs is used for dynamic blocks in region_configs
+func fillReplicationSpecsWithDynamicRegionConfigs(specbSrc *hclwrite.Body, root attrVals) (dynamicBlock, error) {
+	d, err := getDynamicBlock(specbSrc, nConfigSrc)
+	if err != nil || !d.IsPresent() {
+		return dynamicBlock{}, err
+	}
+	repSpec := hclwrite.NewEmptyFile()
+	repSpecb := repSpec.Body()
+	if zoneName := hcl.GetAttrExpr(specbSrc.GetAttribute(nZoneName)); zoneName != "" {
+		repSpecb.SetAttributeRaw(nZoneName, hcl.TokensFromExpr(zoneName))
+	}
+	regionFor, err := getDynamicBlockRegionConfigsRegionArray(d, root)
+	if err != nil {
+		return dynamicBlock{}, err
+	}
+	priorityFor := hcl.TokensComment(commentPriorityFor)
+	priorityFor = append(priorityFor, hcl.TokensFromExpr(fmt.Sprintf("for %s in range(%d, %d, -1) : ", nPriority, valMaxPriority, valMinPriority))...)
+	priorityFor = append(priorityFor, regionFor...)
+	repSpecb.SetAttributeRaw(nConfig, hcl.TokensFuncFlatten(priorityFor))
+
+	shards := specbSrc.GetAttribute(nNumShards)
+	if shards == nil {
+		return dynamicBlock{}, fmt.Errorf("%s: %s not found", errRepSpecs, nNumShards)
+	}
+	tokens := hcl.TokensFromExpr(fmt.Sprintf("for i in range(%s) :", hcl.GetAttrExpr(shards)))
+	tokens = append(tokens, hcl.EncloseBraces(repSpec.BuildTokens(nil), true)...)
+	d.tokens = hcl.EncloseBracketsNewLines(tokens)
+	return d, nil
+}
+
 func fillRegionConfigs(specb, specbSrc *hclwrite.Body, root attrVals) error {
 	var configs []*hclwrite.Body
 	for {
 		configSrc := specbSrc.FirstMatchingBlock(nConfigSrc, nil)
 		if configSrc == nil {
 			break
 		}
-		config, err := getRegionConfig(configSrc, root)
+		config, err := getRegionConfig(configSrc, root, false)
 		if err != nil {
 			return err
 		}
@@ -323,34 +359,28 @@ func fillRegionConfigs(specb, specbSrc *hclwrite.Body, root attrVals) error {
 	if len(configs) == 0 {
 		return fmt.Errorf("%s: %s not found", errRepSpecs, nConfigSrc)
 	}
-	sort.Slice(configs, func(i, j int) bool {
-		pi, _ := hcl.GetAttrInt(configs[i].GetAttribute(nPriority), errPriority)
-		pj, _ := hcl.GetAttrInt(configs[j].GetAttribute(nPriority), errPriority)
-		return pi > pj
-	})
+	configs = sortConfigsByPriority(configs)
 	specb.SetAttributeRaw(nConfig, hcl.TokensArray(configs))
 	return nil
 }
 
-func getRegionConfig(configSrc *hclwrite.Block, root attrVals) (*hclwrite.File, error) {
+func getRegionConfig(configSrc *hclwrite.Block, root attrVals, isDynamicBlock bool) (*hclwrite.File, error) {
 	file := hclwrite.NewEmptyFile()
 	fileb := file.Body()
 	fileb.SetAttributeRaw(nProviderName, root.req[nProviderName])
 	if err := hcl.MoveAttr(configSrc.Body(), fileb, nRegionName, nRegionName, errRepSpecs); err != nil {
 		return nil, err
 	}
-	if err := setPriority(fileb, configSrc.Body().GetAttribute(nPriority)); err != nil {
+	if err := hcl.MoveAttr(configSrc.Body(), fileb, nPriority, nPriority, errRepSpecs); err != nil {
 		return nil, err
 	}
-	electableSpecs, errElec := getSpecs(configSrc, nElectableNodes, root)
-	if errElec != nil {
-		return nil, errElec
+	if electable, _ := getSpecs(configSrc, nElectableNodes, root, isDynamicBlock); electable != nil {
+		fileb.SetAttributeRaw(nElectableSpecs, electable)
 	}
-	fileb.SetAttributeRaw(nElectableSpecs, electableSpecs)
-	if readOnly, _ := getSpecs(configSrc, nReadOnlyNodes, root); readOnly != nil {
+	if readOnly, _ := getSpecs(configSrc, nReadOnlyNodes, root, isDynamicBlock); readOnly != nil {
 		fileb.SetAttributeRaw(nReadOnlySpecs, readOnly)
 	}
-	if analytics, _ := getSpecs(configSrc, nAnalyticsNodes, root); analytics != nil {
+	if analytics, _ := getSpecs(configSrc, nAnalyticsNodes, root, isDynamicBlock); analytics != nil {
 		fileb.SetAttributeRaw(nAnalyticsSpecs, analytics)
 	}
 	if autoScaling := getAutoScalingOpt(root.opt); autoScaling != nil {
@@ -359,7 +389,7 @@ func getRegionConfig(configSrc *hclwrite.Block, root attrVals) (*hclwrite.File,
 	return file, nil
 }
 
-func getSpecs(configSrc *hclwrite.Block, countName string, root attrVals) (hclwrite.Tokens, error) {
+func getSpecs(configSrc *hclwrite.Block, countName string, root attrVals, isDynamicBlock bool) (hclwrite.Tokens, error) {
 	var (
 		file  = hclwrite.NewEmptyFile()
 		fileb = file.Body()
@@ -382,7 +412,11 @@ func getSpecs(configSrc *hclwrite.Block, countName string, root attrVals) (hclwr
 	if root.opt[nDiskIOPSSrc] != nil {
 		fileb.SetAttributeRaw(nDiskIOPS, root.opt[nDiskIOPSSrc])
 	}
-	return hcl.TokensObject(fileb), nil
+	tokens := hcl.TokensObject(fileb)
+	if isDynamicBlock {
+		tokens = encloseDynamicBlockRegionSpec(tokens, countName)
+	}
+	return tokens, nil
 }
 
 func getAutoScalingOpt(opt map[string]hclwrite.Tokens) hclwrite.Tokens {
@@ -440,6 +474,17 @@ func getResourceLabel(resource *hclwrite.Block) string {
 	return labels[1]
 }
 
+type dynamicBlock struct {
+	block   *hclwrite.Block
+	forEach *hclwrite.Attribute
+	content *hclwrite.Block
+	tokens  hclwrite.Tokens
+}
+
+func (d dynamicBlock) IsPresent() bool {
+	return d.block != nil
+}
+
 func checkDynamicBlock(body *hclwrite.Body) error {
 	for _, block := range body.Blocks() {
 		name := getResourceName(block)
@@ -451,12 +496,6 @@ func checkDynamicBlock(body *hclwrite.Body) error {
 	return nil
 }
 
-type dynamicBlock struct {
-	block   *hclwrite.Block
-	forEach *hclwrite.Attribute
-	content *hclwrite.Block
-}
-
 func getDynamicBlock(body *hclwrite.Body, name string) (dynamicBlock, error) {
 	for _, block := range body.Blocks() {
 		if block.Type() != nDynamic || name != getResourceName(block) {
@@ -481,6 +520,55 @@ func replaceDynamicBlockExpr(attr *hclwrite.Attribute, blockName, attrName strin
 	return strings.ReplaceAll(expr, fmt.Sprintf("%s.%s", blockName, attrName), attrName)
 }
 
+func encloseDynamicBlockRegionSpec(specTokens hclwrite.Tokens, countName string) hclwrite.Tokens {
+	tokens := hcl.TokensFromExpr(fmt.Sprintf("%s.%s > 0 ?", nRegion, countName))
+	tokens = append(tokens, specTokens...)
+	return append(tokens, hcl.TokensFromExpr(": null")...)
+}
+
+// getDynamicBlockRegionConfigsRegionArray returns the region array for a dynamic block in replication_specs.
+// e.g. [ for region in var.replication_specs.regions_config : { ... } if priority == region.priority ]
+func getDynamicBlockRegionConfigsRegionArray(d dynamicBlock, root attrVals) (hclwrite.Tokens, error) {
+	transformDynamicBlockReferences(d.content.Body())
+	priorityStr := hcl.GetAttrExpr(d.content.Body().GetAttribute(nPriority))
+	if priorityStr == "" {
+		return nil, fmt.Errorf("%s: %s not found", errRepSpecs, nPriority)
+	}
+	region, err := getRegionConfig(d.content, root, true)
+	if err != nil {
+		return nil, err
+	}
+	tokens := hcl.TokensFromExpr(fmt.Sprintf("for %s in %s :", nRegion, hcl.GetAttrExpr(d.forEach)))
+	tokens = append(tokens, hcl.EncloseBraces(region.BuildTokens(nil), true)...)
+	tokens = append(tokens, hcl.TokensFromExpr(fmt.Sprintf("if %s == %s", nPriority, priorityStr))...)
+	return hcl.EncloseBracketsNewLines(tokens), nil
+}
+
+// transformDynamicBlockReferences changes value references in all attributes, e.g. regions_config.value.electable_nodes to region.electable_nodes
+func transformDynamicBlockReferences(configSrcb *hclwrite.Body) {
+	for name, attr := range configSrcb.Attributes() {
+		expr := hcl.GetAttrExpr(attr)
+		expr = strings.ReplaceAll(expr,
+			fmt.Sprintf("%s.%s.", nConfigSrc, nValue),
+			fmt.Sprintf("%s.", nRegion))
+		configSrcb.SetAttributeRaw(name, hcl.TokensFromExpr(expr))
+	}
+}
+
+func sortConfigsByPriority(configs []*hclwrite.Body) []*hclwrite.Body {
+	for _, config := range configs {
+		if _, err := hcl.GetAttrInt(config.GetAttribute(nPriority), errPriority); err != nil {
+			return configs // don't sort priorities if any is not a numerical literal
+		}
+	}
+	sort.Slice(configs, func(i, j int) bool {
+		pi, _ := hcl.GetAttrInt(configs[i].GetAttribute(nPriority), errPriority)
+		pj, _ := hcl.GetAttrInt(configs[j].GetAttribute(nPriority), errPriority)
+		return pi > pj
+	})
+	return configs
+}
+
 func setKeyValue(body *hclwrite.Body, key, value *hclwrite.Attribute) {
 	keyStr, err := hcl.GetAttrString(key, "")
 	if err == nil {
@@ -494,21 +582,6 @@ func setKeyValue(body *hclwrite.Body, key, value *hclwrite.Attribute) {
 	body.SetAttributeRaw(keyStr, value.Expr().BuildTokens(nil))
 }
 
-func setPriority(body *hclwrite.Body, priority *hclwrite.Attribute) error {
-	if priority == nil {
-		return fmt.Errorf("%s: %s not found", errRepSpecs, nPriority)
-	}
-	valPriority, err := hcl.GetAttrInt(priority, errPriority)
-	if err != nil {
-		return err
-	}
-	if valPriority < valMinPriority || valPriority > valMaxPriority {
-		return fmt.Errorf("%s: %s is %d but must be between %d and %d", errPriority, nPriority, valPriority, valMinPriority, valMaxPriority)
-	}
-	hcl.SetAttrInt(body, nPriority, valPriority)
-	return nil
-}
-
 // popRootAttrs deletes the attributes common to all replication_specs/regions_config and returns them.
 func popRootAttrs(body *hclwrite.Body) (attrVals, error) {
 	var (

internal/convert/testdata/clu2adv/adv_config_bi_connector_pinned_fcv.out.tf

@@ -32,5 +32,5 @@ resource "mongodbatlas_advanced_cluster" "this" {
   }
 
   # Generated by atlas-cli-plugin-terraform.
-  # Please confirm that all references to this resource are updated.
+  # Please review the changes and confirm that references to this resource are updated.
 }

internal/convert/testdata/clu2adv/analytics_read_only_all_params.in.tf

@@ -18,3 +18,24 @@ resource "mongodbatlas_cluster" "ar" {
     }
   }
 }
+
+resource "mongodbatlas_cluster" "ar_not_electable" {
+  project_id                  = var.project_id
+  name                        = "ar"
+  cluster_type                = "REPLICASET"
+  provider_name               = "AWS"
+  provider_instance_size_name = "M10"
+  disk_size_gb                = 90
+  provider_volume_type        = "PROVISIONED"
+  provider_disk_iops          = 100
+  replication_specs {
+    num_shards = 1
+    regions_config {
+      region_name     = "US_EAST_1"
+      priority        = 7
+      electable_nodes = 0
+      analytics_nodes = 2
+      read_only_nodes = 1
+    }
+  }
+}