Skip to content

bug: handling generate in stacks#5715

Open
denis256 wants to merge 9 commits intomainfrom
5702-tf-stack-output
Open

bug: handling generate in stacks#5715
denis256 wants to merge 9 commits intomainfrom
5702-tf-stack-output

Conversation

@denis256
Copy link
Copy Markdown
Member

@denis256 denis256 commented Mar 20, 2026

Description

  • updated stacks handling to handle generating block
  • added experiment flag to enable this functionality
  • added tests to track this functionality

Fixes #5702.

TODOs

Read the Gruntwork contribution guidelines.

  • I authored this code entirely myself
  • I am submitting code based on open source software (e.g. MIT, MPL-2.0, Apache)]
  • I am adding or upgrading a dependency or adapted code and confirm it has a compatible open source license
  • Update the docs.
  • Run the relevant tests successfully, including pre-commit checks.
  • Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Added / Removed / Updated [X].

Migration Guide

Summary by CodeRabbit

  • New Features

    • Added stacks-generate-block experiment that immediately processes generate blocks during stack unit generation, resolving issues with dependency resolution observing stale generated state.
  • Documentation

    • Documented the new experiment, including stabilization criteria and feedback channels.
  • Tests

    • Added integration tests and fixtures validating the experiment behavior.

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
terragrunt-docs Ready Ready Preview, Comment Apr 1, 2026 5:52pm

Request Review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 20, 2026

📝 Walkthrough

Walkthrough

Introduces a new experiment flag stacks-generate-block that processes Terragrunt generate blocks immediately after unit file copying during stack generation. This ensures generated files are available when dependency resolution (via terraform output -json) occurs, preventing conflicts from stale generated content.

Changes

Cohort / File(s) Summary
Experiment Registration
internal/experiment/experiment.go
Added new StacksGenerateBlock constant and registered it in the experiment discovery system.
Core Implementation
pkg/config/stack.go
Extended GenerateStackFile to invoke new processUnitGenerateBlocks helper when the experiment is enabled; helper parses unit configs and writes generate block outputs to the unit directory.
Documentation
docs/src/content/docs/04-reference/04-experiments.md
Added reference documentation for the stacks-generate-block experiment, including behavior description and stabilization criteria.
Test Fixture Structure
test/fixtures/stacks/generate-if-disabled/live/terragrunt.stack.hcl, test/fixtures/stacks/generate-if-disabled/units/.../...
Added stack fixture with two units and conditional generate blocks (alpha and beta providers) to test dynamic provider generation based on input values.
Integration Test
test/integration_stacks_test.go
Added TestStackGenerateIfDisabledRemove to verify generate blocks are processed before filtered dependency resolution, ensuring terraform output -json succeeds on dependent units.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 62.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'bug: handling generate in stacks' is vague and does not clearly convey what the change accomplishes; it uses generic phrasing that lacks specificity about the actual fix. Use a more descriptive title such as 'feat: process generate blocks during stack unit execution' or 'fix: enable terragrunt generate block processing in stack runs' to better communicate the primary change.
✅ Passed checks (3 passed)
Check name Status Explanation
Description check ✅ Passed The PR description covers key aspects (updated stacks handling, experiment flag, tests) and includes the issue reference (#5702), but lacks detailed context about what problem is being solved and how the solution works.
Linked Issues check ✅ Passed The PR implements the fix for issue #5702 by adding generate block processing during stack unit execution via an experiment flag, including necessary tests and documentation as required.
Out of Scope Changes check ✅ Passed All changes are directly related to implementing generate block processing for stack units and resolving issue #5702; no unrelated or out-of-scope modifications were detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 5702-tf-stack-output

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@denis256 denis256 marked this pull request as ready for review April 1, 2026 18:37
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/config/stack.go`:
- Around line 320-344: In processUnitGenerateBlocks, add a defensive nil check
for cfg before iterating over cfg.GenerateConfigs: if cfg is nil, return nil (or
otherwise skip processing) to avoid a potential nil map access; locate the
ParseConfig call that assigns cfg and ensure you check "if cfg == nil { return
nil }" before the for name, genCfg := range cfg.GenerateConfigs loop so the
function safely handles a (nil, nil) return from ParseConfig.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e7f58264-08b9-4abc-8e6b-f871fe2a75fd

📥 Commits

Reviewing files that changed from the base of the PR and between d1525ed and d0cf1f5.

📒 Files selected for processing (11)
  • docs/src/content/docs/04-reference/04-experiments.md
  • internal/experiment/experiment.go
  • pkg/config/stack.go
  • test/fixtures/stacks/generate-if-disabled/live/terragrunt.stack.hcl
  • test/fixtures/stacks/generate-if-disabled/units/first/alpha.tf
  • test/fixtures/stacks/generate-if-disabled/units/first/beta.tf
  • test/fixtures/stacks/generate-if-disabled/units/first/main.tf
  • test/fixtures/stacks/generate-if-disabled/units/first/terragrunt.hcl
  • test/fixtures/stacks/generate-if-disabled/units/second/main.tf
  • test/fixtures/stacks/generate-if-disabled/units/second/terragrunt.hcl
  • test/integration_stacks_test.go

Comment on lines +320 to 344
func processUnitGenerateBlocks(ctx context.Context, l log.Logger, pctx *ParsingContext, unitDir string) error {
configPath := filepath.Join(unitDir, DefaultTerragruntConfigPath)
if !util.FileExists(configPath) {
return nil
}

unitPctx := pctx.Clone()
unitPctx.TerragruntConfigPath = configPath
unitPctx.WorkingDir = unitDir
unitPctx.SkipOutputsResolution = true

cfg, err := ParseConfigFile(ctx, unitPctx, l, configPath, nil)
if err != nil {
l.Debugf("Could not parse generate blocks for %s during stack generation: %v", configPath, err)
return nil
}

for name, genCfg := range cfg.GenerateConfigs {
if err := codegen.WriteToFile(l, unitDir, &genCfg); err != nil {
return errors.Errorf("failed to process generate block %q (path %q) in %s: %w", name, genCfg.Path, unitDir, err)
}
}

return nil
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify if ParseConfigFile can return nil config without error
ast-grep --pattern $'func ParseConfigFile($$$) ($_, $_) {
  $$$
}'

Repository: gruntwork-io/terragrunt

Length of output: 4479


🏁 Script executed:

#!/bin/bash
# Find and examine the ParseConfig function
ast-grep --pattern $'func ParseConfig($$$) ($_, $_) {
  $$$
}'

Repository: gruntwork-io/terragrunt

Length of output: 8443


Add a nil check for cfg before iterating over GenerateConfigs.

The ParseConfig function uses a MultiError pattern that allows it to accumulate errors and continue processing. While unlikely in normal operation, if convertToTerragruntConfig returns nil and no other errors occur, ParseConfig could theoretically return (nil, nil). A defensive nil check here prevents potential panics on nil map access.

Proposed defensive nil check
 	cfg, err := ParseConfigFile(ctx, unitPctx, l, configPath, nil)
 	if err != nil {
 		l.Debugf("Could not parse generate blocks for %s during stack generation: %v", configPath, err)
 		return nil
 	}
+	if cfg == nil {
+		return nil
+	}

 	for name, genCfg := range cfg.GenerateConfigs {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func processUnitGenerateBlocks(ctx context.Context, l log.Logger, pctx *ParsingContext, unitDir string) error {
configPath := filepath.Join(unitDir, DefaultTerragruntConfigPath)
if !util.FileExists(configPath) {
return nil
}
unitPctx := pctx.Clone()
unitPctx.TerragruntConfigPath = configPath
unitPctx.WorkingDir = unitDir
unitPctx.SkipOutputsResolution = true
cfg, err := ParseConfigFile(ctx, unitPctx, l, configPath, nil)
if err != nil {
l.Debugf("Could not parse generate blocks for %s during stack generation: %v", configPath, err)
return nil
}
for name, genCfg := range cfg.GenerateConfigs {
if err := codegen.WriteToFile(l, unitDir, &genCfg); err != nil {
return errors.Errorf("failed to process generate block %q (path %q) in %s: %w", name, genCfg.Path, unitDir, err)
}
}
return nil
}
func processUnitGenerateBlocks(ctx context.Context, l log.Logger, pctx *ParsingContext, unitDir string) error {
configPath := filepath.Join(unitDir, DefaultTerragruntConfigPath)
if !util.FileExists(configPath) {
return nil
}
unitPctx := pctx.Clone()
unitPctx.TerragruntConfigPath = configPath
unitPctx.WorkingDir = unitDir
unitPctx.SkipOutputsResolution = true
cfg, err := ParseConfigFile(ctx, unitPctx, l, configPath, nil)
if err != nil {
l.Debugf("Could not parse generate blocks for %s during stack generation: %v", configPath, err)
return nil
}
if cfg == nil {
return nil
}
for name, genCfg := range cfg.GenerateConfigs {
if err := codegen.WriteToFile(l, unitDir, &genCfg); err != nil {
return errors.Errorf("failed to process generate block %q (path %q) in %s: %w", name, genCfg.Path, unitDir, err)
}
}
return nil
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/config/stack.go` around lines 320 - 344, In processUnitGenerateBlocks,
add a defensive nil check for cfg before iterating over cfg.GenerateConfigs: if
cfg is nil, return nil (or otherwise skip processing) to avoid a potential nil
map access; locate the ParseConfig call that assigns cfg and ensure you check
"if cfg == nil { return nil }" before the for name, genCfg := range
cfg.GenerateConfigs loop so the function safely handles a (nil, nil) return from
ParseConfig.


#### `stacks-generate-block` - What it does

When Terragrunt generates a stack (via `stack generate` or `stack run`), units may contain `generate` blocks with conditional logic such as `if_disabled = "remove"`. Without this experiment, those generate blocks are not processed during stack generation. This means that when a subsequent filtered run (`--filter`) triggers dependency resolution via `terraform output -json`, the generated unit directory may contain stale or conflicting files (e.g., duplicate `required_providers` blocks), causing Terraform to fail.
Copy link
Copy Markdown
Collaborator

@yhakbar yhakbar Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When Terragrunt generates a stack (via `stack generate` or `stack run`), units may contain `generate` blocks with conditional logic such as `if_disabled = "remove"`. Without this experiment, those generate blocks are not processed during stack generation. This means that when a subsequent filtered run (`--filter`) triggers dependency resolution via `terraform output -json`, the generated unit directory may contain stale or conflicting files (e.g., duplicate `required_providers` blocks), causing Terraform to fail.
When Terragrunt generates a stack (via `stack generate` or `stack run`), units may contain `generate` blocks with conditional logic such as `if_disabled = "remove"`. Without this experiment, those generate blocks are not processed during stack generation. This means that when a subsequent filtered run (`--filter`) triggers dependency resolution via `tofu/terraform output -json`, the generated unit directory may contain stale or conflicting files (e.g., duplicate `required_providers` blocks), causing an error in OpenTofu/Terraform.


#### `stacks-generate-block` - What it does

When Terragrunt generates a stack (via `stack generate` or `stack run`), units may contain `generate` blocks with conditional logic such as `if_disabled = "remove"`. Without this experiment, those generate blocks are not processed during stack generation. This means that when a subsequent filtered run (`--filter`) triggers dependency resolution via `terraform output -json`, the generated unit directory may contain stale or conflicting files (e.g., duplicate `required_providers` blocks), causing Terraform to fail.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't get this. Why does tofu output -json get called from --filter or queue construction? It should only get called just in time before the dependent unit runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Terragrunt uses terraform output -json instead of terragrunt output -json

2 participants