[Terragrunt] Add default Terragrunt Parser for Stdout, Consolidated output mode #142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

justushar wants to merge 12 commits into master from Terragrunt/Parser

README.md

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -10,20 +10,24 @@ tfnotify
  
    [goreportcard]: https://goreportcard.com/report/github.com/mercari/tfnotify

    [goreportcard-svg]: https://goreportcard.com/badge/github.com/mercari/tfnotify

    tfnotify parses Terraform commands' execution result and applies it to an arbitrary template and then notifies it to GitHub comments etc.

    Tfnotify is a Go-based template engine that creates comments on Terraform operations. It pipes commands directly and captures operation outputs, including exit codes and error details and offers to post them as a Github Comment, with AI summaries and analysis based on custom templates.

    ## Motivation

    There are commands such as `plan` and `apply` on Terraform command, but many developers think they would like to check if the execution of those commands succeeded.

    Terraform commands are often executed via CI like Circle CI, but in that case you need to go to the CI page to check it.

    This is very troublesome. It is very efficient if you can check it with GitHub comments or Slack etc.

    You can do this by using this command.

    Terraform commands are often executed via CI like Cloudbuild CI/Github Actions, but in that case you need to go to the CI page to check it.

    This is very troublesome. It is very efficient if you can check it with the supported CI.

    <img src="./misc/images/1.png" width="600">

    ### Key Features:

    <img src="./misc/images/2.png" width="500">

    <img src="./misc/images/3.png" width="600">

    - **Flexible Templating**: Go template system with custom functions for formatting notifications

    - **Intelligent Parsing**: Regex-based engine that extracts meaningful information from Terraform plan and apply outputs.

    - **Multi-Platform Support**: Integrates with GitHub, CircleCI, Cloudbuild.

    - **CI/CD Awareness**: Automatically detects and integrates with 9+ CI/CD platforms

    - **Label Management**: Automatically creates and manages GitHub labels based on plan results

    - **AI-Powered Summaries**: Generates optional AI summaries of infrastructure changes using multiple providers

    - **Security Features**: Built-in masking system prevents sensitive data exposure

    - **Comment Management**: Updates existing comments instead of creating duplicates

    ## Installation

    @@ -35,16 +39,18 @@ or
  
    $ go get -u github.com/mercari/tfnotify

    ```

    ### Supported Platforms

    ### What tfnotify does

    1. Parse the execution result of Terraform

    2. Bind parsed results to Go templates

    3. Notify it to any platform (e.g. GitHub) as you like

    Tfnotify supports the following platforms:

    Detailed specifications such as templates and notification destinations can be customized from the configuration files (described later).

    - CI

        - CircleCI

        - CodeBuild

        - CloudBuild

        - GitHub Actions

    - Notifier

        - GitHub

    ## Usage

    ### Basic

    @@ -53,7 +59,7 @@ tfnotify is just CLI command. So you can run it from your local after grabbing t
  
    Basically tfnotify waits for the input from Stdin. So tfnotify needs to pipe the output of Terraform command like the following:

    ```console

    $ terraform plan | tfnotify plan

    $ tfnotify plan

    ```

    For `plan` command, you also need to specify `plan` as the argument of tfnotify. In the case of `apply`, you need to do `apply`. Currently supported commands can be checked with `tfnotify --help`.

    @@ -62,7 +68,7 @@ For `plan` command, you also need to specify `plan` as the argument of tfnotify.
  
    When running tfnotify, you can specify the configuration path via `--config` option (if it's omitted, it defaults to `{.,}tfnotify.y{,a}ml`).

    The example settings of GitHub and GitHub Enterprise, Slack, [Typetalk](https://www.typetalk.com/) are as follows. Incidentally, there is no need to replace TOKEN string such as `$GITHUB_TOKEN` with the actual token. Instead, it must be defined as environment variables in CI settings.

    The example settings of GitHub and GitHub Enterprise. Incidentally, there is no need to replace TOKEN string such as `$GITHUB_TOKEN` with the actual token. Instead, it must be defined as environment variables in CI settings.

    [template](https://golang.org/pkg/text/template/) of Go can be used for `template`. The templates can be used in `tfnotify.yaml` are as follows:

    @@ -265,145 +271,6 @@ terraform:
  
    </details>

    <details>

    <summary>For GitLab</summary>

    ```yaml

    ---

    ci: gitlabci

    notifier:

      gitlab:

        token: $GITLAB_TOKEN

        base_url: $GITLAB_BASE_URL

        repository:

          owner: "mercari"

          name: "tfnotify"

    terraform:

      fmt:

        template: |

          {{ .Title }}

          {{ .Message }}

          {{ .Result }}

          {{ .Body }}

      plan:

        template: |

          {{ .Title }} <sup>[CI link]( {{ .Link }} )</sup>

          {{ .Message }}

          {{if .Result}}

          <pre><code> {{ .Result }}

          </pre></code>

          {{end}}

          <details><summary>Details (Click me)</summary>

          <pre><code> {{ .Body }}

          </pre></code></details>

      apply:

        template: |

          {{ .Title }}

          {{ .Message }}

          {{if .Result}}

          <pre><code> {{ .Result }}

          </pre></code>

          {{end}}

          <details><summary>Details (Click me)</summary>

          <pre><code> {{ .Body }}

          </pre></code></details>

    ```

    </details>

    <details>

    <summary>For Slack</summary>

    ```yaml

    ---

    ci: circleci

    notifier:

      slack:

        token: $SLACK_TOKEN

        channel: $SLACK_CHANNEL_ID

        bot: $SLACK_BOT_NAME

    terraform:

      plan:

        template: |

          {{ .Message }}

          {{if .Result}}

          ```

          {{ .Result }}

          ```

          {{end}}

          ```

          {{ .Body }}

          ```

    ```

    Sometimes you may want not to HTML-escape Terraform command outputs.

    For example, when you use code block to print command output, it's better to use raw characters instead of character references (e.g. `-/+` -> `-/&#43;`, `"` -> `&#34;`).

    You can disable HTML escape by adding `use_raw_output: true` configuration.

    With this configuration, Terraform doesn't HTML-escape any Terraform output.

    ~~~yaml

    ---

    # ...

    terraform:

      use_raw_output: true

      # ...

      plan:

      # ...

    ~~~

    </details>

    <details>

    <summary>For Typetalk</summary>

    ```yaml

    ---

    ci: circleci

    notifier:

      typetalk:

        token: $TYPETALK_TOKEN

        topic_id: $TYPETALK_TOPIC_ID

    terraform:

      plan:

        template: |

          {{ .Message }}

          {{if .Result}}

          ```

          {{ .Result }}

          ```

          {{end}}

          ```

          {{ .Body }}

          ```

    ```

    </details>

    ### Supported CI

    Currently, supported CI are here:

    - Circle CI

    - Travis CI

    - AWS CodeBuild

    - TeamCity

    - Drone

    - Jenkins

    - GitLab CI

    - GitHub Actions

    - Google Cloud Build

    ### Private Repository Considerations

    GitHub private repositories require the `repo` and `write:discussion` permissions.

    ### Jenkins Considerations

    - Plugin

      - [Git Plugin](https://wiki.jenkins.io/display/JENKINS/Git+Plugin)

    - Environment Variable

      - `PULL_REQUEST_NUMBER` or `PULL_REQUEST_URL` are required to set by user for Pull Request Usage

    ### Google Cloud Build Considerations

    @@ -417,10 +284,6 @@ GitHub private repositories require the `repo` and `write:discussion` permission
  
      - `terraform plan`: Pull request

      - `terraform apply`: Push to branch

    ## Committers

     * Masaki ISHIYAMA ([@b4b4r07](https://github.com/b4b4r07))

    ## Contribution

    Please read the CLA below carefully before submitting your contribution.

go.mod

-Original file line number
+Diff line change
@@ Expand Up / @@ -9,6 +9,7 @@ require ( @@
     	github.com/mattn/go-colorable v0.1.14
     	github.com/shurcooL/githubv4 v0.0.0-20240429030203-be2daab69064
     	github.com/sirupsen/logrus v1.9.3
+    	github.com/slack-go/slack v0.17.3
     	github.com/suzuki-shunsuke/gen-go-jsonschema v0.1.0
     	github.com/suzuki-shunsuke/github-comment-metadata v0.1.0
     	github.com/suzuki-shunsuke/go-ci-env/v3 v3.2.0
@@ Expand All / @@ -28,6 +29,7 @@ require ( @@
     	github.com/buger/jsonparser v1.1.1 // indirect
     	github.com/google/go-querystring v1.1.0 // indirect
     	github.com/google/uuid v1.6.0 // indirect
+    	github.com/gorilla/websocket v1.5.3 // indirect
     	github.com/huandu/xstrings v1.5.0 // indirect
     	github.com/invopop/jsonschema v0.12.0 // indirect
     	github.com/mailru/easyjson v0.7.7 // indirect
@@ Expand Down @@

go.sum

-Original file line number
+Diff line change
@@ Expand Up @@
     github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
     github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
     github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
+    github.com/go-test/deep v1.1.1 h1:0r/53hagsehfO4bzD2Pgr/+RgHqhmf+k1Bpse2cTu1U=
+    github.com/go-test/deep v1.1.1/go.mod h1:5C2ZWiW0ErCdrYzpqxLbTX7MG14M9iiw8DgHncVwcsE=
     github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
     github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
     github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
@@ Expand All @@
     github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
     github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
     github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+    github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
+    github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
     github.com/huandu/xstrings v1.5.0 h1:2ag3IFq9ZDANvthTwTiqSSZLjDc+BedvHPAp5tJy2TI=
     github.com/huandu/xstrings v1.5.0/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
     github.com/invopop/jsonschema v0.12.0 h1:6ovsNSuvn9wEQVOyc72aycBMVQFKz7cPdMJn10CvzRI=
@@ Expand Down Expand Up @@
     github.com/shurcooL/graphql v0.0.0-20200928012149-18c5c3165e3a/go.mod h1:AuYgA5Kyo4c7HfUmvRGs/6rGlMMV/6B1bVnB9JxJEEg=
     github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
     github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
+    github.com/slack-go/slack v0.17.3 h1:zV5qO3Q+WJAQ/XwbGfNFrRMaJ5T/naqaonyPV/1TP4g=
+    github.com/slack-go/slack v0.17.3/go.mod h1:X+UqOufi3LYQHDnMG1vxf0J8asC6+WllXrVrhl8/Prk=
     github.com/spf13/cast v1.7.0 h1:ntdiHjuueXFgm5nzDRdOS4yfT43P5Fnud6DH50rz/7w=
     github.com/spf13/cast v1.7.0/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
     github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ Expand Down @@

pkg/cli/app.go

-Original file line number
+Diff line change
@@ Expand Up @@
     						Usage:   "Disable to add or update a label",
     						Sources: cli.EnvVars("TFNOTIFY_DISABLE_LABEL"),
     					},
+    					&cli.BoolFlag{
+    						Name:    "consolidated",
+    						Usage:   "For Terragrunt: consolidate all module results into a single comment instead of posting per module",
+    						Sources: cli.EnvVars("TFNOTIFY_CONSOLIDATED"),
+    					},
     					&cli.BoolFlag{
     						Name:    "summary",
     						Usage:   "Generate AI-powered summary of plan consequences",
@@ Expand Down Expand Up @@
     $ tfnotify [<global options>] apply -- terraform apply [<terraform apply options>]`,
     				Action: cmdApply,
     				Flags: []cli.Flag{
+    					&cli.BoolFlag{
+    						Name:    "consolidated",
+    						Usage:   "For Terragrunt: consolidate all module results into a single comment instead of posting per module",
+    						Sources: cli.EnvVars("TFNOTIFY_CONSOLIDATED"),
+    					},
     					&cli.BoolFlag{
     						Name:    "summary",
     						Usage:   "Generate AI-powered summary of apply consequences",
@@ Expand Down @@

pkg/cli/apply.go

-Original file line number
+Diff line change
@@ Expand Up / @@ -47,9 +47,23 @@ func cmdApply(ctx context.Context, cmd *cli.Command) error { @@
     	// Get session ID if provided
     	sessionID := cmd.String("session-id")
+    	// Check if consolidated flag is set
+    	if cmd.Bool("consolidated") {
+    		cfg.Terraform.Consolidated = true
+    		logrus.Info("Terragrunt consolidated mode enabled")
+    	}
+    	// Select parser based on configuration
+    	var parser terraform.Parser
+    	if cfg.Terraform.Consolidated {
+    		parser = terraform.NewTerragruntParser(true)
+    	} else {
+    		parser = terraform.NewApplyParser()
+    	}
     	t := &controller.Controller{
     		Config:             cfg,
-    		Parser:             terraform.NewApplyParser(),
+    		Parser:             parser,
     		Template:           terraform.NewApplyTemplate(cfg.Terraform.Apply.Template),
     		ParseErrorTemplate: terraform.NewApplyParseErrorTemplate(cfg.Terraform.Apply.WhenParseError.Template),
     	}
@@ Expand Down @@

pkg/cli/plan.go

-Original file line number
+Diff line change
@@ Expand Up / @@ -46,9 +46,23 @@ func cmdPlan(ctx context.Context, cmd *cli.Command) error { @@
     	// Get session ID if provided
     	sessionID := cmd.String("session-id")
+    	// Check if consolidated flag is set
+    	if cmd.Bool("consolidated") {
+    		cfg.Terraform.Consolidated = true
+    		logrus.Info("Terragrunt consolidated mode enabled")
+    	}
+    	// Select parser based on configuration
+    	var parser terraform.Parser
+    	if cfg.Terraform.Consolidated {
+    		parser = terraform.NewTerragruntParser(true)
+    	} else {
+    		parser = terraform.NewPlanParser()
+    	}
     	t := &controller.Controller{
     		Config:             cfg,
-    		Parser:             terraform.NewPlanParser(),
+    		Parser:             parser,
     		Template:           terraform.NewPlanTemplate(cfg.Terraform.Plan.Template),
     		ParseErrorTemplate: terraform.NewPlanParseErrorTemplate(cfg.Terraform.Plan.WhenParseError.Template),
     	}
@@ Expand Down @@

pkg/config/config.go

-Original file line number
+Diff line change
@@ Expand Up / @@ -14,6 +14,7 @@ import ( @@
     type Config struct {
     	CI                 CI                `json:"-" yaml:"-"`
     	Terraform          Terraform         `json:"terraform,omitempty"`
+    	Slack              Slack             `json:"slack,omitempty"`
     	Vars               map[string]string `json:"-" yaml:"-"`
     	EmbeddedVarNames   []string          `json:"embedded_var_names,omitempty" yaml:"embedded_var_names"`
     	Templates          map[string]string `json:"templates,omitempty"`
@@ Expand Down Expand Up / @@ -58,11 +59,29 @@ type Log struct { @@
     	// Format string
     }
+    // Slack represents slack notification configurations
+    type Slack struct {
+    	Enabled            bool   `json:"enabled,omitempty"`
+    	Token              string `json:"-" yaml:"-"`                                                   // Read from env var SLACK_BOT_TOKEN
+    	ChannelID          string `json:"-" yaml:"-"`                                                   // Read from env var SLACK_CHANNEL_ID
+    	BotName            string `json:"-" yaml:"-"`                                                   // Read from env var SLACK_BOT_NAME
+    	Title              string `json:"title,omitempty"`                                              // Default title template for Slack messages
+    	Message            string `json:"message,omitempty"`                                            // Default message template for Slack messages
+    	ApplyTitle         string `json:"apply_title,omitempty" yaml:"apply_title"`                     // Title template for apply notifications
+    	ApplyMessage       string `json:"apply_message,omitempty" yaml:"apply_message"`                 // Message template for apply notifications
+    	PlanTitle          string `json:"plan_title,omitempty" yaml:"plan_title"`                       // Title template for plan notifications
+    	PlanMessage        string `json:"plan_message,omitempty" yaml:"plan_message"`                   // Message template for plan notifications
+    	NotifyOnPlanError  bool   `json:"notify_on_plan_error,omitempty" yaml:"notify_on_plan_error"`   // Send Slack notification on plan failures
+    	NotifyOnApplyError bool   `json:"notify_on_apply_error,omitempty" yaml:"notify_on_apply_error"` // Send Slack notification on apply failures
+    	UseThreads         bool   `json:"use_threads,omitempty" yaml:"use_threads"`                     // Send error details in a thread reply
+    }
     // Terraform represents terraform configurations
     type Terraform struct {
     	Plan         Plan  `json:"plan,omitempty"`
     	Apply        Apply `json:"apply,omitempty"`
     	UseRawOutput bool  `json:"use_raw_output,omitempty" yaml:"use_raw_output"`
+    	Consolidated bool  `json:"consolidated,omitempty" yaml:"consolidated"`
     }
     // Plan is a terraform plan config
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Terragrunt] Add default Terragrunt Parser for Stdout, Consolidated output mode #142

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

[Terragrunt] Add default Terragrunt Parser for Stdout, Consolidated output mode #142

Are you sure you want to change the base?

Uh oh!

[Terragrunt] Add default Terragrunt Parser for Stdout, Consolidated output mode #142

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!