Initial cross-account log sharing module #176
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rfairburn
requested review from
a team,
ddribeiro and
edwardsb
as code owners
Member
|
@coderabbitai full review |
There was a problem hiding this comment.
Pull request overview
This PR introduces an initial “bring your own” cross-account CloudWatch Logs sharing set of Terraform modules, including AWS target-account destinations (Kinesis / Firehose+S3), a GCP target (Pub/Sub→GCS), and an AWS Lambda bridge to forward CloudWatch Logs to GCP Pub/Sub.
Changes:
- Added Terraform modules for CloudWatch Logs sharing (source account subscription + multiple target account implementations).
- Added an AWS Lambda “pubsub-bridge” module (Go) with optional DLQ, alerting, and an SQS replayer Lambda.
- Added a GCP target module (Pub/Sub topic, publisher service account, GCS bucket, and subscription) plus a helper publish script.
Reviewed changes
Copilot reviewed 52 out of 53 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| addons/monitoring/lambda/go.mod | Bumps Go version used by the existing monitoring Lambda module. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/version.tf | Adds Terraform/provider version constraints for the Kinesis target module. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/variables.tf | Defines inputs/validations for CloudWatch destination + Kinesis stream configuration. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/outputs.tf | Exposes destination/Kinesis details for hand-off to source-account config. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/kinesis.tf | Provisions the destination Kinesis stream and enforces same-account providers. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/iam.tf | Creates IAM role/policy allowing CloudWatch Logs to write to Kinesis. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/destination.tf | Creates CloudWatch Logs destination + destination policy for cross-account subscribe. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/README.md | Documents usage and inputs/outputs for the Kinesis target module. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/.terraform-docs.yml | Configures terraform-docs header injection for this module. |
| addons/byo-cloudwatch-log-sharing/target-account-kinesis/.header.md | Header content used by terraform-docs for this module. |
| addons/byo-cloudwatch-log-sharing/target-account-gcp/version.tf | Adds Terraform/provider constraints for the GCP target module. |
| addons/byo-cloudwatch-log-sharing/target-account-gcp/variables.tf | Defines GCP target inputs/validations (Pub/Sub, SA, GCS, delivery). |
| addons/byo-cloudwatch-log-sharing/target-account-gcp/main.tf | Provisions Pub/Sub topic, SA/key, bucket, IAM, and GCS sink subscription. |
| addons/byo-cloudwatch-log-sharing/target-account-gcp/outputs.tf | Exposes values needed by the AWS bridge (topic/project + SA credentials). |
| addons/byo-cloudwatch-log-sharing/target-account-gcp/scripts/publish_pubsub_test.sh | Adds helper script to publish test messages to Pub/Sub. |
| addons/byo-cloudwatch-log-sharing/target-account-gcp/README.md | Documents module usage, hand-off values, and test script. |
| addons/byo-cloudwatch-log-sharing/target-account-gcp/.terraform-docs.yml | Configures terraform-docs header injection for this module. |
| addons/byo-cloudwatch-log-sharing/target-account-gcp/.header.md | Header content used by terraform-docs for this module. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/version.tf | Adds Terraform/provider constraints for the Firehose target module. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/variables.tf | Defines inputs/validations for Firehose, destination, and S3 settings. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/s3.tf | Provisions the Firehose destination S3 bucket and baseline security controls. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/firehose.tf | Provisions IAM role/policy for Firehose and the delivery stream itself. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/iam.tf | Creates IAM role/policy allowing CloudWatch Logs to write to Firehose. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/destination.tf | Creates CloudWatch Logs destination + destination policy for cross-account subscribe. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/outputs.tf | Exposes destination/Firehose/S3 details for hand-off to source-account config. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/README.md | Documents usage and behavior (including compression handling). |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/.terraform-docs.yml | Configures terraform-docs header injection for this module. |
| addons/byo-cloudwatch-log-sharing/target-account-firehose/.header.md | Header content used by terraform-docs for this module. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/version.tf | Adds Terraform/provider constraints for the AWS→GCP Pub/Sub bridge module. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/variables.tf | Defines inputs/validations for subscription, Lambda, GCP Pub/Sub, DLQ, alerting, replayer. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/lambda.tf | Builds/packages the Go bridge Lambda during apply and configures env vars. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/subscription.tf | Creates the CloudWatch Logs subscription filter targeting the bridge Lambda. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/iam.tf | Provisions IAM roles/policies for bridge and optional replayer (Secrets Manager, SQS, KMS, Lambda invoke). |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/dlq.tf | Provisions optional DLQ and async invoke failure destination config. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/alerts.tf | Adds CloudWatch alarms for Lambda errors and DLQ backlog. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/replayer.tf | Adds optional Go replayer Lambda build/package + SQS event source mapping. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/outputs.tf | Exposes operational details (Lambda, subscription, pubsub config, DLQ, alarms, replayer). |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/lambda/go.mod | Adds Go module definition for the bridge Lambda implementation. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/lambda/go.sum | Adds dependency lockfile for bridge Lambda module. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/lambda/main.go | Implements CloudWatch Logs payload decode + publish to GCP Pub/Sub. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/lambda/main_test.go | Adds unit tests for secret parsing, decoding, batching, and handler behavior. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/lambda/replayer/main.go | Implements DLQ replay to re-invoke bridge Lambda with original payload. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/lambda/replayer/main_test.go | Adds unit tests for replay extraction, invoke behavior, and batch failures. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/README.md | Documents the bridge module usage and operational features (DLQ, alarms, replay). |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/.terraform-docs.yml | Configures terraform-docs header injection for this module. |
| addons/byo-cloudwatch-log-sharing/pubsub-bridge/.header.md | Header content used by terraform-docs for this module. |
| addons/byo-cloudwatch-log-sharing/cloudwatch/version.tf | Adds Terraform/provider constraints for the source-account subscription module. |
| addons/byo-cloudwatch-log-sharing/cloudwatch/variables.tf | Defines subscription inputs/validations (dest type, distribution). |
| addons/byo-cloudwatch-log-sharing/cloudwatch/main.tf | Provisions the CloudWatch Logs subscription filter in the source account. |
| addons/byo-cloudwatch-log-sharing/cloudwatch/outputs.tf | Exposes created subscription filter details. |
| addons/byo-cloudwatch-log-sharing/cloudwatch/README.md | Documents usage for Firehose and Kinesis destination types. |
| addons/byo-cloudwatch-log-sharing/cloudwatch/.terraform-docs.yml | Configures terraform-docs header injection for this module. |
| addons/byo-cloudwatch-log-sharing/cloudwatch/.header.md | Header content used by terraform-docs for this module. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Author
|
@coderabbitai review |
Member
@lukeheath Can you enable Code Rabbit AI on this repo? |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 52 out of 53 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.