Skip to content

Split observability da #366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Aug 25, 2025
Merged

Split observability da #366

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
0fbb506
rally
rajatagarwal-ibm Jun 19, 2025
6b96129
Merge branch 'main' of github.com:terraform-ibm-modules/terraform-ibm…
rajatagarwal-ibm Jun 24, 2025
d102a5e
Merge branch 'main' of github.com:terraform-ibm-modules/terraform-ibm…
rajatagarwal-ibm Jul 16, 2025
516164f
split observability DA consumption
rajatagarwal-ibm Jul 16, 2025
805a290
split observability DA consumption
rajatagarwal-ibm Jul 16, 2025
5a2fb16
split observability DA consumption
rajatagarwal-ibm Aug 1, 2025
deb06ea
Merge branch 'main' into split-observability-da
rajatagarwal-ibm Aug 1, 2025
18af21e
split observability DA consumption
rajatagarwal-ibm Aug 1, 2025
a02c2f9
Merge branch 'split-observability-da' of github.com:terraform-ibm-mod…
rajatagarwal-ibm Aug 1, 2025
2e256a9
Merge branch 'main' into split-observability-da
rajatagarwal-ibm Aug 5, 2025
0b50afa
split observability DA consumption
rajatagarwal-ibm Aug 5, 2025
9f3f008
Merge branch 'main' into split-observability-da
rajatagarwal-ibm Aug 6, 2025
20c82c2
split observability DA consumption
rajatagarwal-ibm Aug 7, 2025
3afb7c9
Merge branch 'split-observability-da' of github.com:terraform-ibm-mod…
rajatagarwal-ibm Aug 7, 2025
5690f9c
split observability DA consumption
rajatagarwal-ibm Aug 7, 2025
fced7f8
split observability DA consumption
rajatagarwal-ibm Aug 7, 2025
4b80f7b
split observability DA consumption
rajatagarwal-ibm Aug 7, 2025
2540e95
split observability DA consumption
rajatagarwal-ibm Aug 8, 2025
43c266b
split observability DA consumption
rajatagarwal-ibm Aug 13, 2025
655cfc3
split observability DA consumption
rajatagarwal-ibm Aug 13, 2025
d3d3c91
split observability DA consumption
rajatagarwal-ibm Aug 13, 2025
ec39f2e
Merge branch 'main' into split-observability-da
rajatagarwal-ibm Aug 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 68 additions & 26 deletions ibm_catalog.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"solution"
],
"short_description": "Cloud architecture including Secrets Manager instance and optional security, logging and notification services.",
"long_description": "This deployable architecture is used to provision and configure an [IBM Cloud Secrets Manager](https://www.ibm.com/products/secrets-manager) instance. **Optionally**, supports creating and/or configuring:\n* [IBM Cloud account](https://cloud.ibm.com/docs/account?topic=account-account-getting-started): To set up IBM Cloud accounts settings.\n* [Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-getting-started-tutorial): For data encryption using customer-managed keys.\n* [Cloud Logs](https://cloud.ibm.com/docs/cloud-logs?topic=cloud-logs-getting-started): Logging and monitoring platform logs.\n* [Cloud Monitoring](https://cloud.ibm.com/docs/monitoring?topic=monitoring-getting-started):Measure how users and applications interact with the Secrets Manager instance.\n* [Event Notifications](https://cloud.ibm.com/docs/event-notifications?topic=event-notifications-getting-started): Send notifications of events to other users, or destinations, by using email, SMS or other supported delivery channels.\n\nℹ️ This Terraform-based automation is part of a broader suite of IBM-maintained Infrastructure as Code (IaC) assets, each following the naming pattern \"Cloud automation for *servicename*\" and focusing on single IBM Cloud service. These single-service deployable architectures can be used on their own to streamline and automate service deployments through an [IaC approach](https://cloud.ibm.com/docs/secure-enterprise?topic=secure-enterprise-understanding-projects), or assembled together into a broader [automated IaC stack](https://cloud.ibm.com/docs/secure-enterprise?topic=secure-enterprise-config-stack) to automate the deployment of an end-to-end solution architecture.",
"long_description": "This deployable architecture is used to provision and configure an [IBM Cloud Secrets Manager](https://www.ibm.com/products/secrets-manager) instance. **Optionally**, supports creating and/or configuring:\n* [IBM Cloud account](https://cloud.ibm.com/docs/account?topic=account-account-getting-started): To set up IBM Cloud accounts settings.\n* [Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-getting-started-tutorial): For data encryption using customer-managed keys.\n* [Cloud Logs](https://cloud.ibm.com/docs/cloud-logs?topic=cloud-logs-getting-started): Logging and monitoring platform logs.\n* [Cloud Monitoring](https://cloud.ibm.com/docs/monitoring?topic=monitoring-getting-started):Measure how users and applications interact with the Secrets Manager instance.\n*[Activity Tracker Event Routing](https://cloud.ibm.com/docs/atracker?topic=atracker-getting-started): Configures how to route auditing events.\n* [Event Notifications](https://cloud.ibm.com/docs/event-notifications?topic=event-notifications-getting-started): Send notifications of events to other users, or destinations, by using email, SMS or other supported delivery channels.\n\nℹ️ This Terraform-based automation is part of a broader suite of IBM-maintained Infrastructure as Code (IaC) assets, each following the naming pattern \"Cloud automation for *servicename*\" and focusing on single IBM Cloud service. These single-service deployable architectures can be used on their own to streamline and automate service deployments through an [IaC approach](https://cloud.ibm.com/docs/secure-enterprise?topic=secure-enterprise-understanding-projects), or assembled together into a broader [automated IaC stack](https://cloud.ibm.com/docs/secure-enterprise?topic=secure-enterprise-config-stack) to automate the deployment of an end-to-end solution architecture.",
"offering_docs_url": "https://github.com/terraform-ibm-modules/terraform-ibm-secrets-manager/blob/main/README.md",
"offering_icon_url": "https://raw.githubusercontent.com/terraform-ibm-modules/terraform-ibm-secrets-manager/main/images/secrets_manager.svg",
"provider_name": "IBM",
Expand Down Expand Up @@ -149,30 +149,20 @@
},
{
"key": "enable_platform_metrics",
"type": "string",
"default_value": "true",
"type": "boolean",
"default_value": false,
"description": "When set to `true`, the IBM Cloud Monitoring instance will be configured to collect platform metrics from the provided region. You can configure 1 instance only of the IBM Cloud Monitoring service per region to collect platform metrics in that location. Check with the account or service administrator if another monitoring instance has already been configured. You may not have permissions to see all monitoring instances in the region. [Learn more](https://cloud.ibm.com/docs/monitoring?topic=monitoring-platform_metrics_enabling).",
"required": true,
"virtual": true,
"options": [
{
"displayname": "true",
"value": "true"
},
{
"displayname": "false",
"value": "false"
}
]
"virtual": true
},
{
"key": "logs_routing_tenant_regions",
"type": "list(string)",
"default_value": "[]",
"description": "To manage platform logs that are generated by IBM Cloud services in a region of IBM Cloud, you must create a tenant in each region that you operate. Pass a list of regions to create a tenant in. [Learn more](https://cloud.ibm.com/docs/logs-router?topic=logs-router-about-platform-logs).",
"required": true,
"default_value": "[]",
"virtual": true,
"custom_config": {
"type": "array",
"grouping": "deployment",
"original_grouping": "deployment",
"config_constraints": {
Expand Down Expand Up @@ -349,15 +339,15 @@
"crn:v1:bluemix:public:iam::::role:Editor"
],
"service_name": "sysdig-monitor",
"notes": "[Optional] Required if you are consuming the Observability deployable architecture which sets up Cloud Monitoring."
"notes": "[Optional] Required if you are consuming the Cloud Monitoring deployable architecture."
},
{
"role_crns": [
"crn:v1:bluemix:public:iam::::serviceRole:Manager",
"crn:v1:bluemix:public:iam::::role:Editor"
],
"service_name": "logs",
"notes": "[Optional] Required if you are consuming the Observability deployable architecture which sets up Cloud Logs."
"notes": "[Optional] Required if you are consuming the Cloud logs deployable architecture."
},
{
"role_crns": [
Expand Down Expand Up @@ -460,14 +450,47 @@
"version": "v5.1.4"
},
{
"name": "deploy-arch-ibm-observability",
"description": "Enables provisioning and configuration of IBM Cloud Logs to collect, route, analyze, and visualize platform logs and events — including those generated by the Event Notifications instance — for enhanced visibility, alerting, and troubleshooting.",
"name": "deploy-arch-ibm-cloud-logs",
Copy link
Contributor

@whoffler whoffler Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just started looking at doing this for Event Streams does Cloud Logs have a dependency on COS?

image

Copy link
Member Author

@rajatagarwal-ibm rajatagarwal-ibm Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think yes, Cloud Logs have dependency on COS

Copy link
Contributor

@whoffler whoffler Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so COS is automatically added as an addon because it is a dependency of an enabled addon

When COS is disabled the user would have to supply the existing_cos_instance_crn themselves... got it, thanks

"description": "Enable to provision and configure IBM Cloud Logs for analysing logs generated by the Event Notifications instance.",
"flavors": [
"fully-configurable"
],
"id": "63d8ae58-fbf3-41ce-b844-0fb5b85882ab-global",
"catalog_id": "7a4d68b4-cf8b-40cd-a3d1-f49aff526eb3",
"input_mapping": [
{
"dependency_input": "prefix",
"version_input": "prefix",
"reference_version": true
},
{
"dependency_input": "region",
"version_input": "region",
"reference_version": true
},
{
"dependency_input": "logs_routing_tenant_regions",
"version_input": "logs_routing_tenant_regions",
"reference_version": true
}
],
"optional": true,
"on_by_default": true,
"version": "v1.5.6"
},
{
"name": "deploy-arch-ibm-cloud-monitoring",
"description": "Enable to provision and configure IBM Cloud Activity Tracker for routing events generated by the Event Notifications instance to a Cloud logs instance.",
"flavors": [
"instances"
"fully-configurable"
],
"id": "a3137d28-79e0-479d-8a24-758ebd5a0eab-global",
"id": "73debdbf-894f-4c14-81c7-5ece3a70b67d-global",
"catalog_id": "7a4d68b4-cf8b-40cd-a3d1-f49aff526eb3",
"input_mapping": [
{
"dependency_output": "cloud_monitoring_crn",
"version_input": "existing_monitoring_crn"
},
{
"dependency_input": "prefix",
"version_input": "prefix",
Expand All @@ -482,16 +505,35 @@
"dependency_input": "enable_platform_metrics",
"version_input": "enable_platform_metrics",
"reference_version": true
}
],
"optional": true,
"on_by_default": true,
"version": "v1.3.0"
},
{
"name": "deploy-arch-ibm-activity-tracker",
"description": "Enable to provision and configure IBM Cloud Activity Tracker event routing for analysing logs and metrics generated by the Event Notifications instance.",
"flavors": [
"fully-configurable"
],
"id": "918453c3-4f97-4583-8c4a-83ef12fc7916-global",
"catalog_id": "7a4d68b4-cf8b-40cd-a3d1-f49aff526eb3",
"input_mapping": [
{
"dependency_input": "prefix",
"version_input": "prefix",
"reference_version": true
},
{
"dependency_input": "logs_routing_tenant_regions",
"version_input": "logs_routing_tenant_regions",
"dependency_input": "region",
"version_input": "region",
"reference_version": true
}
],
"optional": true,
"on_by_default": true,
"version": "v3.0.3"
"version": "v1.1.3"
},
{
"name": "deploy-arch-ibm-event-notifications",
Expand Down Expand Up @@ -519,7 +561,7 @@
],
"optional": true,
"on_by_default": true,
"version": "v2.3.7"
"version": "v2.6.5"
}
],
"dependency_version_2": true,
Expand Down
56 changes: 29 additions & 27 deletions tests/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@ go 1.24.2
toolchain go1.24.3

require (
github.com/gruntwork-io/terratest v0.49.0
github.com/gruntwork-io/terratest v0.50.0
github.com/stretchr/testify v1.10.0
github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper v1.51.1
github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper v1.58.8
)

require (
dario.cat/mergo v1.0.0 // indirect
github.com/IBM-Cloud/bluemix-go v0.0.0-20240719075425-078fcb3a55be // indirect
github.com/IBM-Cloud/power-go-client v1.11.0 // indirect
github.com/IBM/cloud-databases-go-sdk v0.7.1 // indirect
github.com/IBM/go-sdk-core/v5 v5.19.1 // indirect
github.com/IBM/platform-services-go-sdk v0.82.0 // indirect
github.com/IBM-Cloud/power-go-client v1.12.0 // indirect
github.com/IBM/cloud-databases-go-sdk v0.8.0 // indirect
github.com/IBM/go-sdk-core/v5 v5.21.0 // indirect
github.com/IBM/platform-services-go-sdk v0.85.1 // indirect
github.com/IBM/project-go-sdk v0.3.6 // indirect
github.com/IBM/schematics-go-sdk v0.4.0 // indirect
github.com/IBM/vpc-go-sdk v1.0.2 // indirect
Expand All @@ -31,23 +31,23 @@ require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/fsnotify/fsnotify v1.8.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gabriel-vasile/mimetype v1.4.9 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.6.2 // indirect
github.com/go-git/go-git/v5 v5.16.0 // indirect
github.com/go-git/go-git/v5 v5.16.2 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/analysis v0.21.5 // indirect
github.com/go-openapi/analysis v0.23.0 // indirect
github.com/go-openapi/errors v0.22.1 // indirect
github.com/go-openapi/jsonpointer v0.20.1 // indirect
github.com/go-openapi/jsonreference v0.20.3 // indirect
github.com/go-openapi/loads v0.21.3 // indirect
github.com/go-openapi/runtime v0.26.0 // indirect
github.com/go-openapi/spec v0.20.12 // indirect
github.com/go-openapi/jsonpointer v0.21.1 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/loads v0.22.0 // indirect
github.com/go-openapi/runtime v0.28.0 // indirect
github.com/go-openapi/spec v0.21.0 // indirect
github.com/go-openapi/strfmt v0.23.0 // indirect
github.com/go-openapi/swag v0.22.5 // indirect
github.com/go-openapi/validate v0.22.4 // indirect
github.com/go-openapi/swag v0.23.1 // indirect
github.com/go-openapi/validate v0.24.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.26.0 // indirect
Expand All @@ -69,7 +69,7 @@ require (
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.16.7 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mailru/easyjson v0.9.0 // indirect
github.com/mattn/go-zglob v0.0.4 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
Expand All @@ -82,21 +82,23 @@ require (
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/tmccombs/hcl2json v0.6.4 // indirect
github.com/ulikunitz/xz v0.5.11 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/zclconf/go-cty v1.16.2 // indirect
go.mongodb.org/mongo-driver v1.17.3 // indirect
go.opentelemetry.io/otel v1.29.0 // indirect
go.opentelemetry.io/otel/metric v1.29.0 // indirect
go.opentelemetry.io/otel/trace v1.29.0 // indirect
golang.org/x/crypto v0.38.0 // indirect
golang.org/x/mod v0.18.0 // indirect
golang.org/x/net v0.39.0 // indirect
golang.org/x/sync v0.14.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.25.0 // indirect
golang.org/x/tools v0.22.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
golang.org/x/crypto v0.40.0 // indirect
golang.org/x/mod v0.25.0 // indirect
golang.org/x/net v0.41.0 // indirect
golang.org/x/sync v0.16.0 // indirect
golang.org/x/sys v0.34.0 // indirect
golang.org/x/text v0.27.0 // indirect
golang.org/x/tools v0.34.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Loading