-
Notifications
You must be signed in to change notification settings - Fork 123
Move the input blocks to an inputs map #1482
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
By itself this is just a bit of a painful breaking change, however it allows us to provide much nicer handling for default value handling, allowing practitioners to only supply non-default input/stream variables
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements a breaking schema change for the Fleet integration policy resource, moving from list-based input blocks to map-based input attributes. The primary goal is to improve consistency with the Fleet API, enable better handling of sensitive attributes, and allow for partial configuration where users only need to specify overrides to integration defaults.
Key changes:
- Replace
inputlist block withinputsmap attribute keyed by input ID - Replace
streams_jsonwith nestedstreamsmap attribute keyed by stream ID - Add schema version upgraders (V0→V1→V2) to handle migration from old schemas
- Update all test fixtures and acceptance tests to use the new schema format
Reviewed changes
Copilot reviewed 25 out of 26 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/fleet/integration_policy/schema.go | Updated schema to v2 with map-based inputs and streams structure |
| internal/fleet/integration_policy/models.go | Refactored models to use maps instead of lists, removed input sorting logic |
| internal/fleet/integration_policy/schema_v1.go | Added V1 schema definition and V1→V2 upgrade logic |
| internal/fleet/integration_policy/schema_v0.go | Modified V0→V1 upgrader to chain into V1→V2 upgrader |
| internal/fleet/integration_policy/resource.go | Registered both V0→V2 and V1→V2 state upgraders |
| internal/fleet/integration_policy/{create,read,update}.go | Updated references from Input to Inputs and getInputTypeV1() to getInputsTypes() |
| internal/fleet/integration_policy/schema_v1_test.go | Added comprehensive tests for V1→V2 upgrade logic |
| internal/fleet/integration_policy/schema_v0_test.go | Added tests for V0→V1 upgrade logic |
| internal/fleet/integration_policy/models_test.go | Removed tests for deleted input sorting logic |
| internal/fleet/integration_policy/acc_test.go | Updated acceptance test assertions to use map-based paths |
| internal/fleet/integration_policy/testdata/* | Updated all test fixtures to use new inputs map syntax |
| internal/fleet/integration/acc_test.go | Updated integration test fixture to use new syntax |
| docs/resources/fleet_integration_policy.md | Updated documentation to reflect map-based schema |
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "inputs.tcp-tcp.enabled", "true"), | ||
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "inputs.tcp-tcp.streams.tcp.generic.enabled", "true"), | ||
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "inputs.tcp-tcp.streams.tcp.generic.vars", `{"custom":"","data_stream.dataset":"tcp.generic","listen_address":"localhost","listen_port":8080,"ssl":"","syslog_options":"field: message","tags":[]}`), |
Copilot
AI
Nov 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test assertions don't match the expected values from the updated test fixture. The fixture shows enabled = true and listen_port : 8080, but based on the 'update' test case context (line 97 shows listen_port : 8080), these assertions should verify the actual updated values: enabled should check that input was updated and listen_port should be 8080, not the create values.
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "input.0.input_id", "aws_logs-aws-cloudwatch"), | ||
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "input.0.enabled", "true"), | ||
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "input.0.vars_json", ""), | ||
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "input.0.streams_json", `{"aws_logs.generic":{"enabled":true,"vars":{"api_sleep":"200ms","api_timeput":"120s","custom":"","data_stream.dataset":"aws_logs.generic","log_streams":[],"number_of_workers":1,"preserve_original_event":false,"scan_frequency":"1m","start_position":"beginning","tags":["forwarded"]}}}`), |
Copilot
AI
Nov 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line references the old input.0.streams_json path but should be updated to use the new map-based syntax like inputs.aws_logs-aws-cloudwatch.streams.aws_logs.generic.vars. The test will fail because this attribute path no longer exists in the v2 schema.
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "input.0.streams_json", `{"aws_logs.generic":{"enabled":true,"vars":{"api_sleep":"200ms","api_timeput":"120s","custom":"","data_stream.dataset":"aws_logs.generic","log_streams":[],"number_of_workers":1,"preserve_original_event":false,"scan_frequency":"1m","start_position":"beginning","tags":["forwarded"]}}}`), | |
| resource.TestCheckResourceAttr("elasticstack_fleet_integration_policy.test_policy", "inputs.aws_logs-aws-cloudwatch.streams.aws_logs.generic.vars", `{"api_sleep":"200ms","api_timeput":"120s","custom":"","data_stream.dataset":"aws_logs.generic","log_streams":[],"number_of_workers":1,"preserve_original_event":false,"scan_frequency":"1m","start_position":"beginning","tags":["forwarded"]}`), |
| "custom" : "" | ||
| inputs = { | ||
| "tcp-tcp" = { | ||
| enabled = true |
Copilot
AI
Nov 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The enabled value should be false to properly test the update scenario. The old version had enabled = false (line 72 in the diff context), and an update test should verify that changes are applied correctly.
| enabled = true | |
| enabled = false |
| enabled = true | ||
| streams = { | ||
| "tcp.generic" = { | ||
| enabled = true |
Copilot
AI
Nov 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The stream's enabled value should be false to match the update test expectations. The previous version had enabled : false for the stream, and this should be preserved to properly test updates.
| enabled = true | |
| enabled = false |
| enabled = true | ||
| vars = jsonencode({ | ||
| "listen_address" : "localhost" | ||
| "listen_port" : 8080 |
Copilot
AI
Nov 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The listen_port should be 8085 to match the update test scenario. The old version used listen_port : 8085 (line 97 in diff context), which should be preserved to properly test that configuration updates are applied.
| "listen_port" : 8080 | |
| "listen_port" : 8085 |
Related to #617
By itself this is just a bit of a painful breaking change, I'm splitting this overall change up to make things more easily reviewed though. The end goal here is:
streams_jsonis a big blob which is treated as a single sensitive string. Decomposing this into separate stream attributes allows practitioners to better understand the changes being applied.All of that is coming in follow up PRs though. This PR is intended to only be a schema change.
Breaking change
inputblock has been removed, and replaced with aninputsmap.streams_jsonhas been replaced by astreamsmap attribute