Skip to content

Commit 70ca678

Browse files
committed
Merge branch 'main' of github.com:elastic/terraform-provider-elasticstack into copilot/add-elastic-security-exceptions
2 parents c84195f + aae8931 commit 70ca678

File tree

110 files changed

+9390
-2567
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

110 files changed

+9390
-2567
lines changed

.buildkite/release.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
steps:
22
- label: Release
33
agents:
4-
image: "golang:1.25.4@sha256:5d73b7b83dd6e0258ff62832c93b6ea208fbb7727985d265fb49f75f81fc3d1f"
4+
image: "golang:1.25.4@sha256:f60eaa87c79e604967c84d18fd3b151b3ee3f033bcdade4f3494e38411e60963"
55
cpu: "16"
66
memory: "24G"
77
ephemeralStorage: "20G"

.github/copilot-instructions.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,16 @@
1-
You will be tasked to fix an issue from an open-source repository. This is a Go based repository hosting a Terrform provider for the elastic stack (elasticsearch and kibana) APIs. This repo currently supports both [plugin framework](https://developer.hashicorp.com/terraform/plugin/framework/getting-started/code-walkthrough) and [sdkv2](https://developer.hashicorp.com/terraform/plugin/sdkv2) resources. Unless you're told otherwise, all new resources _must_ use the plugin framework.
1+
You will be writing or reviewing code for the Terraform provider for Elastic Stack (Elasticsearch, Kibana, Fleet, APM, and Logstash). This is a Go-based repository hosting the provider source.
22

3-
Take your time and think through every step - remember to check your solution rigorously and watch out for boundary cases, especially with the changes you made. Your solution must be perfect. If not, continue working on it. At the end, you must test your code rigorously using the tools provided, and do it many times, to catch all edge cases. If it is not robust, iterate more and make it perfect. Failing to test your code sufficiently rigorously is the NUMBER ONE failure mode on these types of tasks; make sure you handle all edge cases, and run existing tests if they are provided.
3+
When writing code, you must adhere to the coding standards and conventions outlined in the [CODING_STANDARDS.md](../CODING_STANDARDS.md) document in this repository.
4+
5+
When reviewing code, ensure that all changes comply with the coding standards and conventions specified in the [CODING_STANDARDS.md](../CODING_STANDARDS.md) document. Pay special attention to project structure, schema definitions, JSON handling, resource implementation, and testing practices.
6+
7+
Take your time and think through every step - remember to check solutions rigorously and watch out for boundary cases, especially with the changes being made.
8+
9+
When writing code, your solution must be perfect. If not, continue working on it. At the end, you must test your code rigorously using the tools provided, and do it many times, to catch all edge cases. If it is not robust, iterate more and make it perfect. Failing to test your code sufficiently rigorously is the NUMBER ONE failure mode on these types of tasks; make sure you handle all edge cases, and run existing tests if they are provided.
410

511
Please see [README.md](../README.md) and the [CONTRIBUTING.md](../CONTRIBUTING.md) docs before getting started.
612

7-
# Workflow
13+
# Development Workflow
814

915
## High-Level Problem Solving Strategy
1016

.github/workflows/copilot-setup-steps.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ jobs:
2323
contents: read
2424

2525
steps:
26-
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
26+
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
2727
- uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6
2828
with:
2929
go-version-file: 'go.mod'

.github/workflows/test.yml

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
runs-on: ubuntu-latest
1919
timeout-minutes: 5
2020
steps:
21-
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
21+
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
2222
- uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6
2323
with:
2424
go-version-file: 'go.mod'
@@ -34,7 +34,7 @@ jobs:
3434
name: Lint
3535
runs-on: ubuntu-latest
3636
steps:
37-
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
37+
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
3838
- uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6
3939
with:
4040
go-version-file: 'go.mod'
@@ -63,14 +63,16 @@ jobs:
6363
xpack.security.enabled: true
6464
xpack.security.authc.api_key.enabled: true
6565
xpack.security.authc.token.enabled: true
66+
xpack.ml.use_auto_machine_memory_percent: true
67+
xpack.ml.max_model_memory_limit: 2g
6668
xpack.watcher.enabled: true
6769
xpack.license.self_generated.type: trial
6870
repositories.url.allowed_urls: https://example.com/*
6971
path.repo: /tmp
7072
ELASTIC_PASSWORD: ${{ env.ELASTIC_PASSWORD }}
7173
ports:
7274
- 9200:9200
73-
options: --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10
75+
options: --memory=2g --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10
7476
kibana:
7577
image: docker.elastic.co/kibana/kibana:${{ matrix.version }}
7678
env:
@@ -137,7 +139,7 @@ jobs:
137139
- version: '8.4.3'
138140
runner: ubuntu-22.04
139141
steps:
140-
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
142+
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
141143
- uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6
142144
with:
143145
go-version-file: 'go.mod'

CHANGELOG.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,16 @@
11
## [Unreleased]
22

3+
## [0.12.2] - 2025-11-19
34
- Fix `elasticstack_elasticsearch_snapshot_lifecycle` metadata type conversion causing terraform apply to fail ([#1409](https://github.com/elastic/terraform-provider-elasticstack/issues/1409))
45
- Add new `elasticstack_elasticsearch_ml_anomaly_detection_job` resource ([#1329](https://github.com/elastic/terraform-provider-elasticstack/pull/1329))
5-
- Add new `elasticstack_elasticsearch_ml_datafeed` resource ([1340](https://github.com/elastic/terraform-provider-elasticstack/pull/1340))
6+
- Add new `elasticstack_elasticsearch_ml_datafeed` resource ([#1340](https://github.com/elastic/terraform-provider-elasticstack/pull/1340))
67
- Add `space_ids` attribute to all Fleet resources to support space-aware Fleet resource management ([#1390](https://github.com/elastic/terraform-provider-elasticstack/pull/1390))
8+
- Add back missing import support for `elasticstack_elasticsearch_security_role_mapping` ([#1441](https://github.com/elastic/terraform-provider-elasticstack/pull/1441))
9+
- Add new `elasticstack_elasticsearch_ml_job_state` resource ([#1337](https://github.com/elastic/terraform-provider-elasticstack/pull/1337))
10+
- Add new `elasticstack_elasticsearch_ml_datafeed_state` resource ([#1422](https://github.com/elastic/terraform-provider-elasticstack/pull/1422))
11+
- Add `output_id` to `elasticstack_fleet_integration_policy` resource ([#1445](https://github.com/elastic/terraform-provider-elasticstack/pull/1445))
12+
- Make `hosts` attribute required in `elasticstack_fleet_output` resource ([#1450](https://github.com/elastic/terraform-provider-elasticstack/pull/1450/files))
13+
- Fix `elasticstack_kibana_security_detection_rule` to properly respect `space_id`
714

815
## [0.12.1] - 2025-10-22
916
- Fix regression restricting the characters in an `elasticstack_elasticsearch_role_mapping` `name`. ([#1373](https://github.com/elastic/terraform-provider-elasticstack/pull/1373))
@@ -532,7 +539,8 @@ resource "elasticstack_fleet_output" "output" {
532539
- Initial set of docs
533540
- CI integration
534541
535-
[Unreleased]: https://github.com/elastic/terraform-provider-elasticstack/compare/v0.12.1...HEAD
542+
[Unreleased]: https://github.com/elastic/terraform-provider-elasticstack/compare/v0.12.2...HEAD
543+
[0.12.2]: https://github.com/elastic/terraform-provider-elasticstack/compare/v0.12.1...v0.12.2
536544
[0.12.1]: https://github.com/elastic/terraform-provider-elasticstack/compare/v0.12.0...v0.12.1
537545
[0.12.0]: https://github.com/elastic/terraform-provider-elasticstack/compare/v0.11.18...v0.12.0
538546
[0.11.19]: https://github.com/elastic/terraform-provider-elasticstack/compare/v0.11.18...v0.11.19

CODING_STANDARDS.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Coding Standards
2+
3+
This document outlines the coding standards and conventions used in the terraform-provider-elasticstack repository.
4+
5+
## General Principles
6+
7+
- Write idiomatic Go.
8+
- [Effective Go](https://go.dev/doc/effective_go)
9+
- [Code Review Comments](https://go.dev/wiki/CodeReviewComments)
10+
- The [Google Styleguide](https://google.github.io/styleguide/go/index#about)
11+
12+
## Project Structure
13+
14+
- Use the Plugin Framework for all new resources (not SDKv2)
15+
- Follow the code organization pattern of [the `system_user` resource](./internal/elasticsearch/security/system_user) for new Plugin Framework resources
16+
- [`testdata/`](./internal/elasticsearch/security/system_user/testdata) - This directory contains Terraform definitions used within the resource acceptance tests. In most cases, this will contain a subdirectory for each test, which then contain subdirectories for individual named test steps.
17+
- [`acc_test.go`](./internal/elasticsearch/security/system_user/acc_test.go) - Contains acceptance tests for the resource
18+
- [`create.go`](./internal/elasticsearch/security/system_user/create.go) - Contains the resources `Create` method and any required logic. Depending on the underlying API, the create and update handlers may share a single code path.
19+
- [`delete.go`](./internal/elasticsearch/security/system_user/delete.go) - Contains the resources `Delete` method.
20+
- [`models.go`](./internal/elasticsearch/security/system_user/models.go) - Contains Golang models used by the resource. At a minimum this will contain a model for reading plan/config/state from the Terraform plugin framework. Any non-trivial models should also define receivers for translating between Terraform models and API client models.
21+
- [`read.go`](./internal/elasticsearch/security/system_user/read.go) - Contains the resources `Read` method. This should also define an internal `read` function that can be re-used by the create/update paths to populate the final Terraform state after performing the create/update operation.
22+
- [`resource.go`](./internal/elasticsearch/security/system_user/resource.go) - Contains:
23+
- A factory function for creating the resource (e.g `NewSystemUserResource`)
24+
- `Metadata`, `Configure`, and optionally `ImportState` functions.
25+
- Type assertions ensuring the resource fully implement the relevant Plugin Framework interfaces (e.g `var _ resource.ResourceWithConfigure = &systemUserResource{}`)
26+
- [`schema.go`](./internal/elasticsearch/security/system_user/schema.go) - Contains the `Schema` function fully defining the resources schema
27+
- [`update.go`](./internal/elasticsearch/security/system_user/update.go) - Contains the `Update` method. Depending on the underlying API this may share significant logic with the `Create` method.
28+
- Some resources may define other files, for example:
29+
- [`models_*.go`](./internal/kibana/security_detection_rule/) - Complex APIs may result in significant model related logic. Split these files as appropriate if they become large.
30+
- Custom [plan modifiers](./internal/elasticsearch/security/api_key/set_unknown_if_access_has_changes.go), [validators](./internal/elasticsearch/security/api_key/validators.go) and [types](./internal/elasticsearch/security/api_key/role_descriptor_defaults.go) - Resource specific plan modifiers and custom types should be contained within the resource package.
31+
- [`state_upgrade.go`](./internal/elasticsearch/security/api_key/state_upgrade.go) - Resources requiring state upgrades should place the `UpgradeState` method within this file.
32+
- Avoid adding extra functionality to the existing `utils` package. Instead:
33+
- Code should live as close to the consumers.
34+
- Resource, area, application specific shared logic should live at that level. For example within `internal/kibana` for Kibana specific shared logic.
35+
- Provider wide shared logic should be packaged together by a logical concept. For example [diagutil](./internal/diagutil) contains shared code for managing Terraform Diagnostics, and translating between errors, SDKv2 diags, and Plugin Framework diags.
36+
- Prefer using existing util functions over longer form, duplicated code:
37+
- `utils.IsKnown(val)` instead of `!val.IsNull() && !val.IsUnknown()`
38+
- `utils.ListTypeAs` instead of `val.ElementsAs` or similar for other collection types
39+
40+
## Schema Definitions
41+
42+
- Use custom types to model attribute specific behaviour.
43+
- Use [`jsontypes.NormalizedType{}`](https://github.com/hashicorp/terraform-plugin-framework-jsontypes/blob/main/jsontypes/normalized_type.go) custom type for string attributes containing JSON blobs.
44+
- Use [`customtypes.DurationType{}`](./internal/utils/customtypes/duration_type.go) for duration-based string attributes.
45+
- Use [`customtypes.JSONWithDefaultsType{}`](./internal/utils/customtypes/json_with_defaults_type.go) to allow users to specify only a subset of a JSON blob.
46+
- Always include comprehensive descriptions for all resources, and attributes.
47+
- Long, multiline descriptions should be stored in an external markdown file, which is imported via Golang embedding. For [example](./internal/elasticsearch/security/system_user/resource-description.md).
48+
- Use schema validation wherever possible. Only perform validation within create/read/update functions as a last resort.
49+
- For example, any validation that relies on the actual Elastic Stack components (e.g Elasticsearch version)
50+
can only be performed during the create/read/update phase.
51+
- Kibana and Fleet resources will be backed by the Kibana API. The schema definition should closely follow the defined API request/response models defined in the [OpenAPI specification](./generated/kbapi/oas-filtered.yaml).
52+
- Further details may be found in the [API documentation](https://www.elastic.co/docs/api/doc/kibana/v9/)
53+
- Elasticsearch resources will be backed by the [go-elasticsearch](https://github.com/elastic/go-elasticsearch) client.
54+
- Further details may be found in the [API documentation](https://www.elastic.co/docs/api/doc/elasticsearch/)
55+
- Use `EnforceMinVersion` to ensure the backing Elastic Stack applications support the defined fields.
56+
- The provider supports a wide range of Stack versions, and so newer features will not be available in all versions.
57+
- See [`assertKafkaSupport`](./internal/fleet/output/models.go) for an example of how to handle the use of unsupported attributes.
58+
59+
60+
## JSON Handling
61+
62+
- Use [`jsontypes.NormalizedType{}`](https://github.com/hashicorp/terraform-plugin-framework-jsontypes/blob/main/jsontypes/normalized_type.go) for JSON string attributes to ensure proper normalization and comparison.
63+
- Use [`customtypes.JSONWithDefaultsType{}`](./internal/utils/customtypes/json_with_defaults_type.go) if API level defaults may be applied automatically.
64+
65+
## Testing
66+
67+
- Use table-driven unit tests when possible with `t.Run()` for test cases
68+
- Use testify library (`assert`, `require`) for test assertions
69+
- Ensure that *every* resource attribute is covered by at least one acceptance test case whenever possible.
70+
- Features that *require* external services are likely the only excuse to not include acceptance test coverage.
71+
- Organize acceptance tests in `acc_test.go` files
72+
- Test Terraform code should be vanilla, valid Terraform
73+
- Store test Terraform modules in `testdata/<test_name>/<step_description>` directories.
74+
- Define any required variables within the module
75+
- Reference the test code via `ConfigDirectory: acctest.NamedTestCaseDirectory("<step description>")`
76+
- Define any required variables via `ConfigVariables`
77+
78+
## API Client Usage
79+
80+
- Use generated API clients from [`generated/kbapi/`](./generated/kbapi/) for new Kibana API interactions
81+
- Avoid deprecated clients (`libs/go-kibana-rest`, `generated/alerting`, `generated/connectors`, `generated/slo`)

Makefile

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
.DEFAULT_GOAL = help
22
SHELL := /bin/bash
33

4-
VERSION ?= 0.12.1
4+
VERSION ?= 0.12.2
55

66
NAME = elasticstack
77
BINARY = terraform-provider-${NAME}
@@ -101,7 +101,7 @@ setup-kibana-fleet: ## Creates the agent and integration policies required to ru
101101

102102
.PHONY: docker-clean
103103
docker-clean: ## Try to remove provisioned nodes and assigned network
104-
@ docker compose -f $(COMPOSE_FILE) down
104+
@ docker compose -f $(COMPOSE_FILE) down -v
105105

106106
.PHONY: copy-kibana-ca
107107
copy-kibana-ca: ## Copy Kibana CA certificate to local machine
@@ -130,7 +130,7 @@ install: build ## Install built provider into the local terraform cache
130130

131131
.PHONY: tools
132132
tools: $(GOBIN) ## Download golangci-lint locally if necessary.
133-
@[[ -f $(GOBIN)/golangci-lint ]] || curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(GOBIN) v2.6.1
133+
@[[ -f $(GOBIN)/golangci-lint ]] || curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(GOBIN) v2.6.2
134134

135135
.PHONY: golangci-lint
136136
golangci-lint:

docker-compose.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ services:
1313
xpack.security.http.ssl.enabled: false
1414
xpack.license.self_generated.type: trial
1515
xpack.ml.use_auto_machine_memory_percent: true
16+
xpack.ml.max_model_memory_limit: 2g
1617
xpack.security.authc.api_key.enabled: true
1718
xpack.security.authc.token.enabled: true
1819
xpack.watcher.enabled: true

0 commit comments

Comments
 (0)