Skip to content

Commit 94dde4f

Browse files
Copilottobio
andauthored
Migrate Elasticsearch enrich policy resource and data source to Terraform Plugin Framework (#1220)
* Initial plan * Implement Plugin Framework migration for enrich policy resource and data source Co-authored-by: tobio <[email protected]> * Final validation - enrich policy migration complete and tested Co-authored-by: tobio <[email protected]> * Address PR feedback: refactor code structure and remove old SDK implementation Co-authored-by: tobio <[email protected]> * Fix code formatting for enrich policy data source Co-authored-by: tobio <[email protected]> * Regenerate documentation for enrich policy migration Co-authored-by: tobio <[email protected]> * Use jsontypes.NormalizedType for query field and extract duplicate read logic Co-authored-by: tobio <[email protected]> * Refactor EnrichPolicy models to separate resource and data source types, fix test assertion Co-authored-by: tobio <[email protected]> * Change indices and enrich_fields to SetAttribute, update data source description Co-authored-by: tobio <[email protected]> * Update documentation after changing indices and enrich_fields to SetAttribute Co-authored-by: tobio <[email protected]> * PR feedback --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: tobio <[email protected]> Co-authored-by: Toby Brain <[email protected]>
1 parent 85cf3f4 commit 94dde4f

File tree

18 files changed

+808
-464
lines changed

18 files changed

+808
-464
lines changed

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
- Add support for `solution` field in `elasticstack_kibana_space` resource and data source ([#1102](https://github.com/elastic/terraform-provider-elasticstack/issues/1102))
44
- Add `slo_id` validation to `elasticstack_kibana_slo` ([#1221](https://github.com/elastic/terraform-provider-elasticstack/pull/1221))
55
- Add `ignore_missing_component_templates` to `elasticstack_elasticsearch_index_template` ([#1206](https://github.com/elastic/terraform-provider-elasticstack/pull/1206))
6+
- Migrate `elasticstack_elasticsearch_enrich_policy` resource and data source to Terraform Plugin Framework ([#1220](https://github.com/elastic/terraform-provider-elasticstack/pull/1220))
67
- Prevent provider panic when a script exists in state, but not in Elasticsearch ([#1218](https://github.com/elastic/terraform-provider-elasticstack/pull/1218))
78
- Allow version changes without a destroy/create cycle with `elasticstack_fleet_integration` ([#1255](https://github.com/elastic/terraform-provider-elasticstack/pull/1255)). This fixes an issue where it was impossible to upgrade integrations which are used by an integration policy.
89
- Add `namespace` attribute to `elasticstack_kibana_synthetics_monitor` resource to support setting data stream namespace independently from `space_id` ([#1247](https://github.com/elastic/terraform-provider-elasticstack/pull/1247))

docs/data-sources/elasticsearch_enrich_policy.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,11 +74,35 @@ output "query" {
7474

7575
- `name` (String) The name of the policy.
7676

77+
### Optional
78+
79+
- `elasticsearch_connection` (Block List, Deprecated) Elasticsearch connection configuration block. (see [below for nested schema](#nestedblock--elasticsearch_connection))
80+
7781
### Read-Only
7882

7983
- `enrich_fields` (Set of String) Fields to add to matching incoming documents. These fields must be present in the source indices.
8084
- `id` (String) Internal identifier of the resource
8185
- `indices` (Set of String) Array of one or more source indices used to create the enrich index.
82-
- `match_field` (String) Field in source indices used to match incoming documents.
86+
- `match_field` (String) Field from the source indices used to match incoming documents.
8387
- `policy_type` (String) The type of enrich policy, can be one of geo_match, match, range.
8488
- `query` (String) Query used to filter documents in the enrich index. The policy only uses documents matching this query to enrich incoming documents. Defaults to a match_all query.
89+
90+
<a id="nestedblock--elasticsearch_connection"></a>
91+
### Nested Schema for `elasticsearch_connection`
92+
93+
Optional:
94+
95+
- `api_key` (String, Sensitive) API Key to use for authentication to Elasticsearch
96+
- `bearer_token` (String, Sensitive) Bearer Token to use for authentication to Elasticsearch
97+
- `ca_data` (String) PEM-encoded custom Certificate Authority certificate
98+
- `ca_file` (String) Path to a custom Certificate Authority certificate
99+
- `cert_data` (String) PEM encoded certificate for client auth
100+
- `cert_file` (String) Path to a file containing the PEM encoded certificate for client auth
101+
- `endpoints` (List of String, Sensitive) A list of endpoints where the terraform provider will point to, this must include the http(s) schema and port number.
102+
- `es_client_authentication` (String, Sensitive) ES Client Authentication field to be used with the JWT token
103+
- `headers` (Map of String, Sensitive) A list of headers to be sent with each request to Elasticsearch.
104+
- `insecure` (Boolean) Disable TLS certificate validation
105+
- `key_data` (String, Sensitive) PEM encoded private key for client auth
106+
- `key_file` (String) Path to a file containing the PEM encoded private key for client auth
107+
- `password` (String, Sensitive) Password to use for API authentication to Elasticsearch.
108+
- `username` (String) Username to use for API authentication to Elasticsearch.

docs/resources/elasticsearch_enrich_policy.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,13 +58,13 @@ resource "elasticstack_elasticsearch_enrich_policy" "policy1" {
5858

5959
### Optional
6060

61-
- `elasticsearch_connection` (Block List, Max: 1, Deprecated) Elasticsearch connection configuration block. This property will be removed in a future provider version. Configure the Elasticsearch connection via the provider configuration instead. (see [below for nested schema](#nestedblock--elasticsearch_connection))
61+
- `elasticsearch_connection` (Block List, Deprecated) Elasticsearch connection configuration block. (see [below for nested schema](#nestedblock--elasticsearch_connection))
6262
- `execute` (Boolean) Whether to call the execute API function in order to create the enrich index.
6363
- `query` (String) Query used to filter documents in the enrich index. The policy only uses documents matching this query to enrich incoming documents. Defaults to a match_all query.
6464

6565
### Read-Only
6666

67-
- `id` (String) The ID of this resource.
67+
- `id` (String) Internal identifier of the resource
6868

6969
<a id="nestedblock--elasticsearch_connection"></a>
7070
### Nested Schema for `elasticsearch_connection`
Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
package enrich_test
2+
3+
import (
4+
"encoding/json"
5+
"fmt"
6+
"net/http"
7+
"testing"
8+
9+
"github.com/elastic/terraform-provider-elasticstack/internal/acctest"
10+
"github.com/elastic/terraform-provider-elasticstack/internal/clients"
11+
sdkacctest "github.com/hashicorp/terraform-plugin-sdk/v2/helper/acctest"
12+
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
13+
"github.com/hashicorp/terraform-plugin-sdk/v2/terraform"
14+
)
15+
16+
func TestAccResourceEnrichPolicyFW(t *testing.T) {
17+
name := sdkacctest.RandStringFromCharSet(10, sdkacctest.CharSetAlphaNum)
18+
resource.Test(t, resource.TestCase{
19+
PreCheck: func() { acctest.PreCheck(t) },
20+
CheckDestroy: checkEnrichPolicyDestroyFW(name),
21+
ProtoV6ProviderFactories: acctest.Providers,
22+
Steps: []resource.TestStep{
23+
{
24+
Config: testAccEnrichPolicyFW(name),
25+
Check: resource.ComposeTestCheckFunc(
26+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "name", name),
27+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "policy_type", "match"),
28+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "match_field", `email`),
29+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "indices.0", name),
30+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "enrich_fields.0", "first_name"),
31+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "enrich_fields.1", "last_name"),
32+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "query", "{\"match_all\": {}}\n"),
33+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "execute", "true"),
34+
),
35+
},
36+
},
37+
})
38+
}
39+
40+
func TestAccDataSourceEnrichPolicyFW(t *testing.T) {
41+
name := sdkacctest.RandStringFromCharSet(10, sdkacctest.CharSetAlphaNum)
42+
resource.Test(t, resource.TestCase{
43+
PreCheck: func() { acctest.PreCheck(t) },
44+
ProtoV6ProviderFactories: acctest.Providers,
45+
Steps: []resource.TestStep{
46+
{
47+
Config: testAccEnrichPolicyDataSourceFW(name),
48+
Check: resource.ComposeTestCheckFunc(
49+
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "name", name),
50+
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "policy_type", "match"),
51+
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "match_field", "email"),
52+
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "indices.0", name),
53+
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "enrich_fields.0", "first_name"),
54+
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "enrich_fields.1", "last_name"),
55+
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "query", "{\"match_all\":{}}"),
56+
),
57+
},
58+
},
59+
})
60+
}
61+
62+
func TestAccResourceEnrichPolicyFromSDK(t *testing.T) {
63+
name := sdkacctest.RandStringFromCharSet(10, sdkacctest.CharSetAlphaNum)
64+
resource.Test(t, resource.TestCase{
65+
PreCheck: func() { acctest.PreCheck(t) },
66+
Steps: []resource.TestStep{
67+
{
68+
// Create the enrich policy with the last provider version where the enrich policy resource was built on the SDK
69+
ExternalProviders: map[string]resource.ExternalProvider{
70+
"elasticstack": {
71+
Source: "elastic/elasticstack",
72+
VersionConstraint: "0.11.17",
73+
},
74+
},
75+
Config: testAccEnrichPolicyFW(name),
76+
Check: resource.ComposeTestCheckFunc(
77+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "name", name),
78+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "policy_type", "match"),
79+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "execute", "true"),
80+
),
81+
},
82+
{
83+
ProtoV6ProviderFactories: acctest.Providers,
84+
Config: testAccEnrichPolicyFW(name),
85+
Check: resource.ComposeTestCheckFunc(
86+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "name", name),
87+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "policy_type", "match"),
88+
resource.TestCheckResourceAttr("elasticstack_elasticsearch_enrich_policy.policy", "execute", "true"),
89+
),
90+
},
91+
},
92+
})
93+
}
94+
95+
func testAccEnrichPolicyFW(name string) string {
96+
return fmt.Sprintf(`
97+
provider "elasticstack" {
98+
elasticsearch {}
99+
}
100+
101+
resource "elasticstack_elasticsearch_index" "my_index" {
102+
name = "%s"
103+
104+
mappings = jsonencode({
105+
properties = {
106+
email = { type = "text" }
107+
first_name = { type = "text" }
108+
last_name = { type = "text" }
109+
}
110+
})
111+
deletion_protection = false
112+
}
113+
114+
resource "elasticstack_elasticsearch_enrich_policy" "policy" {
115+
name = "%s"
116+
policy_type = "match"
117+
indices = [elasticstack_elasticsearch_index.my_index.name]
118+
match_field = "email"
119+
enrich_fields = ["first_name", "last_name"]
120+
query = <<-EOD
121+
{"match_all": {}}
122+
EOD
123+
}
124+
`, name, name)
125+
}
126+
127+
func testAccEnrichPolicyDataSourceFW(name string) string {
128+
return fmt.Sprintf(`
129+
provider "elasticstack" {
130+
elasticsearch {}
131+
}
132+
133+
resource "elasticstack_elasticsearch_index" "my_index" {
134+
name = "%s"
135+
136+
mappings = jsonencode({
137+
properties = {
138+
email = { type = "text" }
139+
first_name = { type = "text" }
140+
last_name = { type = "text" }
141+
}
142+
})
143+
deletion_protection = false
144+
}
145+
146+
resource "elasticstack_elasticsearch_enrich_policy" "policy" {
147+
name = "%s"
148+
policy_type = "match"
149+
indices = [elasticstack_elasticsearch_index.my_index.name]
150+
match_field = "email"
151+
enrich_fields = ["first_name", "last_name"]
152+
query = <<-EOD
153+
{"match_all": {}}
154+
EOD
155+
}
156+
157+
data "elasticstack_elasticsearch_enrich_policy" "test" {
158+
name = elasticstack_elasticsearch_enrich_policy.policy.name
159+
}
160+
`, name, name)
161+
}
162+
163+
func checkEnrichPolicyDestroyFW(name string) func(s *terraform.State) error {
164+
return func(s *terraform.State) error {
165+
client, err := clients.NewAcceptanceTestingClient()
166+
if err != nil {
167+
return err
168+
}
169+
170+
for _, rs := range s.RootModule().Resources {
171+
if rs.Type != "elasticstack_elasticsearch_enrich_policy" {
172+
continue
173+
}
174+
compId, _ := clients.CompositeIdFromStr(rs.Primary.ID)
175+
if compId.ResourceId != name {
176+
return fmt.Errorf("Found unexpectedly enrich policy: %s", compId.ResourceId)
177+
}
178+
esClient, err := client.GetESClient()
179+
if err != nil {
180+
return err
181+
}
182+
req := esClient.EnrichGetPolicy.WithName(compId.ResourceId)
183+
res, err := esClient.EnrichGetPolicy(req)
184+
if err != nil {
185+
return err
186+
}
187+
defer res.Body.Close()
188+
if res.StatusCode == http.StatusFound {
189+
var policiesResponse map[string]any
190+
if err := json.NewDecoder(res.Body).Decode(&policiesResponse); err != nil {
191+
return err
192+
}
193+
if len(policiesResponse["policies"].([]any)) != 0 {
194+
return fmt.Errorf("Enrich policy (%s) still exists", compId.ResourceId)
195+
}
196+
}
197+
}
198+
return nil
199+
}
200+
}
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
package enrich
2+
3+
import (
4+
"context"
5+
6+
"github.com/elastic/terraform-provider-elasticstack/internal/clients"
7+
"github.com/elastic/terraform-provider-elasticstack/internal/clients/elasticsearch"
8+
"github.com/elastic/terraform-provider-elasticstack/internal/models"
9+
"github.com/elastic/terraform-provider-elasticstack/internal/utils"
10+
"github.com/hashicorp/terraform-plugin-framework/diag"
11+
"github.com/hashicorp/terraform-plugin-framework/path"
12+
"github.com/hashicorp/terraform-plugin-framework/resource"
13+
"github.com/hashicorp/terraform-plugin-framework/tfsdk"
14+
"github.com/hashicorp/terraform-plugin-framework/types"
15+
)
16+
17+
func (r *enrichPolicyResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
18+
diags := r.upsert(ctx, req.Plan, &resp.State)
19+
resp.Diagnostics.Append(diags...)
20+
if resp.Diagnostics.HasError() {
21+
return
22+
}
23+
}
24+
25+
func (r *enrichPolicyResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
26+
diags := r.upsert(ctx, req.Plan, &resp.State)
27+
resp.Diagnostics.Append(diags...)
28+
if resp.Diagnostics.HasError() {
29+
return
30+
}
31+
}
32+
33+
func (r *enrichPolicyResource) upsert(ctx context.Context, plan tfsdk.Plan, state *tfsdk.State) diag.Diagnostics {
34+
var data EnrichPolicyDataWithExecute
35+
var diags diag.Diagnostics
36+
diags.Append(plan.Get(ctx, &data)...)
37+
if diags.HasError() {
38+
return diags
39+
}
40+
41+
policyName := data.Name.ValueString()
42+
id, sdkDiags := r.client.ID(ctx, policyName)
43+
diags.Append(utils.FrameworkDiagsFromSDK(sdkDiags)...)
44+
if diags.HasError() {
45+
return diags
46+
}
47+
48+
client, diags := clients.MaybeNewApiClientFromFrameworkResource(ctx, data.ElasticsearchConnection, r.client)
49+
diags.Append(diags...)
50+
if diags.HasError() {
51+
return diags
52+
}
53+
54+
// Convert framework types to model
55+
indices := utils.SetTypeAs[string](ctx, data.Indices, path.Empty(), &diags)
56+
if diags.HasError() {
57+
return diags
58+
}
59+
60+
enrichFields := utils.SetTypeAs[string](ctx, data.EnrichFields, path.Empty(), &diags)
61+
if diags.HasError() {
62+
return diags
63+
}
64+
65+
policy := &models.EnrichPolicy{
66+
Type: data.PolicyType.ValueString(),
67+
Name: policyName,
68+
Indices: indices,
69+
MatchField: data.MatchField.ValueString(),
70+
EnrichFields: enrichFields,
71+
}
72+
73+
if !data.Query.IsNull() && !data.Query.IsUnknown() {
74+
policy.Query = data.Query.ValueString()
75+
}
76+
77+
if sdkDiags := elasticsearch.PutEnrichPolicy(ctx, client, policy); sdkDiags.HasError() {
78+
diags.Append(utils.FrameworkDiagsFromSDK(sdkDiags)...)
79+
return diags
80+
}
81+
82+
data.Id = types.StringValue(id.String())
83+
84+
// Execute policy if requested
85+
if !data.Execute.IsNull() && !data.Execute.IsUnknown() && data.Execute.ValueBool() {
86+
if sdkDiags := elasticsearch.ExecuteEnrichPolicy(ctx, client, policyName); sdkDiags.HasError() {
87+
diags.Append(utils.FrameworkDiagsFromSDK(sdkDiags)...)
88+
return diags
89+
}
90+
}
91+
92+
diags.Append(state.Set(ctx, &data)...)
93+
return diags
94+
}

0 commit comments

Comments
 (0)