Skip to content

Commit cb71318

Browse files
authored
Merge pull request #18 from b3ngriffiths/claude/add-v2-feed-2026
feat: add V2 feed support
2 parents 11889e5 + f168ddc commit cb71318

45 files changed

Lines changed: 3174 additions & 56 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,6 @@ website/vendor
3333
!command/test-fixtures/**/.terraform/
3434

3535
dist/
36+
37+
# Compiled provider binary
38+
terraform-provider-chronicle

.golangci.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,3 +60,4 @@ issues:
6060
linters:
6161
- stylecheck
6262
- unused
63+
- unparam

GNUmakefile

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,17 @@ test: fmtcheck
1818
go test -v . ./chronicle
1919

2020
testacc: fmtcheck
21+
@echo "WARNING: Running both V1 and V2 feed tests together. This may fail if your Chronicle SIEM"
22+
@echo " has both V1 and V2 feeds active. Consider using 'make testacc-v1' or 'make testacc-v2' instead."
23+
@echo ""
2124
TF_ACC=1 go test -v ./chronicle -timeout 120m -parallel 1
2225

26+
testacc-v1: fmtcheck
27+
TF_ACC=1 go test -v ./chronicle -run='TestAcc' -skip='V2|EventDriven' -timeout 120m -parallel 1
28+
29+
testacc-v2: fmtcheck
30+
TF_ACC=1 go test -v ./chronicle -run='TestAcc.*(V2|EventDriven)' -timeout 120m -parallel 1
31+
2332
build:
2433
@go build -mod=vendor -o $(PROJECT_NAME)
2534
@echo "Build succeeded"
@@ -55,4 +64,4 @@ docs:
5564
vendor:
5665
@go mod tidy && go mod vendor && go mod verify
5766

58-
.PHONY: build install lint test clean testacc vet fmt fmtcheck docs vendor
67+
.PHONY: build install lint test clean testacc testacc-v1 testacc-v2 vet fmt fmtcheck docs vendor

README.md

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
[![CI](https://github.com/form3tech-oss/terraform-provider-chronicle/actions/workflows/ci.yaml/badge.svg)](https://github.com/form3tech-oss/terraform-provider-chronicle/actions/workflows/ci.yaml)
99
[![release](https://github.com/form3tech-oss/terraform-provider-chronicle/actions/workflows/release.yaml/badge.svg)](https://github.com/form3tech-oss/terraform-provider-chronicle/actions/workflows/release.yaml)
1010

11-
Terraform provider for Chronicle
11+
Terraform provider for Chronicle (now known as Google Security Operations SIEM)
1212

1313
# Documentation
1414

@@ -52,7 +52,32 @@ In order to test the provider, you can simply run `make test`.
5252
make test
5353
```
5454

55-
In order to run the full suite of Acceptance tests, set the environment variables listed below and run `make testacc`.
55+
### Running Acceptance Tests
56+
57+
Set the required environment variables and use one of these make targets:
58+
59+
- `make testacc-v1` - Run V1 feed tests only
60+
- `make testacc-v2` - Run V2 feed tests only
61+
- `make testacc` - Run all tests (see limitations below)
62+
63+
#### V1 vs V2 Feed Testing Constraints
64+
65+
**Important:** Google Chronicle SIEM instances can only create **one feed version at a time** (either V1 or V2), which affects how you run acceptance tests.
66+
67+
**How feed versions work:**
68+
- When you switch a SIEM to V2 feeds, you can no longer **create** new V1 feeds
69+
- However, any **existing** V1 feeds remain active and continue running (at time of writing)
70+
- This means you cannot run the full `make testacc` suite against a single SIEM instance
71+
72+
**Recommended approach:**
73+
74+
Match your test target to your SIEM configuration:
75+
- **V1-configured SIEM** → use `make testacc-v1`
76+
- **V2-configured SIEM** → use `make testacc-v2`
77+
78+
Running the wrong test suite will fail because the SIEM won't allow creating feeds of the non-configured version.
79+
80+
**Note:** You could theoretically run `make testacc` with multiple SIEM instances (one V1-configured, one V2-configured), but this setup is untested.
5681

5782
The order of precedence for chronicle's API configuration is the following: `Credential file through TF > Access Token through TF > Environment Variable`.
5883
Environment variables always take the lowest precedence

chronicle/provider.go

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -195,14 +195,19 @@ func Provider() *schema.Provider {
195195
"chronicle_rule": resourceRule(),
196196
"chronicle_reference_list": resourceReferenceList(),
197197
"chronicle_feed_amazon_s3": NewResourceFeedAmazonS3().TerraformResource,
198+
"chronicle_feed_amazon_s3_v2": NewResourceFeedAmazonS3V2().TerraformResource,
198199
"chronicle_feed_amazon_sqs": NewResourceFeedAmazonSQS().TerraformResource,
200+
"chronicle_feed_amazon_sqs_v2": NewResourceFeedAmazonSQSV2().TerraformResource,
199201
"chronicle_feed_qualys_vm": NewResourceFeedQualysVM().TerraformResource,
200202
"chronicle_feed_microsoft_office_365_management_activity": NewResourceFeedMicrosoftOffice365ManagementActivity().TerraformResource,
201203
"chronicle_feed_okta_system_log": NewResourceFeedOktaSystemLog().TerraformResource,
202204
"chronicle_feed_okta_users": NewResourceFeedOktaUsers().TerraformResource,
203205
"chronicle_feed_proofpoint_siem": NewResourceFeedProofpointSIEM().TerraformResource,
204206
"chronicle_feed_google_cloud_storage_bucket": NewResourceFeedGoogleCloudStorageBucket().TerraformResource,
207+
"chronicle_feed_google_cloud_storage_v2": NewResourceFeedGoogleCloudStorageV2().TerraformResource,
208+
"chronicle_feed_google_cloud_storage_event_driven": NewResourceFeedGoogleCloudStorageEventDriven().TerraformResource,
205209
"chronicle_feed_azure_blobstore": NewResourceFeedAzureBlobStore().TerraformResource,
210+
"chronicle_feed_azure_blobstore_v2": NewResourceFeedAzureBlobStoreV2().TerraformResource,
206211
"chronicle_feed_thinkst_canary": NewResourceFeedThinkstCanary().TerraformResource,
207212
},
208213
}
@@ -272,6 +277,9 @@ func providerConfigure(ctx context.Context, d *schema.ResourceData, p *schema.Pr
272277
if endpoint, isCustom := customEndpoint(d, "rule_custom_endpoint"); isCustom {
273278
client.WithRuleBasePath(endpoint)
274279
}
280+
if endpoint, isCustom := customEndpoint(d, "feed_custom_endpoint"); isCustom {
281+
client.WithFeedManagementBasePath(endpoint)
282+
}
275283
if endpoint, isCustom := customEndpoint(d, "subjects_custom_endpoint"); isCustom {
276284
client.WithSubjectsBasePath(endpoint)
277285
}
@@ -295,7 +303,7 @@ func getAPIAuthOpts(d *schema.ResourceData) []chronicle.Option {
295303

296304
if v, ok := d.GetOk("backstoryapi_credentials"); ok {
297305
opts = append(opts, chronicle.WithBackstoryAPICredentials(v.(string)))
298-
} else if v, ok := d.GetOk("backstoryapi_credentials"); ok {
306+
} else if v, ok := d.GetOk("backstoryapi_access_token"); ok {
299307
opts = append(opts, chronicle.WithBackstoryAPIAccessToken(v.(string)))
300308
} else {
301309
env := envSearch(chronicle.BackstoryAPIEnvVar)
@@ -306,7 +314,7 @@ func getAPIAuthOpts(d *schema.ResourceData) []chronicle.Option {
306314

307315
if v, ok := d.GetOk("ingestionapi_credentials"); ok {
308316
opts = append(opts, chronicle.WithIngestionAPICredentials(v.(string)))
309-
} else if v, ok := d.GetOk("ingestionapi_credentials"); ok {
317+
} else if v, ok := d.GetOk("ingestionapi_access_token"); ok {
310318
opts = append(opts, chronicle.WithIngestionAPIAccessToken(v.(string)))
311319
} else {
312320
env := envSearch(chronicle.IngestionAPIEnvVar)
@@ -322,7 +330,7 @@ func getAPIAuthOpts(d *schema.ResourceData) []chronicle.Option {
322330
} else {
323331
env := envSearch(chronicle.ForwarderAPIEnvVar)
324332
if env != "" {
325-
opts = append(opts, chronicle.WithBigQueryAPIEnvVar())
333+
opts = append(opts, chronicle.WithForwarderAPIEnvVar())
326334
}
327335
}
328336

chronicle/resource_feed_amazon_s3_test.go

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -200,7 +200,6 @@ func TestAccChronicleFeedAmazonS3_UpdateLogType(t *testing.T) {
200200
})
201201
}
202202

203-
//nolint:unparam
204203
func testAccCheckChronicleFeedAmazonS3AuthUpdated(t *testing.T, n, region, accessKeyID, secretAccessKey string) resource.TestCheckFunc {
205204
return func(s *terraform.State) error {
206205
rs, ok := s.RootModule().Resources[n]
@@ -223,7 +222,6 @@ func testAccCheckChronicleFeedAmazonS3AuthUpdated(t *testing.T, n, region, acces
223222
}
224223
}
225224

226-
//nolint:unparam
227225
func testAccCheckChronicleFeedAmazonS3(displayName, logType, enabled, namespace, labels, s3Uri, s3SourceType,
228226
sourceDeleteOptions, region, accesKeyID, secretAccessKey string) string {
229227
return fmt.Sprintf(
@@ -265,7 +263,7 @@ func testAccCheckChronicleFeedAmazonS3Exists(n string) resource.TestCheckFunc {
265263

266264
func testAccCheckChronicleFeedAmazonS3Destroy(s *terraform.State) error {
267265
for _, rs := range s.RootModule().Resources {
268-
if rs.Type != "chronicle_feed_amazon_s3.test" {
266+
if rs.Type != "chronicle_feed_amazon_s3" {
269267
continue
270268
}
271269

@@ -277,7 +275,6 @@ func testAccCheckChronicleFeedAmazonS3Destroy(s *terraform.State) error {
277275
return nil
278276
}
279277

280-
//nolint:unparam
281278
func feedAmazonS3Ref(name string) string {
282279
return fmt.Sprintf("chronicle_feed_amazon_s3.%v", name)
283280
}
Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
package chronicle
2+
3+
import (
4+
chronicle "github.com/form3tech-oss/terraform-provider-chronicle/client"
5+
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
6+
)
7+
8+
type ResourceFeedAmazonS3V2 struct {
9+
TerraformResource *schema.Resource
10+
}
11+
12+
func NewResourceFeedAmazonS3V2() *ResourceFeedAmazonS3V2 {
13+
details := &schema.Resource{
14+
Schema: map[string]*schema.Schema{
15+
"s3_uri": {
16+
Type: schema.TypeString,
17+
Required: true,
18+
Description: `The S3 bucket URI in the format s3://bucket-name/path/.`,
19+
},
20+
"source_delete_options": {
21+
Type: schema.TypeString,
22+
ValidateDiagFunc: validateFeedV2SourceDeleteOption,
23+
Required: true,
24+
Description: `Whether to delete source files after they have been transferred to Chronicle. Valid values are:
25+
26+
- NEVER: Never delete files from the source.
27+
- ON_SUCCESS: Delete files and empty directories from the source after successful ingestion.`,
28+
},
29+
"max_lookback_days": {
30+
Type: schema.TypeInt,
31+
Optional: true,
32+
Default: 180,
33+
ValidateDiagFunc: validateMaxLookbackDays,
34+
Description: `The maximum number of days in the past to look for files. Must be between 1 and 180. Default is 180 days.`,
35+
},
36+
"authentication": {
37+
Type: schema.TypeList,
38+
Required: true,
39+
MaxItems: 1,
40+
Description: `AWS authentication details. Use either access key credentials or IAM role ARN.`,
41+
Elem: &schema.Resource{
42+
Schema: map[string]*schema.Schema{
43+
"access_key_id": {
44+
Type: schema.TypeString,
45+
Optional: true,
46+
ValidateDiagFunc: validateAWSAccessKeyID,
47+
RequiredWith: []string{"details.0.authentication.0.secret_access_key"},
48+
ConflictsWith: []string{"details.0.authentication.0.aws_iam_role_arn"},
49+
AtLeastOneOf: []string{
50+
"details.0.authentication.0.access_key_id",
51+
"details.0.authentication.0.aws_iam_role_arn",
52+
},
53+
Description: `The 20-character access key ID associated with your Amazon IAM account. Required if not using aws_iam_role_arn.`,
54+
},
55+
"secret_access_key": {
56+
Type: schema.TypeString,
57+
Optional: true,
58+
Sensitive: true,
59+
ValidateDiagFunc: validateAWSSecretAccessKey,
60+
RequiredWith: []string{"details.0.authentication.0.access_key_id"},
61+
ConflictsWith: []string{"details.0.authentication.0.aws_iam_role_arn"},
62+
Description: `The 40-character secret access key associated with your Amazon IAM account. Required if not using aws_iam_role_arn.`,
63+
},
64+
"aws_iam_role_arn": {
65+
Type: schema.TypeString,
66+
Optional: true,
67+
ConflictsWith: []string{"details.0.authentication.0.access_key_id", "details.0.authentication.0.secret_access_key"},
68+
AtLeastOneOf: []string{
69+
"details.0.authentication.0.access_key_id",
70+
"details.0.authentication.0.aws_iam_role_arn",
71+
},
72+
Description: `ARN of the AWS IAM role configured to access S3 bucket. Use this for federated authentication instead of access keys.`,
73+
},
74+
},
75+
},
76+
},
77+
},
78+
}
79+
description := "Creates a V2 feed from Amazon Simple Storage Service (S3). " +
80+
"This feed type uses the Google Cloud Storage Transfer Service for improved ingestion."
81+
resource := &ResourceFeedAmazonS3V2{}
82+
resource.TerraformResource = newFeedResourceSchema(details, resource, description, true)
83+
84+
return resource
85+
}
86+
87+
func (f *ResourceFeedAmazonS3V2) getLogType() string {
88+
return ""
89+
}
90+
91+
func (f *ResourceFeedAmazonS3V2) expandConcreteFeedConfiguration(d *schema.ResourceData) chronicle.ConcreteFeedConfiguration {
92+
resourceDetailsInterface := readSliceFromResource(d, "details")
93+
if resourceDetailsInterface == nil {
94+
return nil
95+
}
96+
97+
resourceDetails := resourceDetailsInterface[0].(map[string]interface{})
98+
authenticationDetails := resourceDetails["authentication"].([]interface{})[0].(map[string]interface{})
99+
100+
config := &chronicle.S3V2FeedConfiguration{
101+
S3URI: resourceDetails["s3_uri"].(string),
102+
SourceDeleteOptions: resourceDetails["source_delete_options"].(string),
103+
MaxLookbackDays: resourceDetails["max_lookback_days"].(int),
104+
Authentication: chronicle.S3V2FeedAuthentication{},
105+
}
106+
107+
// Check which authentication method is used
108+
if iamRoleArn, ok := authenticationDetails["aws_iam_role_arn"].(string); ok && iamRoleArn != "" {
109+
config.Authentication.AWSIAMRoleAuth = &chronicle.S3V2AWSIAMRoleAuth{
110+
AWSIAMRoleArn: iamRoleArn,
111+
}
112+
} else {
113+
config.Authentication.AccessKeySecretAuth = &chronicle.S3V2AccessKeySecretAuth{
114+
AccessKeyID: authenticationDetails["access_key_id"].(string),
115+
SecretAccessKey: authenticationDetails["secret_access_key"].(string),
116+
}
117+
}
118+
119+
return config
120+
}
121+
122+
//nolint:all
123+
func (f *ResourceFeedAmazonS3V2) flattenDetailsFromReadOperation(originalConf chronicle.ConcreteFeedConfiguration, readConf chronicle.ConcreteFeedConfiguration) []map[string]interface{} {
124+
125+
readS3Conf := readConf.(*chronicle.S3V2FeedConfiguration)
126+
127+
// Import Case
128+
if originalConf == nil {
129+
authMap := make(map[string]interface{})
130+
// Only populate non-secret auth fields during import
131+
if readS3Conf.Authentication.AWSIAMRoleAuth != nil && readS3Conf.Authentication.AWSIAMRoleAuth.AWSIAMRoleArn != "" {
132+
authMap["aws_iam_role_arn"] = readS3Conf.Authentication.AWSIAMRoleAuth.AWSIAMRoleArn
133+
}
134+
135+
// Note: access_key_id and secret_access_key are not returned by the API
136+
// and will remain empty in state after import until explicitly set by user
137+
138+
return []map[string]interface{}{{
139+
"s3_uri": readS3Conf.S3URI,
140+
"source_delete_options": readS3Conf.SourceDeleteOptions,
141+
"max_lookback_days": readS3Conf.MaxLookbackDays,
142+
"authentication": []map[string]interface{}{authMap},
143+
}}
144+
}
145+
146+
originalS3Conf := originalConf.(*chronicle.S3V2FeedConfiguration)
147+
// Default Case
148+
authMap := make(map[string]interface{})
149+
if originalS3Conf.Authentication.AWSIAMRoleAuth != nil && originalS3Conf.Authentication.AWSIAMRoleAuth.AWSIAMRoleArn != "" {
150+
authMap["aws_iam_role_arn"] = originalS3Conf.Authentication.AWSIAMRoleAuth.AWSIAMRoleArn
151+
}
152+
if originalS3Conf.Authentication.AccessKeySecretAuth != nil {
153+
authMap["access_key_id"] = originalS3Conf.Authentication.AccessKeySecretAuth.AccessKeyID
154+
authMap["secret_access_key"] = originalS3Conf.Authentication.AccessKeySecretAuth.SecretAccessKey
155+
}
156+
157+
return []map[string]interface{}{{
158+
"s3_uri": readS3Conf.S3URI,
159+
"source_delete_options": originalS3Conf.SourceDeleteOptions, // not returned
160+
"max_lookback_days": readS3Conf.MaxLookbackDays,
161+
// replace authentication block with original values because they are not returned within a read request
162+
"authentication": []map[string]interface{}{authMap},
163+
}}
164+
}

0 commit comments

Comments
 (0)