CSPL-4360 Fix failing integ and helm tests

Author: kasiakoziol

api/v4/objectstorage_types.go

@@ -55,7 +55,7 @@ type S3Spec struct {
 
 // ObjectStorageStatus defines the observed state of ObjectStorage.
 type ObjectStorageStatus struct {
-	// Phase of the large message store
+	// Phase of the object storage
 	Phase Phase `json:"phase"`
 
 	// Resource revision tracker

config/crd/bases/enterprise.splunk.com_objectstorages.yaml

@@ -87,7 +87,7 @@ spec:
                 description: Auxillary message describing CR status
                 type: string
               phase:
-                description: Phase of the large message store
+                description: Phase of the object storage
                 enum:
                 - Pending
                 - Ready

docs/CustomResources.md

@@ -404,21 +404,21 @@ spec:
     endpoint: https://s3.us-west-2.amazonaws.com
 ```
 
-ObjectStorage inputs can be found in the table below. As of now, only S3 provider of large message store is supported.
+ObjectStorage inputs can be found in the table below. As of now, only S3 provider of object storage is supported.
 
 | Key        | Type    | Description                                       |
 | ---------- | ------- | ------------------------------------------------- |
-| provider   | string | [Required] Provider of large message store (Allowed values: s3) |
-| s3   | S3 | [Required if provider=s3] S3 large message store inputs  |
+| provider   | string | [Required] Provider of object storage (Allowed values: s3) |
+| s3   | S3 | [Required if provider=s3] S3 object storage inputs  |
 
-S3 large message store inputs can be found in the table below.
+S3 object storage inputs can be found in the table below.
 
 | Key        | Type    | Description                                       |
 | ---------- | ------- | ------------------------------------------------- |
 | path   | string | [Required] Remote storage location for messages that are larger than the underlying maximum message size  |
 | endpoint   | string | [Optional, if not provided formed based on region] S3-compatible service endpoint
 
-Change of any of the large message queue inputs triggers the restart of Splunk so that appropriate .conf files are correctly refreshed and consumed.
+Change of any of the object storage inputs triggers the restart of Splunk so that appropriate .conf files are correctly refreshed and consumed.
 
 ## MonitoringConsole Resource Spec Parameters
 

docs/IndexIngestionSeparation.md

@@ -44,7 +44,7 @@ SQS message queue inputs can be found in the table below.
 | endpoint   | string | [Optional, if not provided formed based on region] AWS SQS Service endpoint
 | dlq   | string | [Required] Name of the dead letter queue |
 
-**First provisioning or update of any of the bus inputs requires Ingestor Cluster and Indexer Cluster Splunkd restart, but this restart is implemented automatically and done by SOK.**
+**First provisioning or update of any of the queue inputs requires Ingestor Cluster and Indexer Cluster Splunkd restart, but this restart is implemented automatically and done by SOK.**
 
 ## Example
 ```
@@ -67,21 +67,21 @@ ObjectStorage is introduced to store large message (messages that exceed the siz
 
 ## Spec
 
-ObjectStorage inputs can be found in the table below. As of now, only S3 provider of large message store is supported.
+ObjectStorage inputs can be found in the table below. As of now, only S3 provider of object storage is supported.
 
 | Key        | Type    | Description                                       |
 | ---------- | ------- | ------------------------------------------------- |
-| provider   | string | [Required] Provider of large message store (Allowed values: s3) |
-| s3   | S3 | [Required if provider=s3] S3 large message store inputs  |
+| provider   | string | [Required] Provider of object storage (Allowed values: s3) |
+| s3   | S3 | [Required if provider=s3] S3 object storage inputs  |
 
-S3 large message store inputs can be found in the table below.
+S3 object storage inputs can be found in the table below.
 
 | Key        | Type    | Description                                       |
 | ---------- | ------- | ------------------------------------------------- |
 | path   | string | [Required] Remote storage location for messages that are larger than the underlying maximum message size  |
 | endpoint   | string | [Optional, if not provided formed based on region] S3-compatible service endpoint
 
-Change of any of the large message queue inputs triggers the restart of Splunk so that appropriate .conf files are correctly refreshed and consumed.
+Change of any of the object storage inputs triggers the restart of Splunk so that appropriate .conf files are correctly refreshed and consumed.
 
 ## Example
 ```
@@ -108,13 +108,13 @@ In addition to common spec inputs, the IngestorCluster resource provides the fol
 | ---------- | ------- | ------------------------------------------------- |
 | replicas   | integer | The number of replicas (defaults to 3) |
 | queueRef   | corev1.ObjectReference | Message queue reference |
-| objectStorageRef   | corev1.ObjectReference | Large message store reference |
+| objectStorageRef   | corev1.ObjectReference | Object storage reference |
 
 ## Example
 
 The example presented below configures IngestorCluster named ingestor with Splunk ${SPLUNK_IMAGE_VERSION} image that resides in a default namespace and is scaled to 3 replicas that serve the ingestion traffic. This IngestorCluster custom resource is set up with the service account named ingestor-sa allowing it to perform SQS and S3 operations. Queue and ObjectStorage references allow the user to specify queue and bucket settings for the ingestion process. 
 
-In this case, the setup uses the SQS and S3 based configuration where the messages are stored in sqs-test queue in us-west-2 region with dead letter queue set to sqs-dlq-test queue. The large message store is set to ingestion bucket in smartbus-test directory. Based on these inputs, default-mode.conf and outputs.conf files are configured accordingly.
+In this case, the setup uses the SQS and S3 based configuration where the messages are stored in sqs-test queue in us-west-2 region with dead letter queue set to sqs-dlq-test queue. The object storage is set to ingestion bucket in smartbus-test directory. Based on these inputs, default-mode.conf and outputs.conf files are configured accordingly.
 
 ```
 apiVersion: enterprise.splunk.com/v4
@@ -145,13 +145,13 @@ In addition to common spec inputs, the IndexerCluster resource provides the foll
 | ---------- | ------- | ------------------------------------------------- |
 | replicas   | integer | The number of replicas (defaults to 3) |
 | queueRef   | corev1.ObjectReference | Message queue reference |
-| objectStorageRef   | corev1.ObjectReference | Large message store reference |
+| objectStorageRef   | corev1.ObjectReference | Object storage reference |
 
 ## Example
 
 The example presented below configures IndexerCluster named indexer with Splunk ${SPLUNK_IMAGE_VERSION} image that resides in a default namespace and is scaled to 3 replicas that serve the indexing traffic. This IndexerCluster custom resource is set up with the service account named ingestor-sa allowing it to perform SQS and S3 operations. Queue and ObjectStorage references allow the user to specify queue and bucket settings for the indexing process. 
 
-In this case, the setup uses the SQS and S3 based configuration where the messages are stored in and retrieved from sqs-test queue in us-west-2 region with dead letter queue set to sqs-dlq-test queue. The large message store is set to ingestion bucket in smartbus-test directory. Based on these inputs, default-mode.conf, inputs.conf and outputs.conf files are configured accordingly.
+In this case, the setup uses the SQS and S3 based configuration where the messages are stored in and retrieved from sqs-test queue in us-west-2 region with dead letter queue set to sqs-dlq-test queue. The object storage is set to ingestion bucket in smartbus-test directory. Based on these inputs, default-mode.conf, inputs.conf and outputs.conf files are configured accordingly.
 
 ```
 apiVersion: enterprise.splunk.com/v4
@@ -717,7 +717,7 @@ Spec:
     Name:           queue
     Namespace:      default
   Image:  splunk/splunk:${SPLUNK_IMAGE_VERSION}
-  Large Message Store Ref:
+  Object Storage Ref:
     Name:           os
     Namespace:      default
   Replicas:                          3
@@ -741,7 +741,7 @@ Status:
       Endpoint:                 https://sqs.us-west-2.amazonaws.com
       Name:                     sqs-test
     Provider:                   sqs
-  Large Message Store:
+  Object Storage:
     S3:
       Endpoint:  https://s3.us-west-2.amazonaws.com
       Path:      s3://ingestion/smartbus-test

helm-chart/splunk-enterprise/templates/enterprise_v4_indexercluster.yaml

@@ -170,8 +170,8 @@ items:
       namespace: {{ .namespace }}
     {{- end }}
   {{- end }}
-    {{- with $.Values.indexerCluster.objectStoreRef }}
-    objectStoreRef:
+    {{- with $.Values.indexerCluster.objectStorageRef }}
+    objectStorageRef:
       name: {{ .name }}
     {{- if .namespace }}
       namespace: {{ .namespace }}

helm-chart/splunk-enterprise/templates/enterprise_v4_objectstorages.yaml

@@ -1,4 +1,4 @@
-{{- if .Values.objectStorage.enabled }}
+{{- if .Values.objectStorage }}
 {{- if .Values.objectStorage.enabled }}
 apiVersion: enterprise.splunk.com/v4
 kind: ObjectStorage

helm-chart/splunk-enterprise/templates/enterprise_v4_queues.yaml

@@ -26,8 +26,8 @@ spec:
     {{- if .name }}
     name: {{ .name | quote }}
     {{- end }}
-    {{- if .region }}
-    region: {{ .region | quote }}
+    {{- if .authRegion }}
+    authRegion: {{ .authRegion | quote }}
     {{- end }}
     {{- if .volumes }}
     volumes:

kuttl/tests/helm/index-and-ingest-separation/02-assert.yaml

@@ -1,30 +1,30 @@
 ---
-# assert for bus custom resource to be ready
+# assert for queue custom resource to be ready
 apiVersion: enterprise.splunk.com/v4
-kind: Bus
+kind: Queue
 metadata:
-  name: bus
+  name: queue
 spec:
   provider: sqs
   sqs:
-    name: sqs-test
-    region: us-west-2
+    name: index-ingest-separation-test-q
+    authRegion: us-west-2
     endpoint: https://sqs.us-west-2.amazonaws.com
-    dlq: sqs-dlq-test
+    dlq: index-ingest-separation-test-dlq
 status:
   phase: Ready
 
 ---
-# assert for large message store custom resource to be ready
+# assert for object storage custom resource to be ready
 apiVersion: enterprise.splunk.com/v4
-kind: LargeMessageStore
+kind: ObjectStorage
 metadata:
-  name: lms
+  name: os
 spec:
   provider: s3
   s3:
     endpoint: https://s3.us-west-2.amazonaws.com
-    path: s3://ingestion/smartbus-test
+    path: s3://index-ingest-separation-test-bucket/smartbus-test
 status:
   phase: Ready
 
@@ -61,24 +61,24 @@ metadata:
   name: indexer
 spec:
   replicas: 3
-  busRef:
-    name: bus
-  largeMessageStoreRef:
-    name: lms
+  queueRef:
+    name: queue
+  objectStorageRef:
+    name: os
 status:
   phase: Ready
-  bus:
+  queue:
     provider: sqs
     sqs:
-      name: sqs-test
-      region: us-west-2
+      name: index-ingest-separation-test-q
+      authRegion: us-west-2
       endpoint: https://sqs.us-west-2.amazonaws.com
-      dlq: sqs-dlq-test
-  largeMessageStore:
+      dlq: index-ingest-separation-test-dlq
+  objectStorage:
     provider: s3
     s3:
       endpoint: https://s3.us-west-2.amazonaws.com
-      path: s3://ingestion/smartbus-test
+      path: s3://index-ingest-separation-test-bucket/smartbus-test
 
 ---
 # check for stateful set and replicas as configured
@@ -103,7 +103,7 @@ kind: IngestorCluster
 metadata:
   name: ingestor
 spec:
-  replicas: 4
+  replicas: 3
   queueRef:
     name: queue
   objectStorageRef:
@@ -113,15 +113,15 @@ status:
   queue:
     provider: sqs
     sqs:
-      name: sqs-test
-      region: us-west-2
+      name: index-ingest-separation-test-q
+      authRegion: us-west-2
       endpoint: https://sqs.us-west-2.amazonaws.com
-      dlq: sqs-dlq-test
+      dlq: index-ingest-separation-test-dlq
   objectStorage:
     provider: s3
     s3:
       endpoint: https://s3.us-west-2.amazonaws.com
-      path: s3://ingestion/smartbus-test
+      path: s3://index-ingest-separation-test-bucket/smartbus-test
 
 ---
 # check for stateful set and replicas as configured

kuttl/tests/helm/index-and-ingest-separation/03-assert.yaml

@@ -6,24 +6,24 @@ metadata:
   name: ingestor
 spec:
   replicas: 4
-  busRef:
-    name: bus
-  largeMessageStoreRef:
-    name: lms
+  queueRef:
+    name: queue
+  objectStorageRef:
+    name: os
 status:
   phase: Ready
-  bus:
+  queue:
     provider: sqs
     sqs:
-      name: sqs-test
-      region: us-west-2
+      name: index-ingest-separation-test-q
+      authRegion: us-west-2
       endpoint: https://sqs.us-west-2.amazonaws.com
-      dlq: sqs-dlq-test
-  largeMessageStore:
+      dlq: index-ingest-separation-test-dlq
+  objectStorage:
     provider: s3
     s3:
       endpoint: https://s3.us-west-2.amazonaws.com
-      path: s3://ingestion/smartbus-test
+      path: s3://index-ingest-separation-test-bucket/smartbus-test
 
 ---
 # check for stateful sets and replicas updated

kuttl/tests/helm/index-and-ingest-separation/splunk_index_ingest_sep.yaml

@@ -10,10 +10,10 @@ queue:
   name: queue
   provider: sqs
   sqs:
-    name: sqs-test
-    region: us-west-2
+    name: index-ingest-separation-test-q
+    authRegion: us-west-2
     endpoint: https://sqs.us-west-2.amazonaws.com
-    dlq: sqs-dlq-test
+    dlq: index-ingest-separation-test-dlq
     volumes:
       - name: helm-bus-secret-ref-test
         secretRef: s3-secret
@@ -24,7 +24,7 @@ objectStorage:
   provider: s3
   s3:
     endpoint: https://s3.us-west-2.amazonaws.com
-    path: s3://ingestion/smartbus-test
+    path: s3://index-ingest-separation-test-bucket/smartbus-test
 
 ingestorCluster:
   enabled: true