Merge pull request #68 from shivamshishodia/helm_discovery_changes

OCR Image; Version and Changelog changes

Author: santhoshkvuda

CHANGELOG.md

@@ -1,13 +1,21 @@
 # Change Log
 
+## 2024-02-13
+### Added
+- Changes to support Kubernetes Solution Pages Offering by OCI Logging Analytics.
+  - A new role and role binding in the monitoring namespace (which defaults to oci-onm), to manage a ConfigMap.
+  - A new CronJob to handle the Kubernetes Objects discovery and Objects Logs collection using oci-logging-analytics-kubernetes-discovery Gem.
+### Changed
+- Moving forward, Kubernetes Objects logs would be collected using Kubernetes Discovery CronJob along with the (optional) Discovery data instead of Fluentd based Deployment.
+
 ## 2024-01-18
 ### Changed
 - Management Agent docker image has been updated to version 1.2.0
 
 ## 2024-01-09
 ### Changed
 - Concat filter plugin behavior changed to not include newline character as separator while handling CRI partial logs.
-- Dashboards Import is now optional while installing the monitoring solution through RMS Stack. Default behavior remains the same. 
+- Dashboards Import is now optional while installing the monitoring solution through RMS Stack. Default behavior remains the same.
 
 ## 2023-12-01
 ### Added
@@ -16,19 +24,19 @@
 ## 2023-11-30
 ### Added
 - Added resources information that got created through RM Stack to Stack output.
-- Added new auto created policy for Kubernetes Objects discovery (for future release(s) use). 
-- Added new helm variable for cluster's EntityId (ociLAClusterEntityID) (for future release(s) use). 
+- Added new auto created policy for Kubernetes Objects discovery (for future release(s) use).
+- Added new helm variable for cluster's EntityId (ociLAClusterEntityID) (for future release(s) use).
 ### Changed
-- RM Stack is changed to use remote [helm repo](https://oracle-quickstart.github.io/oci-kubernetes-monitoring), instead of a local copy of helm chart source. 
-- RM Stack is modified to skip recreation of Management Agent Key if the Key already created by Stack. 
+- RM Stack is changed to use remote [helm repo](https://oracle-quickstart.github.io/oci-kubernetes-monitoring), instead of a local copy of helm chart source.
+- RM Stack is modified to skip recreation of Management Agent Key if the Key already created by Stack.
 ### Breaking Changes
--  Removed ociLAEntityID input variable of Logan chart. This was an optional and its use-case was not defined so far. Hence, it shouldn't  be a breaking change in general but still documenting for the reference. 
+-  Removed ociLAEntityID input variable of Logan chart. This was an optional and its use-case was not defined so far. Hence, it shouldn't  be a breaking change in general but still documenting for the reference.
 
 ## 2023-11-07
 ### Added
 - Control Plane Logs Collection for OCNE and Standalone Kubernetes Clusters.
 - Support for launching Fluentd containers in privileged mode (default false).
-- Added FAQ for triaging log collection setup issues in OCNE and Standalone Kubernetes Clusters. 
+- Added FAQ for triaging log collection setup issues in OCNE and Standalone Kubernetes Clusters.
 
 ## 2023-10-31
 ### Changed
@@ -40,14 +48,14 @@
 
 ## 2023-08-07
 ### Added
-- Support Fluentd's [Multi Process Workers](https://docs.fluentd.org/deployment/multi-process-workers). 
+- Support Fluentd's [Multi Process Workers](https://docs.fluentd.org/deployment/multi-process-workers).
 - Custom Container Image for Fluentd using OL8-Slim as base Image.
-- PV, PVC Objects Collection 
+- PV, PVC Objects Collection
 ### Changed
 - Instructions and dependency versions updates to custom container image for Fluentd using OL8 as base image.
-- ClusterRole updates to add read permission for `storage.k8s.io` api group to support PV, PVC Objects collection. 
+- ClusterRole updates to add read permission for `storage.k8s.io` api group to support PV, PVC Objects collection.
 ### Deprecating
-- Custom Container Image for Fluentd using Debian and OL8 as base Image. 
+- Custom Container Image for Fluentd using Debian and OL8 as base Image.
 
 ## 2023-07-19
 ### Added
@@ -60,7 +68,7 @@
 ### Changed
 - Refactoring of helm chart, terraform and stack/market place app to support the consolidation of logs, objects and metrics collection.
 ### Breaking Changes
-- The refactoring work done in this version, may cause issues if you upgrade to this version (v3.0.0) from previous versions. Refer [here](README.md#2x-to-3x) for further details. 
+- The refactoring work done in this version, may cause issues if you upgrade to this version (v3.0.0) from previous versions. Refer [here](README.md#2x-to-3x) for further details.
 
 ## 2023-02-07
 ### Added

charts/common/Chart.yaml

@@ -5,5 +5,5 @@ apiVersion: v2
 name: oci-onm-common
 description: Common resources for OCI Kubernetes Monitoring solution helm charts
 type: application
-version: 3.0.2
+version: 3.1.0
 appVersion: "3.0.0"

charts/logan/Chart.yaml

@@ -10,6 +10,6 @@ appVersion: "3.0.0"
 
 dependencies:
 - name: oci-onm-common
-  version: "3.0.2"
+  version: "3.1.0"
   repository: "file://../common"
   condition: oci-onm-common.enabled

charts/logan/templates/discovery-cronjob.yaml

@@ -17,7 +17,7 @@ spec:
   failedJobsHistoryLimit: {{ .Values.k8sDiscovery.objects.failedJobsHistoryLimit }}
   jobTemplate:
     spec:
-      backoffLimit: 2
+      backoffLimit: {{ .Values.k8sDiscovery.objects.backoffLimit }}
       template:
         spec:
           restartPolicy: {{ .Values.k8sDiscovery.objects.restartPolicy }}
@@ -33,7 +33,9 @@ spec:
             {{- end }}
             command:
             {{- /* object discovery script */}}
-              - oci-logan-kubernetes-discovery
+              - bundle
+              - exec
+              - oci-loganalytics-kubernetes-discovery
             {{- /* mandatory inputs */}}
               - --kubernetes_cluster_id
               - {{ $kubernetesClusterId }}
@@ -78,7 +80,7 @@ spec:
             {{- if .Values.k8sDiscovery.objects.log_format }}
               - --log_format
               - {{ .Values.k8sDiscovery.objects.log_format }}
-            {{- end }}                
+            {{- end }}
             {{- if .Values.k8sDiscovery.objects.log_level }}
               - --log_level
               - {{ .Values.k8sDiscovery.objects.log_level }}
@@ -89,7 +91,7 @@ spec:
               - {{ .Values.k8sDiscovery.objects.thread_count | quote }}
             {{- end }}
             {{- if .Values.k8sDiscovery.objects.chunk_limit }}
-              - --chunk_limit 
+              - --chunk_limit
               - {{ .Values.k8sDiscovery.objects.chunk_limit | quote }}
             {{- end }}
             {{- if .Values.k8sDiscovery.objects.collect_warning_events_only }}
@@ -138,4 +140,4 @@ spec:
               sources:
               - secret:
                   name: {{ $resourceNamePrefix }}-oci-config
-          {{- end }}
+          {{- end }}

charts/logan/values.yaml

@@ -48,7 +48,7 @@ image:
    # Image pull secrets for. Secret must be in the namespace defined by namespace
    imagePullSecrets:
    # -- Replace this value with actual docker image url
-   url: iad.ocir.io/id3m3bqjn6oc/k8-objects-discovery:master-5e03e34 #TODO
+   url: container-registry.oracle.com/oci_observability_management/oci-la-fluentd-collector:1.3.0
    # -- Image pull policy
    imagePullPolicy: Always
 
@@ -541,7 +541,7 @@ k8sDiscovery:
       # failedJobsHistoryLimit: specify how many failed jobs should be kept
       failedJobsHistoryLimit : 3
       # ociLALogGroupID: OCI LA Loggroup to store the k8s discovery logs
-      ociLALogGroupID: 
+      ociLALogGroupID:
       # discoveryMode: discovery mode to be used - ["infra", "object"]
       discoveryMode: "object"
       # Log format to be used - ["text", "json"]
@@ -559,6 +559,8 @@ k8sDiscovery:
       # collect_warning_events_only: flag to only collect warning event logs
       # Default: All logs are collected and sent to OCI logging analytics for processing
       collect_warning_events_only: false
+      # backoffLimit: Specify the number of retries before considering a Job as failed
+      backoffLimit: 2
    # kubernetes: kubernetes cluster related inputs for kubernetes disocvery job
    kubeClientOptions:
       # kubernetes_url: Kubernetes API server URL.
@@ -574,4 +576,4 @@ k8sDiscovery:
       # bearer_token_file: Path to bearer token file if basic authentication is used
       bearer_token_file:
       # secret_dir: Path to Kubernetes service account token file to access Kubernetes API
-      secret_dir:
+      secret_dir:

charts/mgmt-agent/Chart.yaml

@@ -28,6 +28,6 @@ appVersion: "1.16.0"
 
 dependencies:
 - name: oci-onm-common
-  version: "3.0.1"
+  version: "3.1.0"
   repository: "file://../common"
   condition: oci-onm-common.enabled

charts/oci-onm/Chart.yaml

@@ -18,7 +18,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 3.2.3
+version: 3.3.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
@@ -28,7 +28,7 @@ appVersion: "3.0.0"
 
 dependencies:
 - name: oci-onm-common
-  version: "3.0.2"
+  version: "3.1.0"
   repository: "file://../common"
   condition: oci-onm-common.enabled
 - name: oci-onm-logan

charts/oci-onm/values.yaml

@@ -31,7 +31,7 @@ oci-onm-logan:
   kubernetesClusterID: "{{ .Values.global.kubernetesClusterID }}"
   kubernetesClusterName: "{{ .Values.global.kubernetesClusterName }}"
   image:
-    url: iad.ocir.io/id3m3bqjn6oc/k8-objects-discovery:master-5e03e34 #TODO
+    url: container-registry.oracle.com/oci_observability_management/oci-la-fluentd-collector:1.3.0
   # Go to OCI Logging Analytics Administration, click Service Details, and note the namespace value.
   ociLANamespace:
   # OCI Logging Analytics Default Log Group OCID
@@ -54,4 +54,4 @@ oci-onm-mgmt-agent:
       # Replace this value with actual docker image URL for Management Agent
       url: container-registry.oracle.com/oci_observability_management/oci-management-agent:1.2.0
       # Image secrets to use for pulling container image (base64 encoded content of ~/.docker/config.json file)
-      secret:
+      secret:

terraform/modules/helm/helm.tf

@@ -102,4 +102,4 @@ data "helm_template" "oci-kubernetes-monitoring" {
   }
 
   count = var.generate_helm_template ? 1 : 0
-}
+}

terraform/oke/developer-options.tf

@@ -12,27 +12,27 @@ variable "toggle_livelab_module" {
   default = true
 }
 
-# Enable/Disable helm module 
+# Enable/Disable helm module
 variable "toggle_helm_module" {
   type    = bool
   default = true
 }
 
-# when false, public helm repo is used for deployment 
+# when false, public helm repo is used for deployment
 variable "toggle_use_local_helm_chart" {
   type    = bool
   default = false
 }
 
-# Enable/Disable helm template. When set as true, 
+# Enable/Disable helm template. When set as true,
 # - helm module will generate template file inside ../modules/helm/local directory
 # - Setting this to true disables/skips the helm release
 variable "toggle_generate_helm_template" {
   type    = bool
   default = false
 }
 
-# Enable/Disable helm installation. 
+# Enable/Disable helm installation.
 variable "toggle_install_helm" {
   type    = bool
   default = true
@@ -60,4 +60,4 @@ variable "toggle_logan_module" {
 variable "toggle_iam_module" {
   type    = bool
   default = true
-}
+}