INFRA:45230: Cronjob fixes for alerting and additional support for backoffLimit (#377)

* INFRA-45230: Support cronjob/job backoffLimit and reduce cronjob/job ttlSecondsAfterFinished setting so alert fires

* Fixes for backoffLimit: 0

* Bump chart version

---------

Co-authored-by: Nick <nbadger@mintel.com>

Author: nabadger

charts/standard-application-stack/CHANGELOG.md

@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+
+## [v11.1.0] - 2026-01-26
+### Changed
+- Update the default value of `ttlSecondsAfterFinished` to 5 minutes. This *must* be greater then the related `KubeJobFailed` alert check.
+- Added support for custom `backoffLimit` configuration in CronJobs and Jobs. The Kubernetes default (6) is used if not specified. Can be configured via `cronjobs.defaults.backoffLimit`, `cronjobs.jobs[].backoffLimit`, `jobDefaults.backoffLimit`, or per-job in `jobs[].backoffLimit`.
+
 ## [v11.0.2] - 2026-01-06
 ### Changed
 - Updated helm and helm-docs versions so the unittest plugin would work again.

charts/standard-application-stack/Chart.yaml

@@ -15,7 +15,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 11.0.2
+version: 11.1.0
 
 dependencies:
   - name: redis

charts/standard-application-stack/README.md

@@ -1,6 +1,6 @@
 # standard-application-stack
 
-![Version: 11.0.2](https://img.shields.io/badge/Version-11.0.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square)
+![Version: 11.1.0](https://img.shields.io/badge/Version-11.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square)
 
 A generic chart to support most common application requirements
 
@@ -65,14 +65,15 @@ A generic chart to support most common application requirements
 | celeryBeat.resources.requests | object | `{}` | The requested resources for the container |
 | command | list | `["/app/docker-entrypoint.sh"]` | Optional command to the container |
 | configMaps | list | `[]` | A list of configuration maps for this application |
-| cronjobs | object | `{"defaults":{"concurrencyPolicy":"Forbid","enableDoNotDisrupt":true,"restartPolicy":"Never","suspend":false,"timezone":null,"ttlSecondsAfterFinished":60},"jobs":[]}` | Define and Configure CronJob's Defaults to same image as main deployment but with defined arguments |
-| cronjobs.defaults | object | `{"concurrencyPolicy":"Forbid","enableDoNotDisrupt":true,"restartPolicy":"Never","suspend":false,"timezone":null,"ttlSecondsAfterFinished":60}` | Defaults for all CronJob's |
+| cronjobs | object | `{"defaults":{"backoffLimit":null,"concurrencyPolicy":"Forbid","enableDoNotDisrupt":true,"restartPolicy":"Never","suspend":false,"timezone":null,"ttlSecondsAfterFinished":600},"jobs":[]}` | Define and Configure CronJob's Defaults to same image as main deployment but with defined arguments |
+| cronjobs.defaults | object | `{"backoffLimit":null,"concurrencyPolicy":"Forbid","enableDoNotDisrupt":true,"restartPolicy":"Never","suspend":false,"timezone":null,"ttlSecondsAfterFinished":600}` | Defaults for all CronJob's |
+| cronjobs.defaults.backoffLimit | string | `nil` | Specifies the number of retries before marking a job as failed. ref: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/job-v1/#JobSpec Only set if you want to override the Kubernetes default (6). If not set, Kubernetes default applies. |
 | cronjobs.defaults.concurrencyPolicy | string | `"Forbid"` | Tells controller how to handle concurrent executions of a CronJob ref: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/cron-job-v1/#CronJobSpec |
 | cronjobs.defaults.enableDoNotDisrupt | bool | `true` | Whether to set the `karpenter.sh/do-not-disrupt`annotation on the CronJob |
 | cronjobs.defaults.restartPolicy | string | `"Never"` | Configure CronJob pod restart Policy ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy |
 | cronjobs.defaults.suspend | bool | `false` | Tells controller to suspend future executions ref: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/cron-job-v1/#CronJobSpec |
 | cronjobs.defaults.timezone | string | `nil` | CronJob schedule will run relative to this timezone. ref: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones |
-| cronjobs.defaults.ttlSecondsAfterFinished | int | `60` | If this field is set, ttlSecondsAfterFinished after the Job finishes, it is eligible to be automatically deleted. ref: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#cronjob-v1beta1-batch |
+| cronjobs.defaults.ttlSecondsAfterFinished | int | `600` | If this field is set, ttlSecondsAfterFinished after the Job finishes, it is eligible to be automatically deleted. ref: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#cronjob-v1beta1-batch This needs to be greater than the associated KubeJobFailed 'for' value |
 | cronjobs.jobs | list | `[]` | List of Cronjob configurations to be defined |
 | cronjobsOnly | bool | `false` | Only show Cronjobs and relevant resources (i.e. if set to `true`, hide the main deployment resource) |
 | dynamodb.enabled | bool | `false` |  |
@@ -166,6 +167,7 @@ A generic chart to support most common application requirements
 | jobDefaults.argo.hookDeletePolicy | string | `nil` | When to delete the job resources in an automated fashion ref: https://argo-cd.readthedocs.io/en/stable/user-guide/resource_hooks/#hook-deletion-policies. |
 | jobDefaults.argo.syncWave | string | `nil` | Sync Wave in which ArgoCD should apply the manifest. ref: https://argo-cd.readthedocs.io/en/stable/user-guide/sync-waves/. |
 | jobDefaults.args | string | `nil` | The command arguments for the main Job container. |
+| jobDefaults.backoffLimit | string | `nil` | Specifies the number of retries before marking a job as failed. ref: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/job-v1/#JobSpec Only set if you want to override the Kubernetes default (6). If not set, Kubernetes default applies. |
 | jobDefaults.command | string | `nil` | The command the main Job container will run. |
 | jobDefaults.enableDoNotDisrupt | bool | `true` | Whether to set the `karpenter.sh/do-not-disrupt`annotation on the Job |
 | jobDefaults.env | list | `[]` | Any env entries you want to add. See includeBaseEnv to add all from main container. ref: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/ |
@@ -180,7 +182,7 @@ A generic chart to support most common application requirements
 | jobDefaults.podSecurityContext | object | `{}` | Add podSecurityContext config to the Job. |
 | jobDefaults.resources | object | `{}` | REQUIRED FOR ALL JOBS. Resource requests/limits. |
 | jobDefaults.restartPolicy | string | `"Never"` | Whether the pod should be restarted on failure ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy) |
-| jobDefaults.ttlSecondsAfterFinished | int | `60` | If this field is set, ttlSecondsAfterFinished after the Job finishes, it is eligible to be automatically deleted. ref: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#cronjob-v1beta1-batch |
+| jobDefaults.ttlSecondsAfterFinished | int | `600` | If this field is set, ttlSecondsAfterFinished after the Job finishes, it is eligible to be automatically deleted. ref: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#cronjob-v1beta1-batch  This needs to be greater than the associated KubeJobFailed 'for' value |
 | jobs | list | `[]` | Define and configure jobs Add a map for each job in this list. Refer to `$.Values.jobDefaults` for a list of supported values (and the defaults that will be applied to all jobs below). |
 | jobsOnly | bool | `false` | Only show Jobs and relevant resources (i.e. if set to `true`, hide the main deployment resource) |
 | kibana.elasticsearchHosts | string | `""` |  |

charts/standard-application-stack/templates/cronjobs.yaml

@@ -26,6 +26,13 @@ spec:
       {{- if (gt $ttl 0) }}
       ttlSecondsAfterFinished: {{ $ttl}}
       {{- end }}
+      {{- $backoffLimit := .backoffLimit }}
+      {{- if eq $backoffLimit nil }}
+      {{- $backoffLimit = $.Values.cronjobs.defaults.backoffLimit }}
+      {{- end }}
+      {{- if ne $backoffLimit nil }}
+      backoffLimit: {{ $backoffLimit | int }}
+      {{- end }}
       template:
         metadata:
           labels: {{ include "mintel_common.labels" $data | nindent 12 }}

charts/standard-application-stack/templates/jobs.yaml

@@ -17,10 +17,17 @@ metadata:
     {{- end }}
   namespace: {{ $.Release.Namespace }}
 spec:
-  {{- $ttl := (.ttlSecondsAfterFinished | default 60) | int }}
+  {{- $ttl := (.ttlSecondsAfterFinished | default 600) | int }}
   {{- if (gt $ttl 0) }}
   ttlSecondsAfterFinished: {{ $ttl}}
   {{- end }}
+  {{- $backoffLimit := .backoffLimit }}
+  {{- if eq $backoffLimit nil }}
+  {{- $backoffLimit = $.Values.jobDefaults.backoffLimit }}
+  {{- end }}
+  {{- if ne $backoffLimit nil }}
+  backoffLimit: {{ $backoffLimit | int }}
+  {{- end }}
   template:
     {{- if (.enableDoNotDisrupt) }}
     metadata: