v0.74.1

• BUGFIX: vldistributed: add validation webhook for VLDistributed resources.

v0.74.0

Update note 1: vmoperator: validation webhooks for prometheus-operator CRDs added to the operator in 0.72.0 is now disabled by default. Use VM_PROMCR_VALIDATION_ENABLED env var to enable it explicitly.

• Dependency: vmoperator: Updated default versions for VM apps to v1.148.0 version

• Dependency: vmoperator: Updated default versions for VL apps to v1.52.0.

• Dependency: vmoperator: Updated default versions for VT apps to v0.10.0 version.

• Dependency: vmoperator: Updated default versions for VMAnomaly to v1.30.0 version

• FEATURE: vmanomaly: support Temporal Envelope models, online-model history strength and warmup settings, causal and complexity-aware autotune options, and separate reader fetch and processing controls introduced in vmanomaly v1.30.0. See #2440.

• FEATURE: vmoperator: add networkPolicy field to all supported CRDs (VMSingle, VMAgent, VMAlert, VMAlertmanager, VMAuth, VLSingle, VLAgent, VTSingle, VMAnomaly, and all cluster sub-components). When set, the operator creates and manages a NetworkPolicy resource that restricts ingress/egress to the component's pods. See #2977.

• BUGFIX: vmoperator: grant the operator's ClusterRole permissions on networking.k8s.io/networkpolicies. The networkPolicy feature above was shipped without this grant, so the operator would fail with a Forbidden error creating/updating/deleting NetworkPolicy objects on a real cluster.

• FEATURE: vmauth: add opt-in spec.waitForConfigReload field. When enabled, the operator confirms that every replica has actually picked up a config change before marking dependent VMUser objects as applied, so their status can be trusted to reflect what's actually running rather than what was merely written out. vmdistributed's internal VMAuth load balancer always waits for this confirmation before rotating a zone in or out of the backend list, regardless of spec.vmAuth.spec.waitForConfigReload, so traffic is never routed to a zone that's still running a stale config.

• FEATURE: vlagent: add basicAuth field support to remoteWrite entries. See #2371.

• FEATURE: vmcluster: support VMCluster extraStorageNodes for vmselect component.

• FEATURE: vmuser: add defaultVMAccessClaim field to spec.jwt, mapped to vmauth's jwt.default_vm_access_claim. It lets a VMUser accept JWTs that don't carry a vm_access claim, matching vmauth v1.147.0+ behavior. See #2375.

• FEATURE: helm-converter: parse the victoria-metrics-auth chart's config value (vmauth's own native config file). Each config.users entry is converted into a standalone VMUser CR, and config.unauthorized_user is converted into the VMAuth CR's spec.unauthorizedUserAccessSpec. The generated VMUser CRs are appended to the same output file as additional YAML documents, and the VMAuth CR's spec.userSelector is set to a dedicated label matching them, so the operator actually loads them (a bare VMAuth CR's default selectors match nothing). See #2397.

• FEATURE: vmoperator: introduce VLDistributed CR, which controls multiple region-distributed VictoriaLogs clusters.

• BUGFIX: vmoperator: support Ignored status for child objects that were not picked, also do not set Failed status if object was applied on at least one parent object. See #2432.

• BUGFIX: vmoperator: removed library/ component, while building CR images that do not contain / in repo name. See #2409.

• BUGFIX: vmoperator: log only the changed key names and value sizes, instead of the full values, when updating ConfigMap. Previously a single data_diff log line could reach multiple megabytes for large ConfigMaps and break downstream log pipelines with per-line size limits. See #2426.

• BUGFIX: vmagent, vmsingle: add missing list verb to config-reloader's secrets RBAC rule. See #2384.

• BUGFIX: vmoperator: return an error instead of panicking when a Labels map value is malformed (missing the = separator) during config parsing.

• BUGFIX: vmalert: when no notifiers are configured, ignore alerting rules from selected VMRules instead of failing reconciliation; recording rules in the same group are kept and still reconciled. See #2388.

• BUGFIX: vmalertmanager: default pod securityContext.fsGroup to 65534 when persistent storage is configured and neither useStrictSecurity nor a user securityContext is set. Without it the alertmanager process cannot write notification log and silences to a freshly provisioned volume, so silences are silently lost on pod restart. User-provided securityContext and useStrictSecurity keep their previous behaviour.

• BUGFIX: vmoperator: allow pinning images by digest via the image.tag field of all operator CRs. When tag is a digest (e.g. sha256:<hex>) it is now joined to the repository with @ instead of :, producing a valid repository@sha256:<digest> reference. Regular tags are unaffected.

• BUGFIX: helm-converter: fix persistentVolume.storageClassName being dropped during conversion due to a stale storageClass field name. See #2389.

• BUGFIX: helm-converter: fix securityContext.runAsNonRoot, runAsUser, runAsGroup, seccompProfile, appArmorProfile, seLinuxOptions, and windowsOptions being silently dropped during conversion; these are now promoted to spec.securityContext.podSecurityContext, matching how the operator applies them to containers. See #2391.

• BUGFIX: helm-converter: fix TLS/mTLS settings (tlsCAFile, tlsCertFile, tlsKeyFile, tlsServerName, tlsInsecureSkipVerify) on remoteWrite entries being dropped during conversion for vmagent, vlagent, vlcollector, and vmalert charts. See #2390.

• BUGFIX: helm-converter: fix cannot unmarshal object into Go struct field ... headers of type []string error when a chart's default values.yaml ships a headers: {} map (e.g. vmalert's datasource.headers, notifier.headers, remoteWrite.headers); such maps are now normalized to the operator's key:value string-slice format before conversion. See #2398.

• BUGFIX: helm-converter: fix extraVolumes/extraVolumeMounts being silently dropped during conversion for vmsingle, vmagent, vmalert, vmanomaly, vmcluster, vlcluster, vtcluster, vtsingle, vlogs, and vmauth charts. See #2424.

• BUGFIX: vmagent, vmanomaly: target spec.vpa at the VMAgent/VMAnomaly custom resource itself instead of its underlying Deployment/StatefulSet. VPA rejects a targetRef whose owner chain includes another scalable controller, so a VerticalPodAutoscaler targeting the workload directly was silently non-functional whenever the CR exposed a scale subresource. See #2415.

• BUGFIX: vlagent: remove a vestigial scale subresource declaration referencing nonexistent spec/status fields, which unconditionally broke spec.vpa for VLAgent the same way as [#2415](https://github.com/VictoriaMetrics...

v0.68.7

Dependency: vmoperator: Updated default versions for VM apps to v1.147.0 version

Dependency: vmoperator: Updated default versions for VT apps to v0.9.4 version.

Dependency: vmoperator: Updated default versions for VMAnomaly to v1.28.7 version

SECURITY: vmagent, vmsingle: remove cluster-wide secrets and configmaps permissions from the operator-managed ClusterRole. Secret access for the config-reloader is now granted via a namespace-scoped Role limited to the single operator-managed config secret. For vmsingle in ingest-only mode (the default), no secret or configmap permissions are granted at all.

BUGFIX: vmanomaly: add missing scatter_infer_jobs field to the periodic scheduler config struct. See #2328.

BUGFIX: vmanomaly: preserve insertion order of keys in ProphetModel seasonalities, tz_seasonalities, compression, and args fields; previously the operator re-emitted them with keys sorted alphabetically, which broke round-trips for configs that specified keys in a non-alphabetical order. Also renamed the singular seasonality/tz_seasonality YAML keys (deprecated) to the plural seasonalities/tz_seasonalities to match the vmanomaly configuration format. See #2356.

BUGFIX: vmagent, vmsingle: create a Role and RoleBinding in each namespace listed in WATCH_NAMESPACES so that vmagent/vmsingle can perform service discovery in all watched namespaces, not only its own. Previously, in namespaced mode, vmagent/vmsingle could only scrape targets from its own namespace due to missing RBAC in other watched namespaces.

BUGFIX: vmagent: fix missing credential secret and config-reloader setup in ingestOnlyMode when remote write entries carry authentication secrets (basicAuth.password, bearerTokenSecret, or oauth2.clientSecret). Previously the operator-managed secret containing the credential files was never created in ingest-only mode, causing vmagent to start with dangling file references. The secret is now reconciled and the config-reloader is configured to watch it for credential rotation.

v0.73.1

• BUGFIX: vmrule: removed strategic-merge list semantics from rules as it breaks many rules, where it's a common practice to use a same rule name with different thresholds and severities.

v0.73.0

• Dependency: vmoperator: Updated default versions for VM apps to v1.147.0 version

• Dependency: vmoperator: Updated default versions for VL apps to v1.51.0.

• Dependency: vmoperator: Updated default versions for VT apps to v0.9.4 version.

• SECURITY: vmagent, vmsingle: remove cluster-wide secrets and configmaps permissions from the operator-managed ClusterRole. Secret access for the config-reloader is now granted via a namespace-scoped Role limited to the single operator-managed config secret. For vmsingle in ingest-only mode (the default), no secret or configmap permissions are granted at all.

• FEATURE: vmuser, vmauth: add access_log field support. See docs and #2359.

• FEATURE: vmuser: add useExtraService boolean field to crd.namespacedName and crd.objects entries in VMUser target references. When set to true, the operator resolves the target URL using the CR's additional service (configured via spec.serviceSpec) instead of the default service. This lets you route VMAuth traffic to a dedicated service with different port mappings or service type without changing the primary service. See related types: CRDRef, NamespacedName. See #2333.

• FEATURE: vmoperator: add useLegacyNaming field to all operator CRs. When enabled, managed resources are named using the standalone Helm chart convention (CR name directly for single-component CRs; <name>-<component> for cluster CRs) instead of the default operator prefix convention. Useful when migrating from standalone charts to operator-managed resources without recreating existing resources.

• FEATURE: vmdistributed: support using VMSingle backends alongside VMCluster backends in separate zones.

• FEATURE: vmoperator: add VM_OPENSHIFT_COMPATIBILITY env var (OpenshiftCompatibilityMode config field, default auto) to enable OpenShift-specific compatibility features for managed pods. In auto mode the operator detects OpenShift at startup via presence of security.openshift.io/v1 API group. Currently covers automatic mounting of the openshift-service-ca.crt ConfigMap into VMAgent and VMSingle pods at /etc/ssl/certs/openshift-service-ca/service-ca.crt, so that targets signed with the OpenShift service signing CA could be verified without additional TLS config. Supported values: auto, enabled, disabled. See #2971.

• BUGFIX: vmcluster: fix CR getting permanently stuck in expanding status after an image update when HPA is enabled on vminsert, vmselect, or vmstorage. The operator was comparing a stale replica count (captured at reconcile start) against live status fields, so the readiness check could never pass once HPA changed the replica count mid-rollout. The same issue affected VMAuth, VLCluster, and VTCluster components with HPA enabled. See #2324.

• BUGFIX: config-reloader: fix possible panic on Secret watch events when the informer's local cache fell out of sync and Kubernetes delivered a stale tombstone entry instead of the Secret object. The config-reloader now unwraps tombstones correctly and logs an error for any other unexpected types.

• BUGFIX: vmanomaly: add missing scatter_infer_jobs field to the periodic scheduler config struct. See #2328.

• BUGFIX: vmoperator: switch default app probes to tcpSocket startupProbe when TLS is enabled on the managed HTTP endpoint. This avoids broken kubelet httpGet checks against TLS and mTLS-protected workloads. See #1824.

• BUGFIX: vmagent, vmsingle: create a Role and RoleBinding in each namespace listed in WATCH_NAMESPACES so that vmagent/vmsingle can perform service discovery in all watched namespaces, not only its own. Previously, in namespaced mode, vmagent/vmsingle could only scrape targets from its own namespace due to missing RBAC in other watched namespaces.

• BUGFIX: vmagent: fix missing credential secret and config-reloader setup in ingestOnlyMode when remote write entries carry authentication secrets (basicAuth.password, bearerTokenSecret, or oauth2.clientSecret). Previously the operator-managed secret containing the credential files was never created in ingest-only mode, causing vmagent to start with dangling file references. The secret is now reconciled and the config-reloader is configured to watch it for credential rotation.

v0.68.6

• Dependency: vmoperator: Updated default versions for VM apps to v1.146.0 version

• Dependency: vmoperator: Updated default versions for VL apps to v1.51.0.

• Dependency: vmoperator: Updated default versions for VT apps to v0.9.3 version.

• FEATURE: vmoperator: add victoriametrics_app=true label to all metrics scraped by the operator. See #2261.

• BUGFIX: config-reloader: fix possible panic on Secret watch events when the informer's local cache fell out of sync and Kubernetes delivered a stale tombstone entry instead of the Secret object. The config-reloader now unwraps tombstones correctly and logs an error for any other unexpected types.

• BUGFIX: vmoperator: update status currentRevision and currentReplicas for StatefulSet with OnDelete update strategy. See #1242.

• BUGFIX: vmoperator: retry reconcile errors, that may lead to expanding state, before resource could hang in expanding state.

• BUGFIX: vmcluster, vlcluster and vtcluster: when storage HPA was enabled, generated -storageNode flags could become incorrect after scaling, which could break expected routing to storage nodes; now the operator derives storage node count from the current StatefulSet state so generated flags stay correct during HPA-driven scaling. See #2117.

• BUGFIX: vmoperator: update status currentRevision and currentReplicas for StatefulSet with OnDelete update strategy. See #1242.

• BUGFIX: config-reloader: fix configreloader_last_reload_success_timestamp_seconds metric to report time in seconds instead of milliseconds.

• BUGFIX: vmoperator: ignore NotFound errors, that may occur during attempt to update status on a missing resource.

• BUGFIX: vmanomaly: pass the configured TLS CA bundle to the reader, writer and monitoring clients. Previously the CA was mounted as a volume but dropped during config generation, so a tlsConfig with only a CA produced no verify_tls reference to it; insecureSkipVerify is now also propagated correctly.

• BUGFIX: config-reloader: fix missed reload for watched files whose names contain .. (e.g. rules..yaml). Previously any path containing .. was silently skipped; now only Kubernetes synthetic entries whose basename starts with .. (e.g. ..data) are ignored. See #2253.

v0.72.0

Update note 1: vmalert: rule ConfigMaps now store gzip-compressed content in binaryData and an init container decompresses them before VMAlert starts. Existing VMAlert pods will be rolled out once during this upgrade.

• FEATURE: vmoperator: add validating webhooks for Prometheus Operator CRDs (ServiceMonitor, PodMonitor, PrometheusRule, Probe, ScrapeConfig, AlertmanagerConfig). Each object is converted to its VM equivalent and validated when webhooks are enabled. See #2270.

• FEATURE: vmscrapeconfig: add support for consulAgentSDConfigs, dockerSDConfigs, dockerSwarmSDConfigs, marathonSDConfigs, and yandexCloudSDConfigs service discovery types, bringing VMScrapeConfig to full parity with VictoriaMetrics sd_configs. See #2265.

• FEATURE: vmalert: rule ConfigMaps now store gzip-compressed rule files in binaryData, allowing larger rule sets within Kubernetes object size limits. An init container decompresses the rules before VMAlert starts.

• FEATURE: vmscrapeconfig: add support for consulAgentSDConfigs, dockerSDConfigs, dockerSwarmSDConfigs, marathonSDConfigs, and yandexCloudSDConfigs service discovery types, bringing VMScrapeConfig to full parity with VictoriaMetrics sd_configs. See #2265.

• FEATURE: vmoperator: add victoriametrics_app=true label to all metrics scraped by the operator. See #2261.

• BUGFIX: vmoperator: skip reconciliation only for CRs with genuine spec parse errors; CRs whose ParsingSpecError is caused solely by unknown fields (e.g. after an operator downgrade) are now reconciled normally instead of being silently skipped.

• BUGFIX: vmoperator: fix potential deadlock in operator_object_status metrics collector when the number of tracked objects exceeds 250. The Collect method previously held a mutex while sending to the prometheus channel, which could deadlock if the channel was full and another goroutine was waiting on the same mutex. See #2239.

• BUGFIX: config-reloader: fix missed reload for watched files whose names contain .. (e.g. rules..yaml). Previously any path containing .. was silently skipped; now only Kubernetes synthetic entries whose basename starts with .. (e.g. ..data) are ignored. See #2253.

• BUGFIX: vmoperator: fix potential deadlock in operator_object_status metrics collector when the number of tracked objects exceeds 250. The Collect method previously held a mutex while sending to the prometheus channel, which could deadlock if the channel was full and another goroutine was waiting on the same mutex. See #2239.

v0.71.0

Update note 1: the new default preStop hook causes a rolling update of all applicable pods on operator upgrade (on Kubernetes >= 1.29). To avoid this, set VM_ENABLE_DEFAULT_PRESTOP_HOOK=false on the operator before upgrading. Once the upgrade is complete, you can re-enable it by removing the override (or setting it to true) to roll out the hook at a time of your choosing. Alternatively, disable the hook per resource by setting spec.preStopSleepSeconds: 0.

• Dependency: vmoperator: Updated default versions for VM apps to v1.145.0 version

• FEATURE: vmoperator: added VM_COMMON_LABELS and VM_COMMON_ANNOTATIONS environment variables to apply common labels/annotations to all Kubernetes resources managed by the operator. These cannot override labels/annotations already set by the operator or via spec.managedMetadata. This also ensures HTTPRoutes and PVCs include ManagedMetadata labels and annotations

• FEATURE: vmoperator: support enableServiceLinks property in all CRs. See #2194.

• FEATURE: vmalertmanagerconfig: add url_file and alert_source_token_file fields to IncidentioConfig, as file-based alternatives to url and alert_source_token. See #2222.

• FEATURE: vmoperator: add status metrics for objects managed by each controller. See #2238.

• FEATURE: vmsingle: added spec.downsampling for structured downsampling configuration and spec.retentionFilters for structured retention filters configuration. Both require an enterprise license.

• FEATURE: vmcluster: added spec.downsampling for structured downsampling configuration (applied to both vmselect and vmstorage) and spec.vmstorage.retentionFilters for structured retention filters configuration. Both require an enterprise license.

• FEATURE: vmoperator: add default preStop lifecycle hook (15s sleep) to applicable component pods to prevent traffic loss during pod termination. Requires Kubernetes >= 1.29. Storage components (VMStorage, VLStorage, VTStorage) are excluded since their clients handle retries. The sleep duration is configurable via spec.preStopSleepSeconds on any CR; set to 0 to disable. See #1834.

• FEATURE: vmanomaly: add missing configuration parameters for vmanomaly config parity: spec.reader.offset and per-query offset, spec.writer.connectionRetryAttempts, spec.server.useReaderConnectionSettings, settings.logger_levels, and the exact/infer_every parameters of the backtesting scheduler. Previously these keys were rejected by strict config parsing.

• FEATURE: vmcluster: add spec.discovery field with enabled, interval and filter properties to configure automatic vmstorage node discovery for vminsert and vmselect. The field can be set globally or overridden per component via spec.vminsert.discovery and spec.vmselect.discovery. This is an enterprise feature and requires a valid license key. See automatic vmstorage discovery.

• BUGFIX: vmoperator: update status currentRevision and currentReplicas for StatefulSet with OnDelete update strategy. See #1242.

• BUGFIX: config-reloader: fix configreloader_last_reload_success_timestamp_seconds metric to report time in seconds instead of milliseconds.

• BUGFIX: vmoperator: enable strict CR spec unmarshalling when creating objects. See #2882.

• BUGFIX: vmoperator: ignore NotFound errors, that may occur during attempt to update status on a missing resource.

• BUGFIX: vmagent, vmanomaly: fix VPA scale subresource lookup failure when spec.shardCount is unset by always reporting at least 1 in status.shards. See #2229.

• BUGFIX: vmagent: fix HPA targeting the underlying Deployment/StatefulSet (pod replicas) instead of the VMAgent CR scale subresource (spec.shardCount); HPA now correctly scales the number of shards. See #2229.

• BUGFIX: vmanomaly: emit the OnlineQuantileModel smoothing parameter under its correct key global_smoothing instead of the unrecognized global_smooth, which vmanomaly silently ignored.

• BUGFIX: vmanomaly: pass the configured TLS CA bundle to the reader, writer and monitoring clients. Previously the CA was mounted as a volume but dropped during config generation, so a tlsConfig with only a CA produced no verify_tls reference to it; insecureSkipVerify is now also propagated correctly.

• BUGFIX: vmanomaly: serialize the tz timezone of spec.reader, of each query, and of the periodic scheduler as a string. It was serialized to {} and could not be parsed from a string, so any configuration that set tz failed to reconcile.

• BUGFIX: vmanomaly: fix the AutoTunedModel trials parameter spelling n_trials, previously it was rendered as n_trails which was silently ignored by vmanomaly.

• BUGFIX: vmanomaly: preserve an explicit 0.0 for the anomaly_score_outside_data_range parameter (both settings and model level), which was dropped by the underlying float field and silently reverted to the vmanomaly default.

• BUGFIX: vmanomaly: validate online models' decay against the (0, 1] range while still allowing it to be omitted.

• BUGFIX: vmanomaly: accept a float value (for example 0.01) for the isolation forest contamination parameter in addition to the string auto.

• BUGFIX: vmanomaly: omit the OnlineQuantileModel min_subseason key when it is unset instead of emitting an empty string.

• BUGFIX: vmanomaly: remove the artificial upper bound on spec.server.maxConcurrentTasks; vmanomaly accepts any positive integer.

v0.68.5

Update note: -eula flag is not set by default anymore for VMBackup and VMRestore. To avoid VMCluster/VMSingle rollouts set spec.vmstorage.vmBackup.acceptEula: true for VMCluster and spec.vmBackup.acceptEula: true for VMSingle and replace it with spec.license during VMSingle/VMCluster upgrade.

• Dependency: vmoperator: Updated default versions for VM apps to v1.144.0 version

• SECURITY: upgrade Go builder from Go1.25.8 to Go1.25.10. See the list of issues addressed in Go1.25.10.

• BUGFIX: vmagent: use volume from spec.volumes as persistent queue volume if its name is persistent-queue-data, previously emptyDir was mounted. See #1677.

• BUGFIX: vmcluster: use volume from spec.vmstorage.volumes and spec.vmselect.volumes as data and cache volumes if its name is vmstorage-db and vmselect-cachedir respectively. See #784.

• BUGFIX: vmoperator: Improve reconcile error handling for Prometheus and VictoriaMetrics controllers.

• BUGFIX: vmoperator: Add acceptEula support for VMBackup/VMRestore.

• BUGFIX: vmdistributed: change default load balancing policy for write requests from first_available to least_loaded. This should allow to evenly distribute write load across all VMAgents.

• BUGFIX: VMCluster, VTCluster and VLCluster: fixed infinite non-default additional service recreation, when requestsLoadBalancer.enabled: true

• BUGFIX: vmoperator: retry reconcile errors, that may lead to expanding state, before resource could hang in expanding state.

• BUGFIX: vmdistributed: expose VMClusterSpec parsing error in status, previously it was just swallowed and led to infinite reconciles. See #2113.

• BUGFIX: vmanomaly and vmagent: Fix incorrect scaling logs for sharded vmagent and vmanomaly.

v0.70.1

• FEATURE: vmauth: support HPA for requests load balancer.

• BUGFIX: vmagent: skip replica count update when VMAgent is in stateful mode and HPA is enabled. See #2190.

• BUGFIX: vmalertmanagerconfig: Remove prefix from empty subroute receiver. See #2185.

• BUGFIX: vmoperator: updated OLM configuration to fix OpenShift catalog publishing. See #2195.