26.3.0

What's Changed

• Remove remote object from node when receiving distrib events by @Hamdy-khader in #743
• Add req id to RPC logs for spdk_proxy by @Hamdy-khader in #752
• Main lvol sync delete (#734) by @Hamdy-khader in #757
• R25.10 hotfix sfam 2471 by @Hamdy-khader in #758
• fix linter and type checker issues by @Hamdy-khader in #759
• Prometheus hostpath by @geoffrey1330 in #761
• Format nvme devices when run sbcli sn configure with --force by @wmousa in #760
• Nsocket env by @geoffrey1330 in #764
• Fix fw connection error handling not to set node status down by @Hamdy-khader in #765
• Fix sfam-2473 by @Hamdy-khader in #766
• Do not install pip package on cluster update by @Hamdy-khader in #749
• Fix calculate total_mem for multi sn nodes on same numa by @wmousa in #767
• use max_size instead as hugepage memory when set by @geoffrey1330 in #754
• refactor node add task runner by @Hamdy-khader in #768
• Fix sfam-2483 by @Hamdy-khader in #773
• fix logger by @Hamdy-khader in #776
• fix SFAM-2476 by @Hamdy-khader in #775
• Fix SFAM-2482 by @Hamdy-khader in #769
• use emptyDir memory medium as socket directory by @geoffrey1330 in #783
• addd endpoint bind_device_to_nvme in kubernetes by @geoffrey1330 in #784
• Create partitions and alcemls on node add in parallel by @Hamdy-khader in #763
• Remove stats from fdb and get it from Prometheus by @Hamdy-khader in #762
• R25.10 hotfix isolate by @wmousa in #792
• Fix sfam-2515 by @Hamdy-khader in #810
• R25.10 hotfix spdk proxy stats by @Hamdy-khader in #812
• Add --cores-percentage to sbctl sn configure and support oracle OS fo… by @wmousa in #813
• Use diff fw port per node by @Hamdy-khader in #817
• Return back max number of distribs to be 12 distribs by @wmousa in #827
• Fix sfam-2555 by @Hamdy-khader in #831
• Fix sfam-2556 by @Hamdy-khader in #835
• Fix node auto restart by @Hamdy-khader in #836
• Adds failed device back to cluster as a new one by @Hamdy-khader in #841
• Fix remove device function by @Hamdy-khader in #840
• fix sfam-2557 by @Hamdy-khader in #843
• Fix sfam-2564 by @Hamdy-khader in #838
• Fix node restart jm raid config by @Hamdy-khader in #851
• Fix sfam-2578 by @Hamdy-khader in #852
• Fix sfam-2579 by @Hamdy-khader in #850
• Adds sbctl -v/--version by @Hamdy-khader in #853
• R25.10 hotfix cluster id from main by @Hamdy-khader in #854
• Add --nvme-names to sn configure and --format-4k to sn add-node by @wmousa in #859
• Improve sync delete handling for lvol bdevs in lvol_monitor.py by @Hamdy-khader in #863
• R25.10 hotfix client data nic by @Hamdy-khader in #874
• Enhance graceful-startup and graceful-shutdown by @wmousa in #876
• R25.10 hotfix dump tree by @Hamdy-khader in #879
• fix-description of max-size by @wmousa in #881
• Add --calculate-hp-only to calculate the minimum required huge pages by @wmousa in #884
• Fix sfam-2620 by @Hamdy-khader in #886
• Adds link detection when pinging data nic by @Hamdy-khader in #897
• Add static numa tomplogy for openshift depoyment by @wmousa in #830
• Add LVol migration feature and related improvements by @schmidt-scaled in #915
• Stress run scheduler fix by @RaunakJalan in #914
• Adding dump validation by @RaunakJalan in #916
• Checking basic placement dump by @RaunakJalan in #917
• Fixing validation for placement dump by @RaunakJalan in #918
• Enable conditional inclusion of ndcs and npcs in lvol_create parameters by @Hamdy-khader in #919
• Test: Fixing fio timeout and merging auto restart fix by @RaunakJalan in #922
• delete fluent pod when removing spdk pod by @geoffrey1330 in #923
• Adding new test case for fault tolerance 2 by @RaunakJalan in #924
• Adding rdma multi outage scenario by @RaunakJalan in #925
• added nodeselector and tolerations by @geoffrey1330 in #928
• updated metrics labels and made prometheus url configurable by @geoffrey1330 in #930
• added prometheus_url and port env across all services by @geoffrey1330 in #931
• added helm dependency to chart by @geoffrey1330 in #932
• feat: add spdk_proxy_image support across cluster and storage node co… by @Hamdy-khader in #933
• R25.10 hotfix lvol clone (#934) by @Hamdy-khader in #941
• Security enabled lvol restore fix by @RaunakJalan in #943
• Checking concurrency param by @RaunakJalan in #926
• feat: add log collection script for Graylog/OpenSearch by @schmidt-scaled in #945
• added functionality to get k8s based logs by @geoffrey1330 in #947
• support foundationdb client arm64 on k8s by @geoffrey1330 in #948
• fix 500 error on cluster delete by @boddumanohar in #952
• Main snapshot replication by @Hamdy-khader in #938
• refactor: update backup controller to use original lvol crypto settings by @Hamdy-khader in #953
• fix lint and generate cli by @boddumanohar in #954
• New flag --enable-journal-device by @noctarius in #955
• Support for backup as a part of cluster create API by @boddumanohar in #946
• Cleaning up of the help texts for command, parameters, attributes by @noctarius in #949
• update openapi.json by @boddumanohar in #958
• raise duplicate cluster name as 409 conflict by @boddumanohar in #959
• Replace IPTables with NFTables in the Firewall wep app by @Hamdy-khader in #950
• Update storage_node.py by @geoffrey1330 in #963
• Adding snapshot backup, dhchap, k8s native cases by @RaunakJalan in #964
• Update init.py by @geoffrey1330 in #966
• Update init.py by @geoffrey1330 in #967
• Update init.py by @geoffrey1330 in #968
• Fixing yaml, sbcli cmd input, k8s runs by @RaunakJalan in #965
• K8s native cases by @RaunakJalan in #969
• Update DTOs with additional fields by @mxsrc in #971
• FTT2 restart refactor, remote-device reconciliation, collect_logs.py, FTT2 test suite by @schmidt-scaled in #956
• Small fixes for the documentation by @noctarius in #960
• fix(openshift): use cluster-scoped MCP with multi-node hostname selec… by @wmousa in #976
• Added list type which implements a comma-separated list by @noctarius in #977
• user ClusterFirstWithHostNet for spdk pod by @boddumanohar in #972
• wait for numa-align device plugin to become ready before exiting cpu… by @geoffrey1330 in #978
• added support for kubernetes services by @geoffrey1330 in #979
• Log API request execution duration by @mxsrc in #980
• added secondary_node_id param to snode get by @geoffrey1330 in #983
• Introduce SNodeClient helper by @mxsrc in #984
• Remove duplicate chart by @mxsrc in #982
• fix: handle empty secondary_node_id before cluster activation by @geoffrey1330 in #985
• Conditionally run TLS-based servers by @mxsrc in #973
• Fix invalid pattern for storage node addresses by @mxsrc in #987
• fix: delete stale spdk job before recreating to prevent 409 conflict … by @geoffrey1330 in #988
• Add tox.ini for convenient test execution by @mxsrc in #989
• Allow test execution customization by @mxsrc in #990
• update volume count on storagenode CR by @geoffrey1330 in #991
• restart: always examine when lvstore not yet surfaced by @schmidt-scaled in #992
• change spdk proxy startup command by @geoffrey1330 in #994
• restart: guard monitor + health service against clobbering in-progress restart by @schmidt-scaled in #995
• SFAM-2696: add host to lvols when added to pool; remove host from lvols when removed from pool by @Hamdy-khader in #981
• restart: data-plane quorum uses NVMe ctrlr state, not JM sync flag by @schmidt-scaled in #993
• restart: adopt peer's hublvol on takeover instead of recreating self's by @schmidt-scaled in #997
• restart: route dead-mgmt peers to takeover, shrink port-block budget by @schmidt-scaled in #996
• Adding input for partitions and jm by @RaunakJalan in #998
• scripts: deterministic 80-scenario outage enumeration in mixed soak by @schmidt-scaled in #999
• support target_node_id in RestoreParams by @boddumanohar in #1000
• fix: wait for numa plugin pod readiness before exiting cpu topology j… by @geoffrey1330 in #1001
• tasks_runner_port_allow: include STATUS_DOWN in candidate filtering logic by @Hamdy-khader in #1002
• Add API endpoint and CLI command to retrieve master lvols by storage pool by @Hamdy-khader in #1005
• tasks_runner_port_allow: pass primary node ID to secondary hublvol check by @Hamdy-khader in #1007
• fix cross cluster backup restore by @boddumanohar in #1004
• Optionally enable TLS by @mxsrc in #1003
• cross cluster restore: fix issue with source-switch to local by @boddumanohar in #1008
• setup LVOL status as restore_failed on failed restore by @boddumanohar in #1009
• updated simplyblock prometheus configmap name by @geoffrey1330 in #1010
• Use redhat:10 base image by @Hamdy-khader in #986
• run spdk proxy without sudo by @geoffrey1330 in #1012
• run with sudo command by @geoffrey1330 in #1014
• expose backup export, source-switch, and sources REST endpoints by @boddumanohar in #1015
• fix(node_utils_k8s): treat 404 on job read as successful completion i… by @geoffrey1330 in #1020
• fix(kubernetes): ignore 404 when deleting SPDK job that was already g… by @geoffrey1330 in #1021
• Fix/snapshot delete race 2 by @Hamdy-khader in #1019
• Fix Lvol connect API response by @boddumanohar in #1024
• Enable mTLS by @mxsrc in #1026
• Fix comparison for TLS auth check by @boddumanohar in #1029
• Fix TLS context helper by @mxsrc in #1030
• support host_nqn as a part of lvol connect API by @boddumanohar in #1028
• Adds --namespaced to lvol add/ and snapshot clone by @Hamdy-khader in #1027
• K8s native yaml, kubernetes changes, backup restore cases, e2e tests improve by @RaunakJalan in #1032
• K8s native yaml by @RaunakJalan in #1033
• K8s native yaml by @RaunakJalan in #1034
• Fixing conflict and stale data issue by @RaunakJalan in #1035
• Fixing client disconnect for k8s test by @RaunakJalan in #1036
• chore: update openapi.json by @boddumanohar in #1037
• Add missing fields for v2 API for clone by @boddumanohar in #1039
• update tag on SIMPLY_BLOCK_DOCKER_IMAGE by @boddumanohar in #1041
• reject duplicate pool name only in the current namespace by @boddumanohar in #1042
• show response body for clone by @boddumanohar in #1040
• fix: update RPC port assignment to use next NVMF port for cluster by @Hamdy-khader in #1043
• WebApp logs: add client_ip, user_agent, request_size, response_size by @boddumanohar in #1045
• Introduce integration with external Hashicorp Vault KMS by @mxsrc in #1031
• fix: remove lvol subsystem from tertiary and secondary nodes first by @Hamdy-khader in #1050
• lvol delete: delete DEK only when lvol.crypto_bdev is present by @boddumanohar in #1052
• docs/kms: update vault cert namespace by @boddumanohar in #1047
• KMS: parameterise transit mount, KV mount and cert role by @boddumanohar in #1049
• setup security.yml by @boddumanohar in #1053
• Implementation of cli command parameter deprecation by @noctarius in #957
• Promoted cluster graceful-shutdown and cluster graceful-startup by @noctarius in #961
• fix security scan CVEs by @boddumanohar in #1054
• tls: update condition in Unicorn SSL configuration by @boddumanohar in #1046
• K8s native yaml by @RaunakJalan in #1038
• Fixing different test changes and pipeline issues by @RaunakJalan in #1059
• pip python kubernetes client version to 35.0.0 by @boddumanohar in #1061
• feat: add --pool option to list snapshots by pool id or name by @Hamdy-khader in #1055
• Main fix lvol by @wmousa in #1062
• Enhamce lvol clone speed by @Hamdy-khader in #1066
• Bump indicated minimum python version by @mxsrc in #1071
• fix: add error handling for cluster retrieval in monitoring services by @Hamdy-khader in #1068
• fix: add interface name pattern validation to PingQuery model in dock… by @Hamdy-khader in #1063
• Validate NQN argument in SNodeAPI by @mxsrc in #1072
• Prevent cross-cluster resource access by @mxsrc in #1070
• fixed collection of docker control plane service logs by @geoffrey1330 in #1073
• Revert "connect_lvol: stop unconditionally injecting pool DHCHAP keys" by @Hamdy-khader in #1074
• SFAM-2703: change log level from exception to error by @Hamdy-khader in #1006
• Fix TLS file validation by @mxsrc in #1076
• hugepages: preserve user's pre-existing reservation per NUMA node by @wmousa in #1077
• add-node: add --spdk-sys-mem option to override minimum system memory by @wmousa in #1079
• Dockerfile: dnf install the latest version of libcap openssh systemd by @boddumanohar in #1080
• Remove outdated API integration tests by @mxsrc in #1081
• Improve LVol cloning speed by @Hamdy-khader in #1078
• Look for available subsystem on node from all online lvols by @Hamdy-khader in #1082
• Harden failure handling & concurrency (audit items 1–3) by @schmidt-scaled in #1084
• Fix sfam-2747: connect lvols to their respective pools based on QOS settings by @Hamdy-khader in #1086
• sfam-2723: add merged status to backup process and update old backup status by @Hamdy-khader in #1087
• Update prometheus.yml.j2 by @geoffrey1330 in #1088
• Prune stale contents by @mxsrc in #1095
• Fix volume resize on equal size by @mxsrc in #1096
• Introduce updated authentication/authorization by @mxsrc in #1085
• Introduce status checks to API v2 by @mxsrc in #1099
• Toggle cluster secret auth by @mxsrc in #1100
• Fix cluster creation API overprovisioning by @mxsrc in #1102
• added command to restart k3s-agent by @geoffrey1330 in #1092
• feat: support custom OpenShift MachineConfigPool via OPENSHIFT_MCP en… by @geoffrey1330 in #1093
• Avoid github bot complaints on generated CLI code by @mxsrc in #1106
• Restructure API v2 module by @mxsrc in #1105
• Fix minor issues in API v2 by @mxsrc in #1107
• Make API creation responses concistent by @mxsrc in #1094
• fix: extend delete_lvol and hublvol creation RPCs with additional parameters by @Hamdy-khader in #1110
• POST /snapshot: return error if value already exists by @boddumanohar in #1108
• fix: correct route param from volume_id to task_id in storage node ad… by @geoffrey1330 in #1112
• Lvol migration fresh hamdy by @Hamdy-khader in #1114
• Deduplicate DHCHAP key registration by @mxsrc in #1109
• fix snapshot API response by @boddumanohar in #1113
• register missing clusters:backups:detail route by @boddumanohar in #1116
• Fix type check errors by @mxsrc in #1117
• Avoid lazy imports in migration API by @mxsrc in #1118
• Adding namespace docker fix by @RaunakJalan in #1060
• Lvol migration fresh by @EbiRider in #1098
• Convert lvol deletion error handling to exceptions by @mxsrc in #1115
• follow up PR for lvol-migration-fresh by @EbiRider in #1120
• fix: correct route name from volumes:detail to storage-pools:detail i… by @geoffrey1330 in #1123
• Use established database helpers in API by @mxsrc in #1119
• Fix incompatibility lvol deletion tests by @mxsrc in #1121
• Single-node cluster expansion (FTT1/FTT2) by @schmidt-scaled in #1097
• Fix full API responses by @mxsrc in #1124
• Move migration API to correct location by @mxsrc in #1125
• Add instructions for AI agents by @mxsrc in #1122
• Fix migration creation by @mxsrc in #1126
• Fix migration import test by @mxsrc in #1129
• Deduplicate cloud info by @mxsrc in #1130
• prevent lvol monitor from handling states for lvol restore by @boddumanohar in #1131
• Migration fix by @EbiRider in #1133
• rebuild base image daily by @boddumanohar in #1067
• Remove user-agent from API log by @mxsrc in #1134
• Prevent secret leakage by @mxsrc in #1103
• Add node mass create obj by @RaunakJalan in #1128
• Categorize tests into unit and integration by @mxsrc in #1136
• Fixing resilient k8s tests and summary cases, adding mass creation max entity tests by @RaunakJalan in #1139
• Fix secret rendering by @mxsrc in #1140
• Fix secret serialization failure in API v1 by @mxsrc in #1141
• Restore compatibility with Python 3.9 by @mxsrc in #1138
• fix: add backup encryption key management to KMS and integrate with backup workflows by @Hamdy-khader in #1111
• Fixing mass create tests and adding log flush to shared path by @RaunakJalan in #1144
• Adding collect log pipeline and reducing threads for mass create to 10 by @RaunakJalan in #1145
• Instruct agents to look for local overrides by @mxsrc in #1151
• Fix SFAM 2776 by @mxsrc in #1152
• Support more appropriate methods for volume actions by @mxsrc in #1148
• Validate Host NQN in API by @mxsrc in #1147
• Harden database accessors by @mxsrc in #1150
• Fixing k8s backup cases by @RaunakJalan in #1146
• fix(health): parse node status as a token so SPDK's failure_domain field doesn't break health checks by @noctarius in #1153
• Fix and coordinate integration tests by @mxsrc in #1154
• Speed up cluster duplicate check by @mxsrc in #1156
• Migration in suspended nodes by @EbiRider in #1159
• allowing node to be suspended for overlap checks on migration by @EbiRider in #1161
• Enable FTT2 and expansion-sim integration tests by @mxsrc in #1160
• Main reduce hp by @wmousa in #1142
• Fix SFAM 2775 by @mxsrc in #1149
• Add API v2 tests by @mxsrc in #1164
• feat: add failure_domain support to storage node and cluster v2 API by @geoffrey1330 in #1168
• fix: update unit tests for enable_failure_domain and failure_domain f… by @geoffrey1330 in #1169
• Fixed hyphenated JSON properties on NvmeConnectEntry by @noctarius in #1170
• Enable migration tests by @mxsrc in #1167
• Run migration tests in separate job by @mxsrc in #1171
• Fix SFAM-2774 by @Hamdy-khader in #1158
• Enforce retry limit on backup tasks by @mxsrc in #1172
• Speed up API unit tests by @mxsrc in #1173
• Various fixes, features, and cleanup by @mxsrc in #1174
• Filter subsystems on server-side by @mxsrc in #1177
• Introduce cluster expansion to the API by @mxsrc in #1178
• Migration followup by @EbiRider in #1135
• Main fdb backup by @Hamdy-khader in #1176
• Add: FoundationDB exporter to Prometheus and Docker Swarm configuration by @Hamdy-khader in #1181
• Fix the Kubernetes Node Add Cycling by @noctarius in #1183
• add: ClusterConfig for managing cluster-wide settings and enhance cluster creation logic by @Hamdy-khader in #1182
• add: implement task retention mechanism to delete old cluster tasks and logs after 30 days by @Hamdy-khader in #1184
• Migration followup-2 by @EbiRider in #1180
• Fix stuck activation rebased 1 by @wmousa in #1179
• feat(node-removal): online node removal as inverse of cluster expansion by @schmidt-scaled in #1104
• Fix bdev re-creation on subsystem-full retry by @RaunakJalan in #1185
• Fixing pipelines with timeout and node health check continuous retry by @RaunakJalan in #1155
• Deduplicate connection strings by @mxsrc in #1188
• Remove unused helper by @mxsrc in #1187
• add disable_monitoring field to DeployConfig by @boddumanohar in #1186
• Fixing add node migration backupup tests by @RaunakJalan in #1189
• Replace use of deprecated functions by @mxsrc in #1191
• Fix CLI generation by @mxsrc in #1192
• Add restart phase for mass create tests by @RaunakJalan in #1190
• Fix unit tests by @mxsrc in #1194
• Remove misleading comment by @mxsrc in #1196
• Adding reserved cpu set in openshift baremetal k8s by @RaunakJalan in #1197
• Deduplicate RPC client by @mxsrc in #1195
• Multipath migration by @EbiRider in #1199
• Main fix add node by @wmousa in #1200
• reworked cleanup implementation for lvol migration by @EbiRider in #1201
• Fixing reserved cpu set in openshift baremetal k8s and changing default values and API parity params change by @RaunakJalan in #1198
• Explicitly ignore existing linter violations by @mxsrc in #1206
• fix(security-scan): stop shipping dev caches; ignore pip-vendored CVEs by @boddumanohar in #1202
• Migration test by @EbiRider in #1205
• fix(cpu-alloc): colocate lvol_poller with jc-singleton, not app_thread by @wmousa in #1203
• Enable execution checks in linter by @mxsrc in #1209
• Update tests.yml by @schmidt-scaled in #1212
• Convert CLI generation to tox-based execution by @mxsrc in #1213
• Bump workflow action versions by @mxsrc in #1214
• Remove outdated systemd service instrumentation by @mxsrc in #1215
• Fix regression in type check fix by @mxsrc in #1216
• Fix/jc disable replication args by @wmousa in #1217
• batch migration web api endpoints by @EbiRider in #1211
• Fix integration tests by @mxsrc in #1220
• cleanup fix: failing sync delete on nodes after async delete, rename fix by @EbiRider in #1219
• Introduce tenacity by @mxsrc in #1221
• fix(port_block): add support for FirewallClient based on SPDK version by @Hamdy-khader in #1223
• Make test execution reproducible by @mxsrc in #1227
• fix(add-node): always cleanup stale in_creation record, even if anoth… by @wmousa in #1222
• fix(node-removal): use subsystem_get instead of misused subsystem_list by @wmousa in #1230
• fix: don't auto-restart k8s-adopted nodes before CR-linkage by @boddumanohar in #1231
• Fix migration outage test by @mxsrc in #1228
• Avoid node-info computation at module load by @mxsrc in #1233
• migration controllers merged under subssytem controller by @EbiRider in #1229
• Add stuck pod describe, fix docker and k8s upgrade tests, pipeline, backup fixes, cross cluster restores by @RaunakJalan in #1208
• fix: sync Pool->StoragePool CRD rename and maxLogicalVolumeCount->maxSubsystemCount field rename by @boddumanohar in #1238
• Lvol migration test fixes by @EbiRider in #1234
• Add kubectl install to Dockerfile with multi-arch support by @geoffrey1330 in #1237
• Remove unsupported parameter by @mxsrc in #1239
• fix(alerting): use ratio query for cluster capacity alerts to preserv… by @geoffrey1330 in #1245
• Fix SFAM 2796 by @mxsrc in #1242
• Introduce metrics support to API v2 by @mxsrc in #1236
• Explicitly disable FDB backups for kubernetes by @mxsrc in #1248
• Refuse import of present backups by @mxsrc in #1243
• Remove second scraper for forward compatibility by @mxsrc in #1251
• Fix cross-cluster backup/restore, upgrade tests, resource preservation, CI pipelines, and Slack thread notifications by @RaunakJalan in #1240
• Compute node info lazily by @mxsrc in #1252
• fix: scope storage-node cluster status updates to the node's cluster by @mxsrc in #1253
• Fix types by @mxsrc in #1254
• Report pool capacity in API by @mxsrc in #1260
• Align replication API with general style by @mxsrc in #1257
• Feat/cluster spdk sizing by @wmousa in #1263
• Feat/cluster spdk sizing by @wmousa in #1264
• feat(replication): hold cutover_pending per-task for operator preconn… by @geoffrey1330 in #1262
• [R26.3] fix: resolve API v1 metric leak by @mxsrc in #1273
• distrib/poller take priority over alceml in core allocation (R26.3) by @wmousa in #1278
• JM/failure-domain node-removal fixes + node-removal-status rework (R26.3) by @wmousa in #1280
• batch migration checksum issue workaround (R26.3) by @wmousa in #1275
• Fix model defaults by @mxsrc in #1277
• fix(docker): upgrade inherited packages explicitly by @boddumanohar in #1300
• Fix/graceful shutdown by @wmousa in #1303
• feat(alerting): cluster event log alert rules for docker clusters (backport of #1301 to R26.3) by @boddumanohar in #1308
• Backport: Batch migration delay (#1288) to R26.3 by @wmousa in #1311

Full Changelog: 25.10.3...26.3.0