Skip to content

Conversation

@nvartolomei
Copy link
Contributor

config_store::read_yaml() would apply property values even when custom
validation failed, only recording the error. This meant invalid configuration
values would silently take effect despite being reported as errors.

This behavior was also inconsistent: conversion exceptions (e.g.,
YAML::BadConversion) correctly prevented values from being set, but
custom validator errors did not.

The fix moves set_value() inside the else branch so values are only applied
when validation succeeds:

  1. Optional properties now correctly retain their default value when validation
    fails.

  2. Required properties now correctly throw std::invalid_argument when
    validation fails. Previously, the throw logic was bypassed because ok was
    set to true even after validation errors.

It's unclear why we don't stop server startup on invalid optional config values.
To avoid breaking something I don't fully understand, this change takes the
conservative approach of retaining defaults.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.3.x
  • v25.2.x
  • v25.1.x

Release Notes

Bug Fixes

  • Fixed config validation to not apply values that fail custom validators, matching the behavior for type conversion errors.

Copilot AI review requested due to automatic review settings December 24, 2025 12:35
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a bug in config_store::read_yaml() where property values would be incorrectly applied even when custom validation failed. The fix ensures that invalid configuration values are rejected and properties retain their default values, matching the behavior for type conversion errors.

Key changes:

  • Moved set_value() call inside the else branch so values are only applied when validation succeeds
  • Added comprehensive test coverage for validation failure scenarios including optional properties, required properties, and throwing validators

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
src/v/config/config_store.h Fixed validation logic to prevent setting values when custom validators fail
src/v/config/tests/config_store_test.cc Added test fixtures and test cases for validation failure scenarios
Comments suppressed due to low confidence (1)

src/v/config/tests/config_store_test.cc:16

  • Two header includes were removed (#include <seastar/core/thread.hh> and #include ) but they may still be needed elsewhere in the file. Verify that these headers are truly unused throughout the entire file before removing them.
#include <seastar/testing/thread_test_case.hh>

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Dec 24, 2025

Retry command for Build#78392

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_shadow_indexing_default_local_retention@{"cloud_storage_type":1,"cluster_remote_write":false,"topic_remote_write":null}
tests/rptest/tests/adjacent_segment_merging_test.py::AdjacentSegmentMergingToggleCompactionTest.test_reupload_of_local_segments@{"acks":1,"cloud_storage_type":1}
tests/rptest/tests/nodes_decommissioning_test.py::NodesDecommissioningTest.test_multiple_decommissions@{"cloud_topic":true}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_shadow_indexing_default_local_retention@{"cloud_storage_type":2,"cluster_remote_write":false,"topic_remote_write":"false"}
tests/rptest/tests/adjacent_segment_merging_test.py::AdjacentSegmentMergingToggleCompactionTest.test_reupload_of_local_segments@{"acks":-1,"cloud_storage_type":2}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_local_time_based_retention_is_overridden@{"cloud_storage_type":2,"local_retention_ms":3600000}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_shadow_indexing_default_local_retention@{"cloud_storage_type":2,"cluster_remote_write":true,"topic_remote_write":"false"}
tests/rptest/tests/e2e_shadow_indexing_test.py::EndToEndSpilloverTest.test_spillover@{"cloud_storage_type":2}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingCloudRetentionTest.test_cloud_retention_deleted_segments_count@{"cloud_storage_type":1}
tests/rptest/tests/adjacent_segment_merging_test.py::AdjacentSegmentMergingToggleCompactionTest.test_reupload_of_local_segments@{"acks":1,"cloud_storage_type":2}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_shadow_indexing_default_local_retention@{"cloud_storage_type":2,"cluster_remote_write":false,"topic_remote_write":null}
tests/rptest/tests/offset_for_leader_epoch_archival_test.py::OffsetForLeaderEpochArchivalTest.test_querying_archive
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_local_time_based_retention_is_overridden@{"cloud_storage_type":1,"local_retention_ms":-1}
tests/rptest/tests/partition_movement_test.py::SIPartitionMovementTest.test_shadow_indexing@{"cloud_storage_type":2,"num_to_upgrade":0,"with_cloud_topics":true}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingCloudRetentionTest.test_cloud_retention_deleted_segments_count@{"cloud_storage_type":2}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_local_time_based_retention_is_overridden@{"cloud_storage_type":2,"local_retention_ms":-1}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_shadow_indexing_default_local_retention@{"cloud_storage_type":1,"cluster_remote_write":false,"topic_remote_write":"false"}
tests/rptest/tests/adjacent_segment_merging_test.py::AdjacentSegmentMergingToggleCompactionTest.test_reupload_of_local_segments@{"acks":-1,"cloud_storage_type":1}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_local_time_based_retention_is_overridden@{"cloud_storage_type":1,"local_retention_ms":3600000}
tests/rptest/tests/retention_policy_test.py::ShadowIndexingLocalRetentionTest.test_shadow_indexing_default_local_retention@{"cloud_storage_type":1,"cluster_remote_write":true,"topic_remote_write":"false"}
tests/rptest/tests/e2e_shadow_indexing_test.py::EndToEndSpilloverTest.test_spillover@{"cloud_storage_type":1}

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Dec 24, 2025

CI test results

test results on build#78392
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
AdjacentSegmentMergingToggleCompactionTest test_reupload_of_local_segments {"acks": -1, "cloud_storage_type": 2} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15bc-4b93-b0e3-aa7cea505013 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AdjacentSegmentMergingToggleCompactionTest&test_method=test_reupload_of_local_segments
AdjacentSegmentMergingToggleCompactionTest test_reupload_of_local_segments {"acks": -1, "cloud_storage_type": 2} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221c-4dc1-b381-567ab280ab0e FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AdjacentSegmentMergingToggleCompactionTest&test_method=test_reupload_of_local_segments
AdjacentSegmentMergingToggleCompactionTest test_reupload_of_local_segments {"acks": -1, "cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15be-4c57-901f-80b12de38c70 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AdjacentSegmentMergingToggleCompactionTest&test_method=test_reupload_of_local_segments
AdjacentSegmentMergingToggleCompactionTest test_reupload_of_local_segments {"acks": -1, "cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221d-4e48-8218-3133b687f030 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AdjacentSegmentMergingToggleCompactionTest&test_method=test_reupload_of_local_segments
AdjacentSegmentMergingToggleCompactionTest test_reupload_of_local_segments {"acks": 1, "cloud_storage_type": 2} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15bf-4229-89e4-3979c178ea77 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AdjacentSegmentMergingToggleCompactionTest&test_method=test_reupload_of_local_segments
AdjacentSegmentMergingToggleCompactionTest test_reupload_of_local_segments {"acks": 1, "cloud_storage_type": 2} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221e-4291-b363-fcc67cb88b5d FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AdjacentSegmentMergingToggleCompactionTest&test_method=test_reupload_of_local_segments
AdjacentSegmentMergingToggleCompactionTest test_reupload_of_local_segments {"acks": 1, "cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c0-4871-b060-a67c670987e3 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AdjacentSegmentMergingToggleCompactionTest&test_method=test_reupload_of_local_segments
AdjacentSegmentMergingToggleCompactionTest test_reupload_of_local_segments {"acks": 1, "cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-2220-4299-a4cb-0ec29d746d10 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AdjacentSegmentMergingToggleCompactionTest&test_method=test_reupload_of_local_segments
DatalakeClusterRestoreTest test_restore_partition_spec {"catalog_type": "rest_hadoop", "cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c7-4ff3-a1f0-5f681f69670d FLAKY 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0041, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DatalakeClusterRestoreTest&test_method=test_restore_partition_spec
EndToEndSpilloverTest test_spillover {"cloud_storage_type": 2} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15bc-4b93-b0e3-aa7cea505013 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=EndToEndSpilloverTest&test_method=test_spillover
EndToEndSpilloverTest test_spillover {"cloud_storage_type": 2} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221c-4dc1-b381-567ab280ab0e FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=EndToEndSpilloverTest&test_method=test_spillover
EndToEndSpilloverTest test_spillover {"cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15be-4c57-901f-80b12de38c70 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=EndToEndSpilloverTest&test_method=test_spillover
EndToEndSpilloverTest test_spillover {"cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221d-4e48-8218-3133b687f030 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=EndToEndSpilloverTest&test_method=test_spillover
NodesDecommissioningTest test_multiple_decommissions {"cloud_topic": true} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c6-4d9d-9517-b351b5da4008 FAIL 0/1 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=NodesDecommissioningTest&test_method=test_multiple_decommissions
OffsetForLeaderEpochArchivalTest test_querying_archive null integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15bf-4229-89e4-3979c178ea77 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=OffsetForLeaderEpochArchivalTest&test_method=test_querying_archive
OffsetForLeaderEpochArchivalTest test_querying_archive null integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221e-4291-b363-fcc67cb88b5d FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=OffsetForLeaderEpochArchivalTest&test_method=test_querying_archive
SIPartitionMovementTest test_shadow_indexing {"cloud_storage_type": 2, "num_to_upgrade": 0, "with_cloud_topics": true} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c9-4007-aaa7-0e760b223f4a FLAKY 8/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0024, p0=0.0003, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=SIPartitionMovementTest&test_method=test_shadow_indexing
ShadowIndexingCloudRetentionTest test_cloud_retention_deleted_segments_count {"cloud_storage_type": 2} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c9-4007-aaa7-0e760b223f4a FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingCloudRetentionTest&test_method=test_cloud_retention_deleted_segments_count
ShadowIndexingCloudRetentionTest test_cloud_retention_deleted_segments_count {"cloud_storage_type": 2} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-2227-4132-b79e-78fc3fb527db FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingCloudRetentionTest&test_method=test_cloud_retention_deleted_segments_count
ShadowIndexingCloudRetentionTest test_cloud_retention_deleted_segments_count {"cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15bc-4b93-b0e3-aa7cea505013 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingCloudRetentionTest&test_method=test_cloud_retention_deleted_segments_count
ShadowIndexingCloudRetentionTest test_cloud_retention_deleted_segments_count {"cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221c-4dc1-b381-567ab280ab0e FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingCloudRetentionTest&test_method=test_cloud_retention_deleted_segments_count
ShadowIndexingLocalRetentionTest test_local_time_based_retention_is_overridden {"cloud_storage_type": 2, "local_retention_ms": -1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c7-4ff3-a1f0-5f681f69670d FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_local_time_based_retention_is_overridden
ShadowIndexingLocalRetentionTest test_local_time_based_retention_is_overridden {"cloud_storage_type": 2, "local_retention_ms": -1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-2226-4e7b-961d-e81aaf6a47e9 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_local_time_based_retention_is_overridden
ShadowIndexingLocalRetentionTest test_local_time_based_retention_is_overridden {"cloud_storage_type": 1, "local_retention_ms": -1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c9-4007-aaa7-0e760b223f4a FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_local_time_based_retention_is_overridden
ShadowIndexingLocalRetentionTest test_local_time_based_retention_is_overridden {"cloud_storage_type": 1, "local_retention_ms": -1} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-2227-4132-b79e-78fc3fb527db FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_local_time_based_retention_is_overridden
ShadowIndexingLocalRetentionTest test_local_time_based_retention_is_overridden {"cloud_storage_type": 2, "local_retention_ms": 3600000} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15bc-4b93-b0e3-aa7cea505013 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_local_time_based_retention_is_overridden
ShadowIndexingLocalRetentionTest test_local_time_based_retention_is_overridden {"cloud_storage_type": 2, "local_retention_ms": 3600000} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221c-4dc1-b381-567ab280ab0e FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_local_time_based_retention_is_overridden
ShadowIndexingLocalRetentionTest test_local_time_based_retention_is_overridden {"cloud_storage_type": 1, "local_retention_ms": 3600000} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15be-4c57-901f-80b12de38c70 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_local_time_based_retention_is_overridden
ShadowIndexingLocalRetentionTest test_local_time_based_retention_is_overridden {"cloud_storage_type": 1, "local_retention_ms": 3600000} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221d-4e48-8218-3133b687f030 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_local_time_based_retention_is_overridden
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 2, "cluster_remote_write": false, "topic_remote_write": null} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15bf-4229-89e4-3979c178ea77 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 2, "cluster_remote_write": false, "topic_remote_write": null} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221e-4291-b363-fcc67cb88b5d FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 1, "cluster_remote_write": false, "topic_remote_write": null} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c0-4871-b060-a67c670987e3 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 1, "cluster_remote_write": false, "topic_remote_write": null} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-2220-4299-a4cb-0ec29d746d10 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 2, "cluster_remote_write": false, "topic_remote_write": "false"} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c2-403c-9af7-58f401e645ae FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 2, "cluster_remote_write": false, "topic_remote_write": "false"} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-2221-43c8-8346-25829c2a04c6 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 1, "cluster_remote_write": false, "topic_remote_write": "false"} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15c3-4a31-988d-41e4a5d0a3c7 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 1, "cluster_remote_write": false, "topic_remote_write": "false"} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-2222-4b29-ace3-a3521d966ac0 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 2, "cluster_remote_write": true, "topic_remote_write": "false"} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15bc-4b93-b0e3-aa7cea505013 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 2, "cluster_remote_write": true, "topic_remote_write": "false"} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221c-4dc1-b381-567ab280ab0e FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 1, "cluster_remote_write": true, "topic_remote_write": "false"} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50a2-15be-4c57-901f-80b12de38c70 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
ShadowIndexingLocalRetentionTest test_shadow_indexing_default_local_retention {"cloud_storage_type": 1, "cluster_remote_write": true, "topic_remote_write": "false"} integration https://buildkite.com/redpanda/redpanda/builds/78392#019b50b9-221d-4e48-8218-3133b687f030 FAIL 0/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowIndexingLocalRetentionTest&test_method=test_shadow_indexing_default_local_retention
src/v/cloud_topics/level_one/compaction/tests/scheduler_test src/v/cloud_topics/level_one/compaction/tests/scheduler_test unit https://buildkite.com/redpanda/redpanda/builds/78392#019b505c-51fd-4024-b9a2-6a9c24f7659e FAIL 0/1
test results on build#78408
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
ShadowLinkingMetricsTests test_link_metrics null integration https://buildkite.com/redpanda/redpanda/builds/78408#019b5268-a6b1-4e6e-b8d9-7fa72fa80780 FLAKY 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0000, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingMetricsTests&test_method=test_link_metrics
ShadowLinkingReplicationTests test_replication_with_transactions null integration https://buildkite.com/redpanda/redpanda/builds/78408#019b5268-a6b5-4302-a893-390101517206 FLAKY 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0053, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_replication_with_transactions
test results on build#78435
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
ClusterConfigBoundedPropertyTest test_bounded_property_clamped_to_minimum null integration https://buildkite.com/redpanda/redpanda/builds/78435#019b73e9-212e-4892-b9d4-0c475d4e9a5f FAIL 0/11 The test was found to be new, and no failures are allowed https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterConfigBoundedPropertyTest&test_method=test_bounded_property_clamped_to_minimum
ClusterConfigBoundedPropertyTest test_bounded_property_clamped_to_minimum null integration https://buildkite.com/redpanda/redpanda/builds/78435#019b73ed-ac47-41ba-a093-985b2c8b8425 FAIL 0/11 The test was found to be new, and no failures are allowed https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterConfigBoundedPropertyTest&test_method=test_bounded_property_clamped_to_minimum
NodesDecommissioningTest test_decommissioning_crashed_node {"cloud_topic": true} integration https://buildkite.com/redpanda/redpanda/builds/78435#019b73ed-ac51-4620-ab9a-44192129817e FLAKY 15/21 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0072, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=NodesDecommissioningTest&test_method=test_decommissioning_crashed_node
SIPartitionMovementTest test_cross_shard {"cloud_storage_type": 2, "num_to_upgrade": 0, "with_cloud_topics": true} integration https://buildkite.com/redpanda/redpanda/builds/78435#019b73ed-ac4a-4d47-bad7-478191001d8a FLAKY 15/21 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0130, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=SIPartitionMovementTest&test_method=test_cross_shard
WriteCachingFailureInjectionE2ETest test_crash_all {"use_transactions": false} integration https://buildkite.com/redpanda/redpanda/builds/78435#019b73ed-ac4c-47b6-ab96-c53bd44e6b1b FLAKY 9/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0762, p0=0.5472, reject_threshold=0.0100. adj_baseline=0.2115, p1=0.3420, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=WriteCachingFailureInjectionE2ETest&test_method=test_crash_all

@nvartolomei nvartolomei force-pushed the nv/config-invalid-values branch from dfed747 to 4b953f7 Compare December 24, 2025 21:40
rockwotj
rockwotj previously approved these changes Dec 27, 2025
Copy link
Contributor

@rockwotj rockwotj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a little worried if there are people taking advantage of this validation being skipped in the config bootstrapping. I think doing the conservative thing is right here (And I think to answer your question about why the config system behaves in a certain way is because it was done very quickly).

@nvartolomei
Copy link
Contributor Author

todo for myself: add a test that shows that for bounded properties instead of ignoring the value we actually bound it and apply it. better backwards compat and sensible behavior.

Test read_yaml behavior with validator errors, type mismatches, and
throwing validators.
Previously, config_store::read_yaml() would apply property values even
when custom validation failed, only recording the error. This meant
invalid configuration values would silently take effect despite being
reported as errors.

This behavior was also inconsistent: conversion exceptions (e.g.,
YAML::BadConversion) correctly prevented values from being set, but
custom validator errors did not.

Fix by moving set_value() inside the else branch so values are only
applied when validation succeeds. This has two effects:

1. Optional properties now correctly retain their default value when
   validation fails.

2. Required properties now correctly throw std::invalid_argument when
   validation fails. Previously, the throw logic was bypassed because
   ok was set to true even after validation errors.

Note: It's unclear why we don't stop server startup on invalid optional
config values. To avoid breaking something I don't fully understand,
this change takes the conservative approach of retaining defaults.
@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Dec 31, 2025

Retry command for Build#78435

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/cluster_config_test.py::ClusterConfigBoundedPropertyTest.test_bounded_property_clamped_to_minimum
tests/rptest/tests/partition_movement_test.py::SIPartitionMovementTest.test_cross_shard@{"cloud_storage_type":2,"num_to_upgrade":0,"with_cloud_topics":true}
tests/rptest/tests/nodes_decommissioning_test.py::NodesDecommissioningTest.test_decommissioning_crashed_node@{"cloud_topic":true}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants