Skip to content

Commit e96d617

Browse files
define-nullclaude
andcommitted
docs: FIXME — don't advance schema bundle on shortage
During a shortage streak the published worker assignment stays frozen while the schema bundle keeps advancing. A worker joining mid-streak downloads the frozen assignment but can't resolve the GC'd schema of a still-named (removing) chunk — the failure schema_bundle_consistency checks for, currently masked by its excluded guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 7873289 commit e96d617

2 files changed

Lines changed: 13 additions & 0 deletions

File tree

src/multistep_scheduler/sim/sut.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -897,6 +897,11 @@ impl<D: SimStorage> SimUnderTest<D> {
897897
Some(id)
898898
}
899899
Err(StorageError::Shortage) => {
900+
// FIXME: on a shortage the published worker assignment stays frozen while the
901+
// schema bundle keeps advancing, so a worker joining mid-streak downloads the
902+
// frozen assignment and can't resolve the GC'd schema of a still-named (removing)
903+
// chunk. Fix: don't advance the schema bundle on `Shortage` — keep it in lockstep
904+
// with the worker assignment clients consume. See `schema_bundle_consistency`.
900905
self.schedule_status = ScheduleStatus::NotEnoughCapacity;
901906
self.is_infeasible = true;
902907
None

src/multistep_scheduler/sim/sut/placement_oracles.rs

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,14 @@ pub(super) fn published_coverage(
127127
/// `worker_assignment` is a cache only refreshed on a successful scheduling cycle, so during a
128128
/// shortage streak it can lag behind the freshly-regenerated `bundle` — a lagging chunk's schema
129129
/// may have already been legitimately GC'd from the bundle, which is not a resolution failure.
130+
///
131+
/// FIXME: the exclusion masks a real production hazard rather than a benign lag. During a shortage
132+
/// streak the published worker assignment stays frozen while the bundle keeps advancing, so a
133+
/// worker that joins mid-streak downloads that frozen assignment and then cannot resolve the
134+
/// GC'd schema of a still-named (removing) chunk — the exact failure this oracle exists to catch.
135+
/// Fix: don't advance the schema bundle when a cycle hits `Shortage`; keep it in lockstep with the
136+
/// worker assignment that clients actually consume, so a schema is never GC'd while an assignment
137+
/// still references its chunk. Once that holds, this exclusion can drop.
130138
pub(super) fn schema_bundle_consistency(
131139
bundle: &SchemaBundle,
132140
worker_assignment: Option<&WorkerAssignment>,

0 commit comments

Comments
 (0)