Skip to content

Commit e5b09f3

Browse files
mraszykclaude
andauthored
fix: remove the ordering on CompoundCycles (#11529)
# Summary `CompoundCycles` pairs a **real** amount, which moves the canister's cycles balance, with a **nominal** one, which feeds the consumed cycles metrics. It derived `Ord`, which orders lexicographically, i.e. by the real part first, so a comparison ignored the nominal parts whenever the real parts differed. There is no meaningful order on a pair of independently accounted amounts, and that one is actively wrong for what the cycles accounting in `ic-cycles-account-manager` compares: an amount recorded when a call is performed against one derived when its response is executed. Under the free cost schedule the real part of an amount made free is zero however large its nominal part, so the amount made free always compares as the smaller one. The derive is removed, so such a comparison is now a compile error, and the type documents why it has no ordering. This also corrects a claim in the same doc comment: the generics enforce the same `CyclesUseCase`, not the same cost schedule, which is not part of the type at all. # The four `min` calls | site | before | after | | --- | --- | --- | | `settle_prepayment_for_unexecuted_response` | `p - base_fee.min(p)` | `p - base_fee` | | `refund_for_response_transmission` | `p - cost.min(p)` | `p - cost` | | `refund_unused_execution_cycles` | `refund.min(p)` | `refund.component_wise_min(p)` | | `CanisterManager::load_canister_snapshot` | `p - cost.min(p)` | `p - cost.component_wise_min(p)` | Subtraction saturates in both parts, so `x - y` already equals `x - x.component_wise_min(y)`: capping a cost before subtracting it from that same prepayment was redundant. Where a cap is wanted, `component_wise_min` replaces the ordering with the minimum of the real parts paired with the minimum of the nominal parts. `load_canister_snapshot` keeps its redundant cap so that it stays identical to the `refund_unused_execution_cycles` refund it recomputes. # Motivation Hardening rather than a fix. Under a single cost schedule the two parts of an amount agree on every ordering previously consulted, so the lexicographic tie-break was never reached: no cycles balance and no consumed cycles metric changes. Removing the derive keeps it out of reach as the accounting changes. Concretely, were the two amounts ever to carry different cost schedules, the removed ordering would produce: - `settle_prepayment_for_unexecuted_response`: for a prepayment made under the free cost schedule and settled under the normal one, the ordering picks the whole prepayment as the charge. The consumed cycles counter then reports the nominal prepayment for the entire instruction limit — for a callback that never ran — instead of the fixed per-message execution fee. - `refund_for_response_transmission`: same shape, same direction. Nothing is refunded and the whole nominal prepayment is reported as consumed. - `refund_unused_execution_cycles`: the cap is decided by the real parts, so it need not bound the nominal one at all. For a prepayment made under the normal cost schedule and refunded under the free one, it selects the refund and leaves that refund's nominal part uncapped: exceeding the prepayment there trips the debug assertion in `refund_cycles`, and saturates both metrics in a release build. The other way round it selects the whole prepayment, refunding all of it and reporting the response as having consumed nothing. None of the three moves a cycles balance: wherever the two orderings disagree, one of the amounts carries the free cost schedule and hence a zero real part, so only the nominal parts, i.e. the consumed cycles metrics, can differ. One comparison of this kind is left after this PR, in `adjust_prepayment_for_response_execution`, which weighs the prepayment for a response against the requirement derived when that response is executed. It is likewise sound only while both carry the same cost schedule. #11431 removes it. # Note #11431 builds on this and currently contains these hunks too. It will be rebased onto this PR, so review this one on its own. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent c7ea996 commit e5b09f3

5 files changed

Lines changed: 128 additions & 18 deletions

File tree

rs/cycles_account_manager/src/cycles_account_manager.rs

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -555,12 +555,13 @@ impl CyclesAccountManager {
555555
}
556556
let num_instructions_to_refund =
557557
std::cmp::min(num_instructions, num_instructions_initially_charged);
558+
// Never refund more than was prepaid, in either the real or the nominal part.
558559
let cycles_to_refund = self
559560
.scale_cost(
560561
self.convert_instructions_to_cycles(num_instructions_to_refund, execution_mode),
561562
subnet_cycles_config,
562563
)
563-
.min(prepaid_execution_cycles);
564+
.component_wise_min(prepaid_execution_cycles);
564565
system_state.refund_cycles(prepaid_execution_cycles, cycles_to_refund);
565566
}
566567

@@ -938,7 +939,8 @@ impl CyclesAccountManager {
938939
///
939940
/// Note that the prepayment is never topped up for such a response: the additional
940941
/// cycles would be refunded right away and, unlike this refund, the withdrawal
941-
/// could fail.
942+
/// could fail. The subtraction below saturates in both the real and the nominal
943+
/// part, so the canister is charged at most what it prepaid in each of them.
942944
pub fn settle_prepayment_for_unexecuted_response(
943945
&self,
944946
system_state: &mut SystemState,
@@ -952,12 +954,12 @@ impl CyclesAccountManager {
952954
subnet_cycles_config,
953955
execution_mode,
954956
);
955-
// The prepayment covers the fixed per-message execution fee, but clamp the
956-
// charge to it so that no more than the prepayment is ever charged.
957-
let charge = base_fee.min(prepayment_for_response_execution);
957+
// The prepayment covers the fixed per-message execution fee. The subtraction
958+
// saturates in both the real and the nominal part, so no more than the
959+
// prepayment is ever charged.
958960
system_state.refund_cycles(
959961
prepayment_for_response_execution,
960-
prepayment_for_response_execution - charge,
962+
prepayment_for_response_execution - base_fee,
961963
);
962964
}
963965

@@ -1000,8 +1002,9 @@ impl CyclesAccountManager {
10001002
self.config.xnet_byte_transmission_fee * transmitted_bytes,
10011003
subnet_cycles_config,
10021004
);
1003-
prepayment_for_response_transmission
1004-
- transmission_cost.min(prepayment_for_response_transmission)
1005+
// The subtraction saturates in both the real and the nominal part, so a
1006+
// transmission cost exceeding the prepayment leaves nothing to refund.
1007+
prepayment_for_response_transmission - transmission_cost
10051008
}
10061009

10071010
////////////////////////////////////////////////////////////////////////////

rs/execution_environment/src/canister_manager.rs

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2400,15 +2400,17 @@ impl CanisterManager {
24002400
&self.log,
24012401
);
24022402
// Record the net cycle charge (prepay minus refund) so it survives
2403-
// the canister state rollback if a subsequent step fails.
2403+
// the canister state rollback if a subsequent step fails. This recomputes
2404+
// what `refund_unused_execution_cycles` above refunded, cap included, so
2405+
// that the two agree by construction.
24042406
let cycles_to_refund = self
24052407
.cycles_account_manager
24062408
.variable_execution_cost(
24072409
instructions_to_refund,
24082410
subnet_cycles_config,
24092411
wasm_execution_mode,
24102412
)
2411-
.min(prepaid_execution_cycles);
2413+
.component_wise_min(prepaid_execution_cycles);
24122414
consumed_cycles.add(
24132415
prepaid_execution_cycles - cycles_to_refund,
24142416
instructions_for_execution,

rs/execution_environment/src/execution/response/tests.rs

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -457,7 +457,10 @@ fn cycles_correct_if_response_fails() {
457457
let execution_cost_before = test.canister_execution_cost(a_id);
458458
test.execute_message(a_id);
459459
let execution_cost_after = test.canister_execution_cost(a_id);
460-
assert_gt!(execution_cost_after, execution_cost_before);
460+
assert_gt!(
461+
execution_cost_after.nominal(),
462+
execution_cost_before.nominal()
463+
);
461464
assert_eq!(
462465
test.canister_state(a_id).system_state.balance(),
463466
initial_cycles
@@ -507,7 +510,10 @@ fn cycles_correct_if_cleanup_fails() {
507510
let execution_cost_before = test.canister_execution_cost(a_id);
508511
test.execute_message(a_id);
509512
let execution_cost_after = test.canister_execution_cost(a_id);
510-
assert_gt!(execution_cost_after, execution_cost_before);
513+
assert_gt!(
514+
execution_cost_after.nominal(),
515+
execution_cost_before.nominal()
516+
);
511517
assert_eq!(
512518
test.canister_state(a_id).system_state.balance(),
513519
initial_cycles
@@ -1188,7 +1194,10 @@ fn response_fail_scenario(test: &mut ExecutionTest) -> (CanisterId, MessageId) {
11881194
let execution_cost_before = test.canister_execution_cost(a_id);
11891195
test.execute_message(a_id);
11901196
let execution_cost_after = test.canister_execution_cost(a_id);
1191-
assert_gt!(execution_cost_after, execution_cost_before);
1197+
assert_gt!(
1198+
execution_cost_after.nominal(),
1199+
execution_cost_before.nominal()
1200+
);
11921201

11931202
let ingress_status = test.ingress_status(&ingress_id);
11941203
let result = check_ingress_status(ingress_status).unwrap_err();
@@ -1237,7 +1246,10 @@ fn cleanup_fail_scenario(test: &mut ExecutionTest) -> (CanisterId, MessageId) {
12371246
let execution_cost_before = test.canister_execution_cost(a_id);
12381247
test.execute_message(a_id);
12391248
let execution_cost_after = test.canister_execution_cost(a_id);
1240-
assert_gt!(execution_cost_after, execution_cost_before);
1249+
assert_gt!(
1250+
execution_cost_after.nominal(),
1251+
execution_cost_before.nominal()
1252+
);
12411253

12421254
let ingress_status = test.ingress_status(&ingress_id);
12431255
let result = check_ingress_status(ingress_status).unwrap_err();
@@ -2025,7 +2037,10 @@ fn reserve_instructions_for_cleanup_callback_scenario(
20252037
let execution_cost_before = test.canister_execution_cost(a_id);
20262038
test.execute_message(a_id);
20272039
let execution_cost_after = test.canister_execution_cost(a_id);
2028-
assert_gt!(execution_cost_after, execution_cost_before);
2040+
assert_gt!(
2041+
execution_cost_after.nominal(),
2042+
execution_cost_before.nominal()
2043+
);
20292044

20302045
// Assert that the response failed with exceeding instructions limit.
20312046
let ingress_status = test.ingress_status(&ingress_id);

rs/execution_environment/tests/hypervisor.rs

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7228,7 +7228,10 @@ fn cycles_correct_if_update_fails() {
72287228
let execution_cost_before = test.canister_execution_cost(b_id);
72297229
test.execute_message(b_id);
72307230
let execution_cost_after = test.canister_execution_cost(b_id);
7231-
assert_gt!(execution_cost_after, execution_cost_before);
7231+
assert_gt!(
7232+
execution_cost_after.nominal(),
7233+
execution_cost_before.nominal()
7234+
);
72327235
assert_eq!(
72337236
test.canister_state(b_id).system_state.balance(),
72347237
initial_cycles - test.canister_execution_cost(b_id).real()

rs/types/cycles/src/compound_cycles.rs

Lines changed: 89 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,10 @@ use std::ops::{Add, AddAssign, Div, Mul, Sub, SubAssign};
5555
///
5656
/// Extra type-safety is added via use of generics and phantom data to enforce
5757
/// that arithmetic operations can only be performed on amounts that were
58-
/// created for the same `CyclesUseCase` and `CanisterCyclesCostSchedule`.
58+
/// created for the same `CyclesUseCase`. The `CanisterCyclesCostSchedule` is not
59+
/// part of the type: `new` folds it into the real part and does not retain it, so
60+
/// nothing stops two amounts created under different cost schedules from being
61+
/// combined (see the note on ordering below).
5962
///
6063
/// E.g. the following code would not compile:
6164
///
@@ -74,7 +77,29 @@ use std::ops::{Add, AddAssign, Div, Mul, Sub, SubAssign};
7477
/// let total = cc_instructions + cc_memory;
7578
/// assert_eq!(total.real(), Cycles::new(30));
7679
/// ```
77-
#[derive(Copy, Clone, Eq, PartialEq, Ord, PartialOrd, Debug, Serialize, Deserialize)]
80+
///
81+
/// # No ordering
82+
///
83+
/// `CompoundCycles` deliberately implements neither `Ord` nor `PartialOrd`: its two
84+
/// parts are accounted for independently and there is no meaningful order on the
85+
/// pair. A derived impl would order lexicographically, i.e. by the real part first,
86+
/// and hence decide a comparison on the real parts alone whenever those differ, no
87+
/// matter how the nominal parts compare.
88+
///
89+
/// Two amounts carrying the same cost schedule are safe to compare that way: under
90+
/// the normal cost schedule the two parts of an amount coincide, and under the free
91+
/// cost schedule the real part of a use case made free is zero on both sides, so the
92+
/// comparison falls through to the nominal parts. Such an order is misleading
93+
/// precisely when the two amounts carry *different* cost schedules, e.g. because one
94+
/// was recorded when a call was performed and the other derived when its response is
95+
/// executed: the one made free has a zero real part and compares as the smaller
96+
/// amount however large its nominal part is.
97+
///
98+
/// Compare `real()` or `nominal()` explicitly instead, or use `component_wise_min`
99+
/// to bound both parts at once. Note also that subtraction saturates in both the
100+
/// real and the nominal part, so capping an amount before subtracting it is
101+
/// redundant: `x - y` already equals `x - x.component_wise_min(y)`.
102+
#[derive(Copy, Clone, Eq, PartialEq, Debug, Serialize, Deserialize)]
78103
pub struct CompoundCycles<T: CyclesUseCaseKind> {
79104
real: Cycles,
80105
nominal: NominalCycles,
@@ -130,6 +155,22 @@ impl<T: CyclesUseCaseKind> CompoundCycles<T> {
130155
self.real.is_zero() && self.nominal.is_zero()
131156
}
132157

158+
/// Returns the component-wise minimum of this amount and `other`, i.e. the
159+
/// minimum of their real parts paired with the minimum of their nominal parts.
160+
///
161+
/// The two components are minimized separately because there is no ordering on
162+
/// the pair (see the note on this type). They coincide under the normal cost
163+
/// schedule; under the free cost schedule the real part of a use case made free
164+
/// is zero, so minimizing by the real parts alone would leave the nominal part
165+
/// of the result unbounded.
166+
pub fn component_wise_min(self, other: Self) -> Self {
167+
Self {
168+
real: self.real.min(other.real),
169+
nominal: self.nominal.min(other.nominal),
170+
_cycles_use_case_marker: self._cycles_use_case_marker,
171+
}
172+
}
173+
133174
/// Returns this amount reduced by the part of `real()` that could not be
134175
/// charged, e.g. because the balance it was to be subtracted from did not
135176
/// cover it. Such an amount is never removed from any balance, so it must
@@ -243,3 +284,49 @@ impl<T: CyclesUseCaseKind> TryFrom<PbCompoundCycles> for CompoundCycles<T> {
243284
})
244285
}
245286
}
287+
288+
#[cfg(test)]
289+
mod tests {
290+
use super::*;
291+
use crate::cycles_use_case::Instructions;
292+
use crate::nominal_cycles::testing::NominalCyclesTesting;
293+
294+
/// An `Instructions` amount has coincident parts under the normal cost schedule,
295+
/// whereas its real part is zero under the free cost schedule. The two amounts
296+
/// below therefore order one way in their real parts and the other way in their
297+
/// nominal parts, which is exactly the case a lexicographic ordering of the pair
298+
/// would decide on the real parts alone.
299+
#[test]
300+
fn arithmetic_is_component_wise() {
301+
let x =
302+
CompoundCycles::<Instructions>::new(Cycles::new(5), CanisterCyclesCostSchedule::Normal);
303+
let y =
304+
CompoundCycles::<Instructions>::new(Cycles::new(10), CanisterCyclesCostSchedule::Free);
305+
assert_eq!(
306+
(x.real(), x.nominal()),
307+
(Cycles::new(5), NominalCycles::new(5))
308+
);
309+
assert_eq!(
310+
(y.real(), y.nominal()),
311+
(Cycles::zero(), NominalCycles::new(10))
312+
);
313+
314+
// Subtracting `y` from `x` without going below zero in either part.
315+
let difference = x - y;
316+
assert_eq!(
317+
(difference.real(), difference.nominal()),
318+
(Cycles::new(5), NominalCycles::zero())
319+
);
320+
321+
// The component-wise minimum takes each part from a different amount.
322+
let minimum = x.component_wise_min(y);
323+
assert_eq!(
324+
(minimum.real(), minimum.nominal()),
325+
(Cycles::zero(), NominalCycles::new(5))
326+
);
327+
assert_eq!(y.component_wise_min(x), minimum);
328+
329+
// Capping before subtracting is redundant.
330+
assert_eq!(x - x.component_wise_min(y), difference);
331+
}
332+
}

0 commit comments

Comments
 (0)