feat(delegations): reactively fetch delegations on new public key and/or canister ranges

[pierugo-dfinity]

During canister migrations and soon subnet splitting, the public key of a subnet and its canister ranges are modified. Because the delegation manager fetches new delegations every 5 minutes independently from the state, there can be up to 5 minutes of additional downtime during those operations just to let each replica fetch a new delegation that matches their state.

This PR proposes to add a new asynchronous path to the delegation manager to fetch new delegations. To enable this, the PR also refactors the delegation reader.

Delegation manager changes

The current "fetch every 5 minutes" logic stays with one important difference: if the fetched delegation does not match the latest certified state, hold it back, i.e. ignore it and try again 5 minutes later. This is to avoid the case where we would have fetched a delegation too early.

Asynchronously, the delegation manager locally checks every 10 seconds whether its latest delegation matches its latest certified state. In almost all cases, they will, and it won't do anything. Though if they do mismatch, then it reactively fetches a new delegation. This allows to reduce the maximum downtime from 5 minutes to 10 seconds.

Delegation reader refactor

The NNSDelegationBuilder/Reader are moved to their own crate ic_nns_delegation_reader, which contains the relevant code for comparing the delegations with the replicated state. By putting the comparison/validation logic directly in NNSDelegationBuilder, this will allow the HTTP handler to check whether its delegation matches the latest certified state and return a 503 error if not instead of serving an inconsistent delegation.
For now, "delegation matching the state" means the same subnet public key and the same canister ranges for one's own subnet, but we can add more variants to CanisterRangesCheck to check whether a specific canister ID is present in both the delegation and state (instead of checking for all ranges).

The motivation for moving it to a different crate is that the query handler will need to depend on it to perform the comparison and it would be better not to also depend on all the dependencies of ic_nns_delegation_manager. In particular, ic_nns_delegation_reader cannot depend on ReplicatedState as it would otherwise create a circular dependency.

[github-actions]

This pull request changes code owned by the Governance team. Therefore, make sure that
you have considered the following (for Governance-owned code):

1. Update unreleased_changelog.md (if there are behavior changes, even if they are
   non-breaking).

2. Are there BREAKING changes?

3. Is a data migration needed?

4. Security review?

How to Satisfy This Automatic Review

1. Go to the bottom of the pull request page.

2. Look for where it says this bot is requesting changes.

3. Click the three dots to the right.

4. Select "Dismiss review".

5. In the text entry box, respond to each of the numbered items in the previous
   section, declare one of the following:

• Done.

• $REASON_WHY_NO_NEED. E.g. for unreleased_changelog.md, "No
  canister behavior changes.", or for item 2, "Existing APIs
  behave as before.".

Brief Guide to "Externally Visible" Changes

"Externally visible behavior change" is very often due to some NEW canister API.

Changes to EXISTING APIs are more likely to be "breaking".

If these changes are breaking, make sure that clients know how to migrate, how to
maintain their continuity of operations.

If your changes are behind a feature flag, then, do NOT add entrie(s) to
unreleased_changelog.md in this PR! But rather, add entrie(s) later, in the PR
that enables these changes in production.

Reference(s)

For a more comprehensive checklist, see here.

GOVERNANCE_CHECKLIST_REMINDER_DEDUP

[zeropath-ai]

✅ No security or compliance issues detected. Reviewed everything up to 566488e.

Security Overview

• 🔎 Scanned files: 16 changed file(s)
• 🔗 Scan Link: https://zeropath.com/app/repositories/f5e9d2bb-29a4-4b64-902a-391d65fd8b2d?scanId=c6691329-a7dd-4e39-8f9f-2b74c755e1f6&codeScanTypes=PrScan&tab=issues

Detected Code Changes

Change Type

Relevant files

Enhancement

► rs/http_endpoints/nns_delegation_manager/src/lib.rs     Expose NNSDelegationReader publicly and re-export types ► rs/http_endpoints/nns_delegation_manager/src/metrics.rs     Rename update_duration to fetch_duration and add new metrics (fetch_errors, state_comparison_errors, held_back_delegations, reactive_fetches) and adjust related labels ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Add state_reader dependency and state-aware delegation validation logic (is_delegation_valid_with_respect_to_state, proactive_fetch, reactive_fetch) and integrate with new metrics ► rs/http_endpoints/nns_delegation_manager/Cargo.toml     Add ic-interfaces-state-manager, ic-nns-delegation-reader, ic-registry-routing-table, ic-replicated-state dependencies; replace criterion with rstest in dev-dependencies ► rs/http_endpoints/nns_delegation_manager/BUILD.bazel     Adjust dependencies to include state_manager, replicated_state, and routing_table; remove serde from rust_ic_bench; switch bench setup and test utilities references ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Introduce proactive and reactive update intervals and state-informed fetch logic ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Update fetch flow to use new is_delegation_valid_with_respect_to_state and enhanced run loop with proactive/reactive intervals ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Update load_root_delegation invocation to align with new delegation validation flow and metric naming ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Add helper function observe_delegation_sizes to record delegation certificate sizes to metrics ► rs/http_endpoints/nns_delegation_manager/src/tests (in-file changes across multiple test blocks)

[zeropath-ai]

✅ No security or compliance issues detected. Reviewed everything up to b1534a2.

Security Overview

• 🔎 Scanned files: 10 changed file(s)
• 🔗 Scan Link: https://zeropath.com/app/repositories/f975f1a5-0a47-4a3b-b7f0-a55609e90390?scanId=228cc8a1-408e-4dd5-930b-d7375686c37e&codeScanTypes=PrScan&tab=issues

Detected Code Changes

Change Type	Relevant files
Enhancement	► rs/canonical_state/src/delegation.rs     Add delegation validation against replicated state ► rs/canonical_state/Cargo.toml     Add rstest as a dev dependency ► rs/canonical_state/src/lib.rs     Expose delegation module publicly
Enhancement	► rs/http_endpoints/nns_delegation_manager/BUILD.bazel     Add canonical_state to deps ► rs/http_endpoints/nns_delegation_manager/Cargo.toml     Add ic-canonical-state as a dependency     Add ic-interfaces-state-manager and ic-replicated-state as deps     Add rstest as dev dependency
Enhancement	► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Use StateReader and CertificateDelegationFormat in imports     Integrate state-aware delegation validation hooks ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Introduce proactive/reactive delegation update intervals and related logic ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Add methods is_delegation_valid_with_respect_to_state, proactive_fetch, reactive_fetch ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Extend DelegationManager with state-aware validation logic ► rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs     Update run loop to handle proactive/reactive updates

1. No behavioral changes
2. No
3. No
4. No

[daniel-wong-dfinity-org-twin]

Woops. I was not looking at the whole PR somehow. Clicked on a wrong link.

[Copilot]

Pull request overview

This PR reduces delegation-related downtime during subnet public-key / canister-range changes by adding a reactive delegation-refresh path driven by the latest certified state, and by extracting/refactoring the delegation read/validation logic into a dedicated ic_nns_delegation_reader crate for broader reuse (e.g., by HTTP handlers).

Changes:

• Add a new ic_nns_delegation_reader crate that builds filtered delegations and validates delegation certificates against certified-state subnet topology (public key + canister ranges).
• Update the NNS delegation manager to (a) proactively refresh on a fixed interval while holding back delegations that are “too new” for the replica’s certified state, and (b) reactively refetch quickly when the cached delegation mismatches the certified state.
• Wire the state reader into replica stack construction and extend metrics to distinguish fetch vs. state-comparison/reactive behavior.

Reviewed changes

Copilot reviewed 14 out of 16 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
rs/replica/src/setup_ic_stack.rs	Passes the state manager/reader into the delegation manager so it can compare delegations against certified state.
rs/http_endpoints/nns_delegation_reader/test_utils/src/lib.rs	Adds reader-focused test utilities for constructing fake certificate delegations.
rs/http_endpoints/nns_delegation_reader/test_utils/Cargo.toml	Renames the test-utils crate to align with the new reader crate.
rs/http_endpoints/nns_delegation_reader/test_utils/BUILD.bazel	Updates Bazel visibility and crate name for the renamed test-utils crate.
rs/http_endpoints/nns_delegation_reader/src/validation.rs	Introduces delegation-vs-state validation logic (public key + canister ranges) and unit tests.
rs/http_endpoints/nns_delegation_reader/src/reader.rs	Refactors delegation builder/reader API; adds verified build path and state-consistency checks.
rs/http_endpoints/nns_delegation_reader/src/lib.rs	Exposes the new reader/validation API surface.
rs/http_endpoints/nns_delegation_reader/Cargo.toml	Defines the new ic-nns-delegation-reader crate and its dependencies/bench config.
rs/http_endpoints/nns_delegation_reader/BUILD.bazel	Adds Bazel targets for the new crate, tests, and benchmarks.
rs/http_endpoints/nns_delegation_reader/benches/nns_delegation_reader.rs	Moves the benchmark to use the new reader crate/test utils.
rs/http_endpoints/nns_delegation_manager/src/nns_delegation_manager.rs	Adds state-driven reactive refresh + proactive hold-back logic; updates tests accordingly.
rs/http_endpoints/nns_delegation_manager/src/metrics.rs	Splits metrics into fetch/state-comparison/held-back/reactive counters.
rs/http_endpoints/nns_delegation_manager/src/lib.rs	Re-exports reader types from ic_nns_delegation_reader and removes the old internal reader module.
rs/http_endpoints/nns_delegation_manager/Cargo.toml	Adds state-manager + replicated-state + reader deps; adjusts dev-deps for updated tests.
rs/http_endpoints/nns_delegation_manager/BUILD.bazel	Updates Bazel deps to include the new reader crate and state-manager mocks; removes old bench target.
Cargo.lock	Records the new crates/dependency graph changes (reader crate + renamed test utils, etc.).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.