feat: add support of new type of subnet (cloud engine) in domain canister matcher#181
feat: add support of new type of subnet (cloud engine) in domain canister matcher#181shilingwang wants to merge 3 commits intomainfrom
Conversation
| let domains = match subnets_info.subnet_type(canister_id) { | ||
| Some(SubnetType::System) => &self.domains_system, | ||
| Some(SubnetType::CloudEngine) => &self.domains_engine, | ||
| _ => &self.domains_app, |
There was a problem hiding this comment.
It's a good practice to explicitly match (instead of _), so if somebody adds a new SubnetType, they get a compile-time error and won't forget to update this code.
You can use Some(SubnetType::...) | None to match two things at the same time.
| impl SubnetType { | ||
| fn from_bytes(bytes: &[u8]) -> Option<Self> { | ||
| match bytes { | ||
| b"application" => Some(Self::Application), |
There was a problem hiding this comment.
Where does the bytes representation come from?
There was a problem hiding this comment.
These byte strings come directly from the NNS state tree. Specifically, they are the raw leaf values read from the path /subnet/<subnet_id>/type in the certified state tree. The NNS encodes the subnet type as a plain UTF-8 string in the tree, so what arrives over the wire is bytes like b"application", b"system", etc.
| } | ||
| } | ||
|
|
||
| #[cfg(test)] |
There was a problem hiding this comment.
Note that this will only be available in uni tests of this crate and not in the tests of any other crates. Probably good enough, just wanted to point out in case you need this method in other tests.
| None => { | ||
| warn!( | ||
| "Unknown subnet type {:?} for subnet {subnet_id}", | ||
| std::str::from_utf8(type_bytes).unwrap_or("<invalid utf8>") |
There was a problem hiding this comment.
You can use from_utf8_lossy which replaces non-utf8 characters with ?, so the string is still somewhat readable.
|
|
||
| /// Reads `/canister_ranges/<subnet_id>` from the NNS state tree for each | ||
| /// subnet and decodes all CBOR-encoded range chunks into a sorted routing | ||
| /// table. |
There was a problem hiding this comment.
Consider mentioning the return value and the fact that its sorted
| mut canister_ranges: Vec<(Principal, Principal, Principal)>, | ||
| subnet_types: AHashMap<Principal, SubnetType>, | ||
| ) -> Self { | ||
| canister_ranges.sort_unstable_by_key(|(lo, _, _)| *lo); |
There was a problem hiding this comment.
optional: Since fetch_canister_ranges also has to sort, you could consider constructing SubnetsInfo with new and always sort in new to ensure the invariant is upheld. Then you don't need the sort in the rest of the code.
| // canister_ranges/<subnet_id>/<chunk_start_bytes> = <cbor blob> | ||
| let mut canister_ranges: Vec<(Principal, Principal, Principal)> = Vec::new(); | ||
|
|
||
| for &subnet_id in subnet_ids { |
There was a problem hiding this comment.
Don't you want to do it in parallel?
2 participants
#Node-1865
Summary
This PR replaces the hardcoded engine-subnet CLI flag with a dynamic,
periodically-refreshed view of the NNS routing table, and extends
DomainCanisterMatcherto route requests based on live subnet typeinformation fetched directly from the NNS state tree.
Motivation
Previously, whether a canister ran on an "engine subnet" was determined
by a static CLI flag. This meant operators had to know and manually
maintain the list of engine subnets. It also relied on a hardcoded
SYSTEM_SUBNETSrange table for system subnet detection. Both arefragile as the network evolves.
Changes
New:
src/routing/ic/subnets_info.rsIntroduces
SubnetsInfoFetcher, a background task (implementsRun)that periodically queries the NNS state tree in three round trips:
/subnet— discovers all subnet IDs./canister_ranges— fetches the full routing table (CBOR-encodedshards per subnet), decoded into a sorted
Vec<(lo, hi, subnet_id)>for O(log n) lookups.
/subnet/<id>/type(batched) — fetches the type for everydiscovered subnet.
The result is stored in
SubnetsInfobehind anArc<ArcSwap<...>>forlock-free, thread-safe access on the hot path.
SubnetsInfo::subnet_type(canister_id)performs a binary-search over the sorted range table.
SubnetTypecovers all four types currently defined in the IC interfacespec:
Application,System,VerifiedApplication, and the newlyadded
CloudEngine(per dfinity/ic#8892).Updated:
DomainCanisterMatcherSYSTEM_SUBNETStable andis_system_subnetfunction. System subnet detection now uses
SubnetType::Systemfromthe live snapshot.
check()takessubnets_info: &SubnetsInfoas an explicit parameter(injected by the middleware layer) instead of holding an
Arc<ArcSwap<SubnetsInfo>>internally — keeping the matcher as purestatic config.
matchonsubnet_type:System→ system domainsCloudEngine→ engine domainsUpdated:
CanisterMatcherStateOwns the
Arc<ArcSwap<SubnetsInfo>>, loads the current snapshot onceper request, and passes it into
check().Updated: CLI (
src/cli.rs)--domain-engine: list of domains to serve cloud-engine subnetcanisters from.
--subnets-info-poll-interval(default5m): how often to refreshthe NNS snapshot (replaces the engine-specific
--domain-engine-poll-interval).Updated:
core.rsSubnetsInfoFetcheris now always started as a background task (nolonger gated on
--domain-enginebeing set), because system subnetrouting also depends on the live snapshot.
Testing
domain_canister.rscovering all routing paths:system / engine / app canister on correct and incorrect domains, the
pre-isolation canister bypass, and empty-snapshot fallback behaviour.
SubnetsInfo::default()(empty snapshot → app-domain fallback), whichis correct for their setup.