-
Notifications
You must be signed in to change notification settings - Fork 6
fix: IPNI validation confirms provider in indexer response #231
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: IPNI validation confirms provider in indexer response #231
Conversation
previously, validateIpniAdvertisement did not validate that the ipni advertisement existed for the given provider. Now, we validate that the response includes the expected provider, and allow consumers to pass extra providers (to support future multiple provider uploads)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR enhances IPNI (InterPlanetary Network Indexer) advertisement validation to verify that specific storage providers have announced content, addressing a gap where the validation previously only checked if any provider advertised the CID. The implementation now validates that expected providers' multiaddrs are present in IPNI responses, with fallback to generic validation when provider information cannot be derived.
Key changes:
- Added provider-specific validation with multiaddr matching against expected providers
- Automatic provider detection from upload context with support for explicit provider specification
- Enhanced error messages showing which provider multiaddrs are missing from advertisements
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/core/utils/validate-ipni-advertisement.ts | Implements provider-specific IPNI validation with multiaddr conversion and matching logic |
| src/test/unit/validate-ipni-advertisement.test.ts | Adds comprehensive test coverage for provider validation scenarios and edge cases |
| src/core/upload/index.ts | Integrates provider validation by passing current provider info to IPNI validation |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
BigLep
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I need to look again with a fresh head. This generally feels like a lot of code has gotten added and I'm wondering if there's ways to simplify (but not able to think clearly currently)
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
I simplified code a bit and inlined some things.. still +500 lines |
SgtPooki
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
self review.. much better now..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
BigLep
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most comments are around terminology. We're validating the advertisement flow (not the advertisement itself). We do this by inspecting the ProviderResults/provider records.
I also was confused by why we have the ability for passing extra multiaddrs to validate.
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
| // Note: If expectedProviders is explicitly [], we respect that (no provider expectations) | ||
| if (expectedProviders != null) { | ||
| validationOptions.expectedProviders = expectedProviders | ||
| } else if (synapseService.providerInfo != null) { | ||
| validationOptions.expectedProviders = [synapseService.providerInfo] | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@BigLep most folks will call through this method, which will set expectedProviders to the correct thing (and to the multi-providers for multi-provider uploads once synapse-sdk is updated)
|
FYI: CI was failing when I migrated to set operations because we only had tsconfig set to ES2022: NodeJS Active LTS is 24 as of Oct 28 2025, and nodejs 22+ and all major browsers support Set.intersection and other set operations, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set/intersection We are already testing on CI for lts/* (24) and current (25), see https://nodejs.org/en/about/previous-releases#looking-for-the-latest-release-of-a-version-branch I updated tsconfig to add |
BigLep
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor comments. Key thing is to get build passing. Approved otherwise.
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
previously, validateIpniAdvertisement did not validate that the ipni advertisement existed for
the given provider. Now, we validate that the response includes the expected provider,
and allow consumers to pass extra providers (to support future multiple provider uploads)
some code inspired by #140