-
Notifications
You must be signed in to change notification settings - Fork 236
IPIP-0524: Remove IPLD translation from IPFS HTTP Gateway #524
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
achingbrain
wants to merge
1
commit into
main
Choose a base branch
from
feat/ipip-remove-ipld-translation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| --- | ||
| title: "IPIP-0524: Remove IPLD Data Model translations from Gateways" | ||
| date: 2025-11-27 | ||
| ipip: proposal | ||
| editors: | ||
| - name: Alex Potsides | ||
| github: achingbrain | ||
| url: https://achingbrain.net | ||
| affiliation: | ||
| name: Shipyard | ||
| url: https://ipshipyard.com | ||
| relatedIssues: | ||
| - https://github.com/ipfs/gateway-conformance/issues/200 | ||
| order: 524 | ||
| tags: ['ipips'] | ||
| --- | ||
|
|
||
| ## Summary | ||
|
|
||
| Make IPFS HTTP Gateway responses easier to reason about by not requiring IPLD | ||
| Data Model translations | ||
|
|
||
| ## Motivation | ||
|
|
||
| When sending an [Accept](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Accept) | ||
| header or [format](https://specs.ipfs.tech/http-gateways/path-gateway/#format-request-query-parameter) | ||
| query parameter to specify the response format of a request, the IPFS HTTP | ||
| Gateway specs [allow translation](https://specs.ipfs.tech/http-gateways/path-gateway/#accept-request-header) | ||
| of the requested content into the [IPLD Data Model](https://ipld.io/docs/data-model/). | ||
|
|
||
| This adds significant complexity to HTTP Gateway implementations, since they | ||
| need to be able to translate between arbitrary data types and handle all the | ||
| various failure states. | ||
|
|
||
| The conversions are also lossy due to differences in supported data types across | ||
| different formats so lack general-purpose utility and are ultimately something | ||
| that could be done on an interested client if required. | ||
|
|
||
| ## Detailed design | ||
|
|
||
| An IPFS HTTP Gateway should make an attempt to verify that requested data can be | ||
| represented in the desired response format (e.g. it should attempt to parse the | ||
| block as JSON string or CBOR data when the relevant formats are requested, | ||
| though in the case of CBOR it does not need to support all [tags](https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml) | ||
| encountered in the block). | ||
|
|
||
| If the block was loaded successfully but cannot be represented in the requested | ||
| format, the server should return a [406](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/406). | ||
|
|
||
| This would be the outcome if, for example a DAG-JSON block was requested with | ||
| the `application/cbor` format. | ||
|
|
||
| Where a human-readable rendering of the data is desired, the `text/html` format | ||
| can be requested. This would allow browsing DAG-PB data, for example. | ||
|
|
||
| A [400](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/400) | ||
| may be returned if the request was invalid (for example an unsupported format | ||
| was requested). | ||
|
|
||
| A [500](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/500) | ||
| may be returned in other circumstances. | ||
|
|
||
| ## Design rationale | ||
|
|
||
| Simplifying the HTTP Gateway spec to remove these format translations and the | ||
| additional logic required makes it more straightforward to create new | ||
| implementations, and makes the returned data more transparent and so easier to | ||
| understand since the data is not modified to fit the output format. | ||
|
|
||
| Clients that wish to translate between different data formats may request raw | ||
| blocks and do the translation themselves. | ||
|
|
||
| ### User benefit | ||
|
|
||
| HTTP Gateways will be simpler to use having fewer options and they will be | ||
| encouraged to adapt their clients for their individual use-cases where more | ||
| advanced functionality is required. | ||
|
|
||
| ### Compatibility | ||
|
|
||
| This is a breaking change. Data will no longer be translated between different | ||
| formats by the server and that logic will shift to interested clients. | ||
|
|
||
| ## Test fixtures | ||
|
|
||
| Implementers can run the [gateway-conformance](https://github.com/ipfs/gateway-conformance/) | ||
| test suite. | ||
|
|
||
| ### Copyright | ||
|
|
||
| Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@achingbrain how strong you feel about doing actual JSON or CBOR parsing here, when plain version is requested?
The main purpose here, in my mind, is not data validation, but to maximize interop ensure correct
Content-Typein response, as there are tools which will make request but wont parse response if its notapplication/jsonetc.Could we simplify spec here noting that if requested CID codec is matching
[dag-]jsonor[dag-]cbor, we dont parse payload, but return the same response asrawbut with specific vanilla content type.It makes implementation simpler (no need to deal with CBOR/JSON parsing edge cases), reduces cost of running gateway (no JSON/CBOR parsing), but we still have the interop benefit of returning vanilla JSON/CBOR when necessary by a third-party system.