Skip to content

Commit c879baf

Browse files
lidellidel
committed
ipip-518: http(s) urls in routing v1
allows http(s) urls alongside multiaddrs in addrs field
1 parent bb946b4 commit c879baf

File tree

3 files changed

+311
-10
lines changed

3 files changed

+311
-10
lines changed

‎src/http-gateways/path-gateway.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ editors:
3535
name: Protocol Labs
3636
url: https://protocol.ai/
3737
xref:
38-
- url
38+
- rfc3986
3939
- trustless-gateway
4040
- subdomain-gateway
4141
- dnslink-gateway
@@ -511,7 +511,7 @@ When deserialized responses are enabled,
511511
and no explicit response format is provided with the request, and the
512512
requested data itself has no built-in content type metadata, implementations
513513
SHOULD perform content type sniffing based on file name
514-
(from :ref[url] path, or optional [`filename`](#filename-request-query-parameter) parameter)
514+
(from URI path, or optional [`filename`](#filename-request-query-parameter) parameter)
515515
and magic bytes to improve the utility of produced responses.
516516

517517
For example:

‎src/ipips/ipip-0518.md‎

Lines changed: 290 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,290 @@
1+
---
2+
title: "IPIP-0518: HTTP(S) URLs in Routing V1 API"
3+
date: 2025-10-13
4+
ipip: proposal
5+
editors:
6+
- name: Marcin Rataj
7+
github: lidel
8+
url: https://lidel.org/
9+
affiliation:
10+
name: Shipyard
11+
url: https://ipshipyard.com
12+
relatedIssues:
13+
- https://github.com/ipfs/specs/issues/192
14+
- https://github.com/ipfs/specs/issues/496
15+
- https://github.com/multiformats/multiaddr/issues/63
16+
- https://github.com/multiformats/multiaddr/issues/87
17+
- https://github.com/ipshipyard/roadmaps/issues/15
18+
- https://github.com/ipfs/specs/pull/518
19+
order: 518
20+
tags: ['ipips']
21+
xref:
22+
- rfc3986
23+
---
24+
25+
## Summary
26+
27+
Allow HTTP(S) URLs alongside multiaddrs in the `Addrs` field of the Peer schema in the Delegated Routing V1 HTTP API to enable easier integration with HTTP-based infrastructure.
28+
29+
## Motivation
30+
31+
The current Delegated Routing V1 HTTP API requires all peer addresses to be encoded as [multiaddrs](https://github.com/multiformats/multiaddr). While multiaddrs provide a flexible and protocol-agnostic way to represent network addresses, many IPFS services are primarily accessible via HTTP(S) endpoints, including:
32+
33+
- IPFS Gateways (both path and subdomain gateways)
34+
- Delegated routing endpoints themselves
35+
- HTTP-based content providers and pinning services
36+
37+
Converting HTTP(S) URLs to multiaddrs requires additional complexity:
38+
- HTTP URLs must be encoded as `/dns4/example.com/tcp/80/http` or `/dns4/example.com/tcp/443/https`
39+
- This conversion is not intuitive for developers familiar with web standards
40+
- It does not capture HTTP semantics where the same website can be exposed on both TCP (HTTP/1.1, HTTP/2) and UDP (HTTP/3)
41+
- A single `https://example.com` URL automatically supports multiple transport protocols, but multiaddr representation requires separate entries for each transport
42+
- Parsing multiaddrs back to URLs requires additional libraries and logic
43+
44+
By allowing native HTTP(S) URLs in the `Addrs` field, we can:
45+
- Simplify integration with existing web infrastructure
46+
- Reduce conversion overhead for HTTP-based services
47+
- Improve developer experience by using familiar URL formats
48+
- Improve interoperability with the wider HTTP and URI ecosystem
49+
- Enable future-proofing for non-HTTP URI schemes in ecosystem experimentation without requiring permission from gatekeepers
50+
- Maintain backward compatibility with existing multiaddr-based implementations
51+
52+
## Detailed design
53+
54+
### Changes to the Peer Schema
55+
56+
The `Addrs` field in the [Peer Schema](https://specs.ipfs.tech/routing/http-routing-v1/#peer-schema) will accept both multiaddr strings and HTTP(S) URL strings:
57+
58+
```json
59+
{
60+
"Schema": "peer",
61+
"ID": "bafz...",
62+
"Addrs": [
63+
"/ip4/192.168.1.1/tcp/4001",
64+
"/dns4/libp2p-peer.example.com/tcp/4001/ws",
65+
"https://trustless-gateway.example.org",
66+
"https://custom-port.example.net:8443"
67+
],
68+
"Protocols": ["transport-bitswap", ...]
69+
}
70+
```
71+
72+
### Parsing Logic
73+
74+
Implementations MUST use the following logic to distinguish between multiaddrs and URLs:
75+
76+
1. If a string in `Addrs` starts with `/` (forward slash), parse it as a multiaddr
77+
2. Otherwise, attempt to parse it as a URI according to :cite[rfc3986]
78+
3. If neither parsing succeeds, or if the address type is not supported by the implementation, the address MUST be ignored (skipped)
79+
4. Processing MUST continue with the remaining addresses in the array
80+
5. Implementations SHOULD log warnings for addresses they cannot parse or do not support
81+
82+
This approach ensures forward compatibility: new address types can be introduced without breaking existing clients, as unsupported addresses are simply skipped.
83+
84+
### Supported URL Schemes
85+
86+
Initially, only the following URL schemes SHOULD be supported:
87+
- `http://` - HTTP endpoints
88+
- `https://` - HTTPS endpoints
89+
90+
Future specifications MAY add support for additional schemes.
91+
92+
### URL Requirements
93+
94+
URLs in the `Addrs` field:
95+
- MUST be absolute URLs (not relative)
96+
- MUST include the scheme (`http://` or `https://`)
97+
- SHOULD NOT include paths, query parameters, and fragments, but clients MUST account for them being present as part of defensive programming and either act on them, ignore them, or skip such addresses
98+
- SHOULD point to endpoints that support IPFS protocols listed in the `Protocols` field
99+
100+
### Examples
101+
102+
#### HTTPS-only Content Provider
103+
104+
```json
105+
{
106+
"Schema": "peer",
107+
"ID": "12D3KooWExample...",
108+
"Addrs": [
109+
"https://trustless-gateway.example.com"
110+
],
111+
"Protocols": ["transport-ipfs-gateway-http"]
112+
}
113+
```
114+
115+
#### Hybrid Peer with Multiple Transports
116+
117+
```json
118+
{
119+
"Schema": "peer",
120+
"ID": "12D3KooWExample...",
121+
"Addrs": [
122+
"/ip4/192.168.1.1/tcp/4001",
123+
"/ip4/192.168.1.1/udp/4001/quic-v1",
124+
"https://my-node.example.org:8080"
125+
],
126+
"Protocols": ["transport-bitswap", "transport-ipfs-gateway-http"]
127+
}
128+
```
129+
130+
## Design rationale
131+
132+
### Why not create a new field?
133+
134+
Adding URLs to the existing `Addrs` field rather than creating a new field (e.g., `URLs`) has several advantages:
135+
- Maintains backward compatibility - existing clients continue to work
136+
- Avoids duplication when the same endpoint can be expressed as both multiaddr and URL
137+
- Simplifies the schema without adding complexity
138+
- Follows the principle that addresses are addresses, regardless of encoding
139+
140+
### Clear disambiguation
141+
142+
The parsing rule (strings starting with `/` are multiaddrs, others are URIs) provides clear, unambiguous disambiguation:
143+
- Multiaddrs ALWAYS start with `/` by specification
144+
- Valid URLs NEVER start with `/` (they start with a scheme like `http://`)
145+
- This makes parsing deterministic and fast
146+
147+
### Incremental adoption
148+
149+
This change allows for incremental adoption:
150+
- Clients that don't understand URLs can simply skip them
151+
- Servers can start including URLs immediately for URL-aware clients
152+
- No flag day or coordinated upgrade required
153+
154+
## User benefit
155+
156+
This change benefits multiple user groups:
157+
158+
### For developers
159+
160+
- Simplified integration with existing HTTP infrastructure
161+
- No need for multiaddr encoding/decoding libraries for HTTP endpoints
162+
- Clearer, more readable configurations and debugging
163+
- Barrier of adoption is removed: developers can implement HTTP-based routing and retrieval without having to re-implement libp2p concepts like Multiaddr, making it orders of magnitude easier to create light IPFS clients
164+
165+
### For service providers
166+
167+
- Easier to advertise HTTP-based services
168+
- Can provide URLs that include paths and query parameters if needed
169+
- Reduced complexity in route announcements
170+
171+
### For end users
172+
173+
- Potentially faster connection establishment to HTTP services
174+
- Better compatibility with web-based IPFS implementations
175+
- Lower barrier for creating new clients gives end users more choice and less vendor lock-in
176+
- Provides viable escape path in case any of the open source projects gets captured by forces that do not put end user's good first
177+
178+
## Compatibility
179+
180+
This IPIP is fully backward and forward compatible:
181+
182+
### For existing clients
183+
184+
- Clients that only understand multiaddrs MUST skip URL entries they don't recognize (this is already implemented and proven to work when new protocols like `/quic`, `/quic-v1`, `/webtransport`, and `/webrtc-direct` were rolled out)
185+
- Clients MUST continue processing remaining addresses even when encountering unsupported entries
186+
- No changes required to existing parsing logic for multiaddr strings
187+
- The `Addrs` field remains an array of strings
188+
189+
### Forward compatibility
190+
191+
- The requirement to skip unsupported addresses ensures that new address types can be added in the future
192+
- Clients MUST NOT fail when encountering unknown address formats
193+
- This allows the ecosystem to evolve without breaking existing implementations or without the need for permission or central coordination
194+
195+
### For existing servers
196+
197+
- Servers can continue sending only multiaddrs
198+
- No changes required if URLs are not used
199+
200+
### Migration path
201+
202+
1. Servers can start including both multiaddrs and URLs for the same endpoints
203+
2. Clients can be updated to parse URLs at their own pace
204+
3. Eventually, servers may choose to only send URLs for HTTP(S) endpoints
205+
206+
## Security
207+
208+
### URL validation
209+
210+
Implementations SHOULD validate URLs to prevent security issues:
211+
- Verify the URL scheme is allowed (`http://` or `https://`)
212+
- Consider rate limiting for URL-based connections if non-success (!=200) responses are received
213+
- Validate URL length limits (DNS names are limited to 253 characters; practical URL length is typically 2048-8192 characters depending on implementation)
214+
215+
### HTTPS preference
216+
217+
Implementations SHOULD ignore `http://` URLs and only act on `https://` URLs for security and performance (HTTP/2 multiplexing) reasons.
218+
219+
The `http://` scheme SHOULD be allowed only for testing and private LAN deployments, and only when an explicit opt-in flag is set by the end user.
220+
221+
### DNS considerations
222+
223+
URLs rely on DNS resolution, which has different security properties than IP-based multiaddrs. The same rules that apply to `/dns`, `/dns4`, and `/dns6` multiaddrs apply here:
224+
- DNS responses can be spoofed if DNSSEC is not used
225+
- Clients SHOULD use secure DNS transports where available
226+
- Certificate validation MUST be performed for HTTPS URLs
227+
228+
## Alternatives
229+
230+
### Separate URL field
231+
232+
Adding a separate `URLs` field was considered but rejected because:
233+
- It would complicate the schema
234+
- It could lead to confusion about which field to use
235+
- It wouldn't be backward compatible
236+
237+
### URL-to-multiaddr conversion requirement
238+
239+
Requiring all HTTP endpoints to be encoded as multiaddrs was the status quo but has proven cumbersome in practice. Multiple implementations on NPM and Golang alone behaved in slightly different fashion around how the schema, default port, optional path, fragment, and HTTP basic-auth were handled. This led to hard-to-debug errors due to multiaddr-URL conversion being ultimately lossy and 1:1 round-trip not being possible (see [multiaddr#63](https://github.com/multiformats/multiaddr/issues/63)).
240+
241+
### Custom multiaddr protocols with keyword arguments
242+
243+
Adding keyword arguments to multiaddr protocols was proposed in [multiaddr#87](https://github.com/multiformats/multiaddr/issues/87) to allow expressing `https://` URLs as multiaddrs without losing any information on conversion. This approach was not adopted because it would add even more complexity that multiaddr implementers would have to deal with.
244+
245+
This solution was not feasible - adding native URI support is better as it removes walls and obstacles, rather than making existing ones taller.
246+
247+
## Test fixtures
248+
249+
Implementations can test compatibility using these example responses:
250+
251+
### Mixed addresses response
252+
253+
```json
254+
{
255+
"Providers": [
256+
{
257+
"Schema": "peer",
258+
"ID": "12D3KooWTest1...",
259+
"Addrs": [
260+
"/ip4/127.0.0.1/tcp/4001",
261+
"http://localhost:8080",
262+
"/dns4/example.com/tcp/443/https",
263+
"https://example.net"
264+
],
265+
"Protocols": ["transport-bitswap", "transport-ipfs-gateway-http"]
266+
}
267+
]
268+
}
269+
```
270+
271+
### URL-only response
272+
273+
```json
274+
{
275+
"Providers": [
276+
{
277+
"Schema": "peer",
278+
"ID": "12D3KooWTest2...",
279+
"Addrs": [
280+
"https://trustless-gateway.example.org"
281+
],
282+
"Protocols": ["transport-ipfs-gateway-http"]
283+
}
284+
]
285+
}
286+
```
287+
288+
## Copyright
289+
290+
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

‎src/routing/http-routing-v1.md‎

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,9 @@ editors:
3636
url: https://ipshipyard.com
3737
xref:
3838
- ipip-0337
39+
- ipip-0518
3940
- ipns-record
41+
- rfc3986
4042
order: 0
4143
tags: ['routing']
4244
---
@@ -52,7 +54,11 @@ As such, human-readable encodings of types are preferred. This specification may
5254
## Common Data Types
5355

5456
- CIDs are always string-encoded using a [multibase]-encoded [CIDv1].
55-
- Multiaddrs are string-encoded according to the [human-readable multiaddr specification][multiaddr].
57+
- Addresses in the `Addrs` field can be:
58+
- Multiaddrs: string-encoded according to the [human-readable multiaddr specification][multiaddr], always starting with `/`
59+
- HTTP(S) URLs: absolute URLs with `http://` or `https://` schemes, parsed as URIs according to :cite[rfc3986]
60+
- Parsing logic: if a string starts with `/`, parse as multiaddr; otherwise, parse as URI
61+
- Unsupported addresses: implementations MUST skip addresses they cannot parse or do not support, and MUST continue processing remaining addresses (see [IPIP-0518](https://specs.ipfs.tech/ipips/ipip-0518/))
5662
- Peer IDs are string-encoded according [PeerID string representation specification][peer-id-representation]: either a Multihash in Base58btc, or a CIDv1 with libp2p-key (`0x72`) codec in Base36 or Base32.
5763
- Multibase bytes are string-encoded according to [the Multibase spec][multibase], and SHOULD use base64.
5864
- Timestamps are Unix millisecond epoch timestamps.
@@ -77,16 +83,18 @@ This API uses a standard version prefix in the path, such as `/v1/...`. If a bac
7783

7884
Optional `?filter-addrs` to apply Network Address Filtering from [IPIP-484](https://specs.ipfs.tech/ipips/ipip-0484/).
7985

80-
- `?filter-addrs=<comma-separated-list>` optional parameter that indicates which network transports to return by filtering the multiaddrs in the `Addrs` field of the [Peer schema](#peer-schema).
86+
- `?filter-addrs=<comma-separated-list>` optional parameter that indicates which network transports to return by filtering the addresses in the `Addrs` field of the [Peer schema](#peer-schema).
8187
- The value of the `filter-addrs` parameter is a comma-separated (`,` or `%2C`) list of network transport protocol _name strings_ as defined in the [multiaddr protocol registry](https://github.com/multiformats/multiaddr/blob/master/protocols.csv), e.g. `?filter-addrs=tls,webrtc-direct,webtransport`.
82-
- `unknown` can be be passed to include providers whose multiaddrs are unknown, e.g. `?filter-addrs=unknown`. This allows for not removing providers whose multiaddrs are unknown at the time of filtering (e.g. keeping DHT results that require additional peer lookup).
83-
- Multiaddrs are filtered by checking if the protocol name appears in any of the multiaddrs (logical OR).
84-
- Negative filtering is done by prefixing the protocol name with `!`, e.g. to skip IPv6 and QUIC addrs: `?filter-addrs=!ip6,!quic-v1`. Note that negative filtering is done by checking if the protocol name does not appear in any of the multiaddrs (logical AND).
88+
- `unknown` can be be passed to include providers whose addresses are unknown, e.g. `?filter-addrs=unknown`. This allows for not removing providers whose addresses are unknown at the time of filtering (e.g. keeping DHT results that require additional peer lookup).
89+
- Addresses are filtered by checking if the protocol name appears in any of the multiaddrs, or if the URI scheme matches for HTTP(S) URLs (logical OR in both cases).
90+
- Example: `http` can be be passed to include providers whose addresses are HTTP-compatible. This will include `http://` `https://` URIs, and `/http` `/https` and `/tls/http` Multiaddrs.
91+
- For the purpose of filtering, implementations SHOULD include `/tls/http` Multiaddrs when `https` is passed as a filter to ensure composed multiaddrs are included in results.
92+
- Negative filtering is done by prefixing the protocol name with `!`, e.g. to skip IPv6 and QUIC addrs: `?filter-addrs=!ip6,!quic-v1`. Note that negative filtering is done by checking if the protocol name does not appear in any of the addresses (logical AND).
8593
- If no parameter is passed, the default behavior is to return the original list of addresses unchanged.
8694
- If only negative filters are provided, addresses not passing any of the negative filters are included.
8795
- If positive filters are provided, only addresses passing at least one positive filter (and no negative filters) are included.
8896
- If both positive and negative filters are provided, the address must pass all negative filters and at least one positive filter to be included.
89-
- If there are no multiaddrs that match the passed transports, the provider is omitted from the response.
97+
- If there are no addresses that match the passed transports, the provider is omitted from the response.
9098
- Filtering is case-insensitive.
9199

92100
##### `filter-protocols` (providers request query parameter)
@@ -315,14 +323,17 @@ The `peer` schema represents an arbitrary peer.
315323
{
316324
"Schema": "peer",
317325
"ID": "bafz...",
318-
"Addrs": ["/ip4/..."],
326+
"Addrs": ["/ip4/...", "https://trustless-gateway.example.com"],
319327
"Protocols": ["transport-bitswap", ...]
320328
...
321329
}
322330
```
323331

324332
- `ID`: the [Peer ID][peer-id] as Multihash in Base58btc or CIDv1 with libp2p-key codec.
325-
- `Addrs`: an optional list of known [multiaddrs][multiaddr] for this peer.
333+
- `Addrs`: an optional list of known addresses for this peer, which can include both:
334+
- [Multiaddrs][multiaddr]: strings starting with `/`, e.g., `/ip4/192.168.1.1/tcp/4001`
335+
- HTTP(S) URLs: absolute URLs with `http://` or `https://` schemes, e.g., `https://trustless-gateway.example.com`
336+
- Implementations MUST skip addresses they cannot parse or do not support and continue with remaining addresses
326337
- If missing or empty, it means the router server is missing that information, and the client should use `ID` to lookup updated peer information.
327338
- `Protocols`: an optional list of protocols known to be supported by this peer.
328339
- If missing or empty, it means the router server is missing that information, and the client should use `ID` and `Addrs` to lookup connect to the peer and use the [libp2p identify protocol](https://github.com/libp2p/specs/tree/master/identify) to learn about supported ones.

0 commit comments

Comments
 (0)