ipip: Streaming Routing V1 HTTP API

Author: hacdias

src/ipips/ipip-0410.md

@@ -0,0 +1,68 @@
+---
+title: "IPIP-0410: Streaming Routing V1 HTTP API"
+date: 2023-05-12
+ipip: proposal
+editors:
+  - name: Henrique Dias
+    github: hacdias
+    url: https://hacdias.com/
+relatedIssues:
+  - https://github.com/ipfs/specs/issues/344
+  - https://github.com/ipfs/boxo/pull/18
+  - https://github.com/ipfs/kubo/pull/9868
+  - https://github.com/ipfs/kubo/pull/9874
+order: 410
+tags: ['ipips']
+---
+
+Introduce backwards-compatible streaming support to the Routing V1 HTTP API.
+
+Introduce backwards-compatible streaming support to the Routing V1 HTTP API by
+using the HTTP Header `Accept` (:cite[rfc9110]) with [Newline Delimited JSO (NDJSON)](http://ndjson.org/).
+
+## Motivation
+
+The main motivation for this change is that a server may take some time before fetching
+all the peers they know providing a certain CID. That will add some latency when sending
+a non-streamed response, as they need to wait for the whole discovery period.
+
+With streaming support, the server is able to respond with a *read provider record*
+as they themselves discover the peers, not needing to wait for the whole discovery
+to return the provider records.
+
+## Detailed design
+
+In summary, streaming is supported by using the `Accept` HTTP header (:cite[rfc9110])
+along with the `application/x-ndjson` content type. For more details regarding the design,
+check :cite[http-routing-v1].
+
+## Design rationale
+
+The idea is to not break compatibility with existing clients and servers. Therefore,
+we can use the `Accept` HTTP header for content negotiation. If the client supports
+streaming, they can advertise so by including `application/x-ndjson` in the headers.
+If the server supports streaming, the response will have `Content-Type` HTTP header
+of `application/x-ndjson` with a streaming response. Otherwise, it will fall back
+to a pure JSON, non-streamed, response.
+
+### User benefit
+
+Users (clients) will benefit from this change as the servers will now be able
+to respond more promptly to provider record requests. Instead of waiting for the whole
+list to be constructed, servers can now return each provider record one by one,
+in a streaming fashion.
+
+### Compatibility
+
+The introduced changes are backwards-compatible. The introduced header is completely
+optional, and a server that does not support streaming is able to respond with a non-streaming
+response to the client. Equally, non-streaming responses are the default. Therefore, a
+client that does not support streaming will not receive a streamed response.
+
+### Security
+
+Security considerations are equivalent as the ones in :cite[ipip-0337].
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

src/routing/http-routing-v1.md

@@ -8,7 +8,10 @@ date: 2023-03-22
 maturity: reliable
 editors:
   - name: Gus Eggert
-    github: guseggert 
+    github: guseggert
+  - name: Henrique Dias
+    url: https://hacdias.com/
+    github: hacdias
 order: 0
 tags: ['routing']
 ---
@@ -67,7 +70,7 @@ Specifications for some transfer protocols are provided in the "Transfer Protoco
 
 ### `GET /routing/v1/providers/{CID}`
 
-#### Response codes
+#### Response Codes
 
 - `200` (OK): the response body contains 0 or more records
 - `404` (Not Found): must be returned if no matching records are found
@@ -97,7 +100,23 @@ This API does not support pagination, but optional pagination can be added in a
 
 ## Streaming
 
-This API does not currently support streaming, however it can be added in the future through a backwards-compatible update by using a content type other than `application/json`.
+This API supports streaming requests made by sending an `Accept` HTTP Header containing
+`application/x-ndjson`. The response will be formatted as [Newline Delimited JSON (ndjson)](http://ndjson.org/),
+with one *read provider record* per line:
+
+
+```json
+{"Protocol": "<protocol_name>", "Schema": "<schema>", ...}
+{"Protocol": "<protocol_name>", "Schema": "<schema>", ...}
+{"Protocol": "<protocol_name>", "Schema": "<schema>", ...}
+...
+```
+
+Streaming is backwards-compatible with clients that do not support streaming. Please note the following:
+
+- Requests without an `Accept` header MUST default to regular, non-streaming, responses.
+- The server MAY respond with non-streaming response even if the client requests streaming.
+- The server MUST NOT respond with streaming response if the client did not request so.
 
 ## Error Codes