Skip to content

Commit edf278d

Browse files
committed
docs: cleanup Bitswap
1 parent 18edccb commit edf278d

File tree

1 file changed

+114
-68
lines changed

1 file changed

+114
-68
lines changed

src/architecture/bitswap.md

Lines changed: 114 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -1,73 +1,102 @@
1-
# ![Status: WIP](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) Bitswap
1+
---
2+
title: Bitswap
3+
description: >
4+
Bitswap is a data exchange protocol for sending and receiving content addressed
5+
blocks of data. It attempts to acquire blocks from the network that have been
6+
requested by the client, and also send blocks to others who want them.
7+
date: 2022-08-26
8+
maturity: reliable
9+
editors:
10+
- name: Adin Schmahmann
11+
github: aschmahmann
12+
affiliation:
13+
name: Protocol Labs
14+
url: https://protocol.ai/
15+
- name: David Dias
16+
github: daviddias
17+
affiliation:
18+
name: Protocol Labs
19+
url: https://protocol.ai/
20+
- name: Jeromy Johnson
21+
github: whyrusleeping
22+
affiliation:
23+
name: Protocol Labs
24+
url: https://protocol.ai/
25+
- name: Juan Benet
26+
github: jbenet
27+
affiliation:
28+
name: Protocol Labs
29+
url: https://protocol.ai/
30+
tags: ['architecture']
31+
order: 1
32+
---
33+
34+
Bitswap is a data exchange protocol for sending and receiving content addressed blocks of data.
35+
Bitswap has two primary jobs:
236

3-
**Author(s)**:
4-
- Adin Schmahmann
5-
- David Dias
6-
- Jeromy Johnson
7-
- Juan Benet
8-
9-
**Maintainer(s)**:
10-
11-
* * *
12-
13-
**Abstract**
14-
15-
Bitswap is a data exchange protocol for sending and receiving content addressed blocks of data. Bitswap has two primary jobs:
1637
1. Attempt to acquire blocks from the network that have been requested by the client.
1738
2. Send blocks in its possession to other peers who want them.
1839

19-
## Organization of this document
20-
21-
- [Introduction](#introduction)
22-
- [Bitswap Protocol Versions](#bitswap-protocol-versions)
23-
- [Bitswap 1.0.0](#bitswap-100)
24-
- [Bitswap 1.1.0](#bitswap-110)
25-
- [Bitswap 1.2.0](#bitswap-120)
26-
- [Implementations](#implementations)
27-
2840
## Introduction
2941

30-
Bitswap is a message-based protocol, as opposed to request-response. All messages contain wantlists, and/or blocks.
31-
Upon receiving a wantlist, a Bitswap server should eventually process and respond to the requester with either information about the block or the block itself.
32-
Upon receiving blocks, the client should send a `Cancel` notification to peers that have asked for the data, signifying that the client no longer wants the block.
42+
Bitswap is a message-based protocol, as opposed to request-response. All messages
43+
contain wantlists, and/or blocks. Upon receiving a wantlist, a Bitswap server SHOULD
44+
eventually process and respond to the requester with either information about the
45+
block or the block itself. Upon receiving blocks, the client SHOULD send a `Cancel`
46+
notification to peers that have asked for the data, signifying that the client no
47+
longer wants the block.
3348

34-
Bitswap aims to be a simple protocol, so that implementations can balance aspects such as throughput, latency, fairness, memory usage, etc. for their specific requirements.
49+
Bitswap aims to be a simple protocol, so that implementations can balance aspects
50+
such as throughput, latency, fairness, memory usage, etc. for their specific
51+
requirements.
3552

3653
## Bitswap Protocol Versions
3754

38-
There are multiple Bitswap versions and more may evolve over time. We give brief overviews as to the changes behind each protocol version.
55+
There are multiple Bitswap versions and more may evolve over time. We give brief
56+
overviews as to the changes behind each protocol version:
3957

4058
- `/ipfs/bitswap/1.0.0` - Initial version
41-
- `/ipfs/bitswap/1.1.0` - Support CIDv1
42-
- `/ipfs/bitswap/1.2.0` - Support Wantlist Have's and Have/DontHave responses
59+
- `/ipfs/bitswap/1.1.0` - Support [CIDv1](https://docs.ipfs.io/concepts/glossary/#cid-v1)
60+
- `/ipfs/bitswap/1.2.0` - Support Wantlist `Have`'s and `Have`/`DontHave` responses
4361

4462
## Block Sizes
4563

46-
Bitswap implementations must support sending and receiving individual blocks of sizes less than or equal to 2MiB. Handling blocks larger than 2MiB is not recommended so as to keep compatibility with implementations which only support up to 2MiB.
64+
Bitswap implementations MUST support sending and receiving individual blocks of
65+
sizes less than or equal to 2MiB. Handling blocks larger than 2MiB is not recommended
66+
so as to keep compatibility with implementations which only support up to 2MiB.
4767

4868
## Bitswap 1.0.0
4969

5070
### Bitswap 1.0.0: Interaction Pattern
5171

52-
Given that a client C wants to fetch data from some server S:
72+
Given that a client *C* wants to fetch data from some server *S*:
5373

54-
1. C sends a message to S for the blocks it wants, via a stream `s_want`
55-
1. C may either send a complete wantlist, or an update to an outstanding wantlist
56-
2. C may reuse this stream to send new wants
57-
2. S sends back blocks on a stream `s_receive`. S may reuse this stream to send back subsequent responses.
58-
1. S should respect the relative priority of wantlist requests from C, with wants that have higher `priority` values being responded to first.
59-
3. When C no longer needs a block it previously asked for, it should send a `Cancel` message for that block to all peers from which it has not received a response about that block
74+
1. *C* sends a message to *S* for the blocks it wants, via a stream `s_want`
75+
1. *C* MAY either send a complete wantlist, or an update to an outstanding wantlist
76+
2. *C* MAY reuse this stream to send new wants
77+
2. *S* sends back blocks on a stream `s_receive`. *S* MAY reuse this stream to send
78+
back subsequent responses.
79+
1. *S* SHOULD respect the relative priority of wantlist requests from *C*, with
80+
wants that have higher `priority` values being responded to first.
81+
3. When *C* no longer needs a block it previously asked for, it SHOULD send a `Cancel`
82+
message for that block to all peers from which it has not received a response
83+
about that block
6084

6185
### Bitswap 1.0.0: Message
6286

63-
A single Bitswap message may contain any of the following content:
87+
A single Bitswap message MAY contain any of the following content:
6488

65-
1. The sender’s wantlist. This wantlist may either be the sender’s complete wantlist or just the changes to the sender’s wantlist that the receiver needs to know.
66-
2. Data blocks. These are meant to be blocks that the receiver has requested (i.e., blocks that are on the receiver’s wantlist as far as the sender is aware at the time of sending).
89+
1. The sender’s wantlist. This wantlist MAY either be the sender’s complete wantlist
90+
or just the changes to the sender’s wantlist that the receiver needs to know.
91+
2. Data blocks. These are meant to be blocks that the receiver has requested (i.e.,
92+
blocks that are on the receiver’s wantlist as far as the sender is aware at the
93+
time of sending).
6794

6895
#### Bitswap 1.0.0: Wire Format
6996

70-
The wire format for Bitswap is simply a stream of Bitswap messages. The following protobuf describes the form of these messages. Note: all protobufs are described using proto3 syntax.
97+
The wire format for Bitswap is simply a stream of Bitswap messages. The following
98+
protobuf describes the form of these messages. Note: all protobufs are described
99+
using [proto3](https://protobuf.dev/programming-guides/proto3/) syntax.
71100

72101
```protobuf
73102
message Message {
@@ -84,25 +113,25 @@ message Message {
84113
85114
Wantlist wantlist = 1;
86115
repeated bytes blocks = 2;
116+
}
87117
```
88118

89119
### Bitswap 1.0.0: Protocol Format
90120

91121
All protocol messages sent over a stream are prefixed with the message length in
92-
bytes, encoded as an unsigned variable length integer as defined by the
93-
[multiformats unsigned-varint spec](https://github.com/multiformats/unsigned-varint).
122+
bytes, encoded as an unsigned variable length integer as defined by the multiformats
123+
[unsigned-varint] spec.
94124

95-
All protocol messages must be less than or equal to 4MiB in size
125+
All protocol messages MUST be less than or equal to 4MiB in size.
96126

97127
## Bitswap 1.1.0
98128

99-
Bitswap 1.1.0 introduces a 'payload' field to the protobuf message and deprecates the
100-
existing 'blocks' field. The 'payload' field is an array of pairs of cid
101-
prefixes and block data. The cid prefixes are used to ensure the correct
102-
codecs and hash functions are used to handle the block on the receiving
103-
end.
129+
Bitswap 1.1.0 introduces a `payload` field to the protobuf message and deprecates the
130+
existing 'blocks' field. The 'payload' field is an array of pairs of CID prefixes
131+
and block data. The CID prefixes are used to ensure the correct codecs and hash
132+
functions are used to handle the block on the receiving end.
104133

105-
It is otherwise identical to 1.0.0
134+
It is otherwise identical to 1.0.0.
106135

107136
### Bitswap 1.1.0: Wire Format
108137

@@ -131,24 +160,39 @@ message Message {
131160
## Bitswap 1.2.0
132161

133162
Bitswap 1.2.0 extends the Bitswap 1.1.0 protocol with the three changes:
134-
1. Being able to ask if a peer has the data, not just to send the data
135-
2. A peer can respond that it does not have some data rather than just not responding
136-
3. Nodes can indicate on messages how much data they have queued to send to the peer they are sending the message to
137163

138-
### Bitswap 1.2.0: Interaction Pattern
164+
1. Being able to ask if a peer has the data, not just to send the data.
165+
2. A peer can respond that it does not have some data rather than just not responding.
166+
3. Nodes can indicate on messages how much data they have queued to send to the peer
167+
they are sending the message to.
139168

140-
Given that a client C wants to fetch data from some server S:
169+
### Bitswap 1.2.0: Interaction Pattern
141170

142-
1. C opens a stream `s_want` to S and sends a message for the blocks it wants
143-
1. C may either send a complete wantlist, or an update to an outstanding wantlist
144-
2. C may reuse this stream to send new wants
145-
3. For each of the items in the wantlist C may ask if S has the block (i.e. a Have request) or for S to send the block (i.e. a block request). C may also ask S to send back a DontHave message in the event it doesn't have the block
146-
2. S responds back on a stream `s_receive`. S may reuse this stream to send back subsequent responses
147-
1. If C sends S a Have request for data S has (and is willing to give to C) it should respond with a Have, although it may instead respond with the block itself (e.g. if the block is very small)
148-
2. If C sends S a Have request for data S does not have (or has but is not willing to give to C) and C has requested for DontHave responses then S should respond with DontHave
149-
3. S may choose to include the number of bytes that are pending to be sent to C in the response message
150-
4. S should respect the relative priority of wantlist requests from C, with wants that have higher `priority` values being responded to first.
151-
3. When C no longer needs a block it previously asked for it should send a Cancel message for that request to any peers that have not already responded about that particular block. It should particularly send Cancel messages for Block requests (as opposed to Have requests) that have not yet been answered.
171+
Given that a client *C* wants to fetch data from some server *S*:
172+
173+
1. *C* opens a stream `s_want` to *S* and sends a message for the blocks it wants:
174+
1. *C* MAY either send a complete wantlist, or an update to an outstanding wantlist.
175+
2. *C* MAY reuse this stream to send new wants.
176+
3. For each of the items in the wantlist *C* MAY ask if *S* has the block
177+
(i.e. a `Have` request) or for *S* to send the block (i.e. a `Block` request).
178+
*C* MAY also ask *S* to send back a `DontHave` message in the event it doesn't
179+
have the block.
180+
2. *S* responds back on a stream `s_receive`. *S* MAY reuse this stream to send
181+
back subsequent responses:
182+
1. If *C* sends *S* a `Have` request for data *S* has (and is willing to give
183+
to *C*) it SHOULD respond with a `Have`, although it MAY instead respond
184+
with the block itself (e.g. if the block is very small).
185+
2. If *C* sends *S* a `Have` request for data *S* does not have (or has but
186+
is not willing to give to *C*) and *C* has requested for `DontHave` responses
187+
then *S* SHOULD respond with `DontHave`.
188+
3. *S* MAY choose to include the number of bytes that are pending to be sent
189+
to *C* in the response message.
190+
4. *S* SHOULD respect the relative priority of wantlist requests from *C*,
191+
with wants that have higher `priority` values being responded to first.
192+
3. When *C* no longer needs a block it previously asked for it SHOULD send a
193+
`Cancel` message for that request to any peers that have not already responded
194+
about that particular block. It SHOULD particularly send `Cancel` messages for
195+
`Block` requests (as opposed to `Have` requests) that have not yet been answered.
152196

153197
### Bitswap 1.2.0: Wire Format
154198

@@ -194,5 +238,7 @@ message Message {
194238

195239
## Implementations
196240

197-
- <https://github.com/ipfs/go-bitswap>
198-
- <https://github.com/ipfs/js-ipfs-bitswap>
241+
- [Boxo Bitswap](https://github.com/ipfs/boxo/tree/main/bitswap)
242+
- [js-ipfs-bitswap](https://github.com/ipfs/js-ipfs-bitswap)
243+
244+
[unsigned-varint]: https://github.com/multiformats/unsigned-varint

0 commit comments

Comments
 (0)