Skip to content

Commit 4631d78

Browse files
Gozalamikeal
Gozala
and
mikeal
authored
feat!: Configuration free implementation (#36)
BREAKING CHANGE! * pkg: drop period * invalidate cache * fix: vendor varint for pure ESM * fix: pass encode offset * fix: remove unecessary ImplicitSha256Digest * chore: add links, tree and get APIs to Block * fix: typo in template literal * fix: vendor base-x for pure ESM * feat: bundle base58btc and base32 encoders * chore: add code type info * fix: switch to field base check over instanceof * core: export ByteView type * fix: browser polyfill for deepStrictEquals doesn't supprot Uint8Arrays * fix: coverage for cid * fix: hashes coverage * chore: disable coverage for vendor * chore: improve test coverage * chore: remove dead code * chore: remove block.js * fix: drop old npm ignore * fix: moving excludes to package.json * fix: browser same handles uint8arrays terribly * chore: update readme * doc: fix typo * fix: drop basics exports BREAKING CHANGE! no more basics export * fix: tests off of basics interface Co-authored-by: Mikeal Rogers <[email protected]>
1 parent b0eca3c commit 4631d78

37 files changed

+2265
-1199
lines changed

‎.npmignore

Lines changed: 0 additions & 2 deletions
This file was deleted.

‎README.md

Lines changed: 113 additions & 120 deletions
Original file line numberDiff line numberDiff line change
@@ -1,161 +1,154 @@
11
# multiformats
22

3-
This library is for building an interface for working with various
4-
inter-related multiformat technologies (multicodec, multihash, multibase,
5-
and CID).
3+
This library defines common interfaces and low level building blocks for varios inter-related multiformat technologies (multicodec, multihash, multibase,
4+
and CID). They can be used to implement custom custom base
5+
encoders / decoders / codecs, codec encoders /decoders and multihash hashers that comply to the interface that layers above assume.
66

7-
The interface contains all you need for encoding and decoding the basic
8-
structures with no codec information, codec encoder/decoders, base encodings
9-
or hashing functions. You can then add codec info, codec encoders/decoders,
10-
base encodings, and hashing functions to the interface.
7+
Library provides implementations for most basics and many others can be found in linked repositories.
118

12-
This allows you to pass around an interface containing only the code you need
13-
which can greatly reduce dependencies and bundle size.
9+
## Interfaces
1410

1511
```js
16-
import { create } from 'multiformats'
17-
import sha2 from 'multiformats/hashes/sha2'
18-
import dagcbor from '@ipld/dag-cbor'
19-
const { multihash, multicodec, CID } = create()
20-
multihash.add(sha2)
21-
multicodec.add(dagcbor)
22-
23-
const buffer = multicodec.encode({ hello, 'world' }, 'dag-cbor')
24-
const hash = await multihash.hash(buffer, 'sha2-256')
25-
// raw codec is the only codec that is there by default
26-
const cid = new CID(1, 'raw', hash)
12+
import CID from 'multiformats/cid'
13+
import json from 'multiformats/codecs/json'
14+
import { sha256 } from 'multiformats/hashes/sha2'
15+
16+
const bytes = json.encode({ hello: 'world' })
17+
18+
const hash = await sha256.digest(bytes)
19+
const cid = CID.create(1, json.code, hash)
20+
//> CID(bagaaierasords4njcts6vs7qvdjfcvgnume4hqohf65zsfguprqphs3icwea)
2721
```
2822

29-
However, if you're doing this much you should probably use multiformats
30-
with the `Block` API.
23+
### Multibase Encoders / Decoders / Codecs
24+
25+
CIDs can be serialized to string representation using multibase encoders that
26+
implement [`MultibaseEncoder`](https://github.com/multiformats/js-multiformats/blob/master/src/bases/interface.ts) interface. Library
27+
provides quite a few implementations that can be imported:
3128

3229
```js
33-
// Import basics package with dep-free codecs, hashes, and base encodings
34-
import multiformats from 'multiformats/basics'
35-
import dagcbor from '@ipld/dag-cbor'
36-
import { create } from '@ipld/block' // Yet to be released Block interface
37-
multiformats.multicodec.add(dagcbor)
38-
const Block = create(multiformats)
39-
const block = Block.encoder({ hello: world }, 'dag-cbor')
40-
const cid = await block.cid()
30+
import { base64 } from "multiformats/bases/base64"
31+
cid.toString(base64.encoder)
32+
//> 'mAYAEEiCTojlxqRTl6svwqNJRVM2jCcPBxy+7mRTUfGDzy2gViA'
4133
```
4234

43-
# Plugins
35+
Parsing CID string serialized CIDs requires multibase decoder that implements
36+
[`MultibaseDecoder`](https://github.com/multiformats/js-multiformats/blob/master/src/bases/interface.ts) interface. Library provides a
37+
decoder for every encoder it provides:
4438

45-
By default, no base encodings, hash functions, or codec implementations are included with `multiformats`.
46-
However, you can import the following bundles to get a `multiformats` interface with them already configured.
39+
```js
40+
CID.parse('mAYAEEiCTojlxqRTl6svwqNJRVM2jCcPBxy+7mRTUfGDzy2gViA', base64.decoder)
41+
//> CID(bagaaierasords4njcts6vs7qvdjfcvgnume4hqohf65zsfguprqphs3icwea)
42+
```
4743

48-
| bundle | bases | hashes | codecs |
49-
|---|---|---|---|
50-
| `multiformats/basics` | `base32`, `base64` | `sha2-256`, `sha2-512` | `json`, `raw` |
44+
Dual of multibase encoder & decoder is defined as multibase codec and it exposes
45+
them as `encoder` and `decoder` properties. For added convenience codecs also
46+
implement `MultibaseEncoder` and `MultibaseDecoder` interfaces so they could be
47+
used as either or both:
5148

52-
## Base Encodings (multibase)
5349

54-
| bases | import | repo |
55-
--- | --- | --- |
56-
`base16` | `multiformats/bases/base16` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/bases) |
57-
`base32`, `base32pad`, `base32hex`, `base32hexpad`, `base32z` | `multiformats/bases/base32` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/bases) |
58-
`base64`, `base64pad`, `base64url`, `base64urlpad` | `multiformats/bases/base64` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/bases) |
59-
`base58btc`, `base58flick4` | `multiformats/bases/base58` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/bases) |
50+
```js
51+
cid.toString(base64)
52+
CID.parse(cid.toString(base64), base64)
53+
```
6054

61-
## Hash Functions (multihash)
55+
**Note:** CID implementation comes bundled with `base32` and `base58btc`
56+
multibase codecs so that CIDs can be base serialized to (version specific)
57+
default base encoding and parsed without having to supply base encoders/decoders:
6258

63-
| hashes | import | repo |
64-
| --- | --- | --- |
65-
| `sha2-256`, `sha2-512` | `multiformats/hashes/sha2` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/hashes) |
66-
| `sha3-224`, `sha3-256`, `sha3-384`,`sha3-512`, `shake-128`, `shake-256`, `keccak-224`, `keccak-256`, `keccak-384`, `keccak-512` | `@multiformats/sha3` | [multiformats/js-sha3](https://github.com/multiformats/js-sha3) |
67-
| `murmur3-128`, `murmur3-32` | `@multiformats/murmur3` | [multiformats/js-murmur3](https://github.com/multiformats/js-murmur3) |
59+
```js
60+
const v1 = CID.parse('bagaaierasords4njcts6vs7qvdjfcvgnume4hqohf65zsfguprqphs3icwea')
61+
v1.toString()
62+
//> 'bagaaierasords4njcts6vs7qvdjfcvgnume4hqohf65zsfguprqphs3icwea'
63+
64+
const v0 = CID.parse('QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n')
65+
v0.toString()
66+
//> 'QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n'
67+
v0.toV1().toString()
68+
//> 'bafybeihdwdcefgh4dqkjv67uzcmw7ojee6xedzdetojuzjevtenxquvyku'
69+
```
6870

69-
## Codec Implementations (multicodec)
71+
### Multicodec Encoders / Decoders / Codecs
7072

71-
| codec | import | repo |
72-
| --- | --- | --- |
73-
| `raw` | `multiformats/codecs/raw` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/codecs) |
74-
| `json` | `multiformats/codecs/json` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/codecs) |
75-
| `dag-cbor` | `@ipld/dag-cbor` | [ipld/js-dag-cbor](https://github.com/ipld/js-dag-cbor) |
76-
| `dag-json` | `@ipld/dag-json` | [ipld/js-dag-json](https://github.com/ipld/js-dag-json) |
73+
Library defines [`BlockEncoder`, `BlockDecoder` and `BlockCodec` interfaces](https://github.com/multiformats/js-multiformats/blob/master/src/codecs/interface.ts)
74+
and utility function to take care of the boilerplate when implementing them:
7775

78-
# API
76+
```js
77+
import { codec } from 'multiformats/codecs/codec'
78+
79+
const json = codec({
80+
name: 'json',
81+
// As per multiformats table
82+
// https://github.com/multiformats/multicodec/blob/master/table.csv#L113
83+
code: 0x0200,
84+
encode: json => new TextEncoder().encode(JSON.stringify(json)),
85+
decode: bytes => JSON.parse(new TextDecoder().decode(bytes))
86+
})
87+
```
7988

80-
# multiformats([table])
89+
Just like with multibase, here codecs are duals of `encoder` and `decoder` parts,
90+
but they also implement both interfaces for convenience:
8191

82-
Returns a new multiformats interface.
92+
```js
93+
const hello = json.encoder.encode({ hello: 'world' })
94+
json.decode(b1)
95+
//> { hello: 'world' }
96+
```
8397

84-
Can optionally pass in a table of multiformat entries.
98+
### Multihash Hashers
8599

86-
# multihash
100+
This library defines [`MultihashHasher` and `MultihashDigest` interfaces](https://github.com/multiformats/js-multiformats/blob/master/src/hashes/interface.ts)
101+
and convinient function for implementing them:
87102

88-
## multihash.encode
103+
```js
104+
import * as hasher from 'multiformats/hashes/hasher')
89105

90-
## multihash.decode
106+
const sha256 = hasher.from({
107+
// As per multiformats table
108+
// https://github.com/multiformats/multicodec/blob/master/table.csv#L9
109+
name: 'sha2-256',
110+
code: 0x12,
91111

92-
## multihash.validate
112+
encode: (input) => new Uint8Array(crypto.createHash('sha256').update(input).digest())
113+
})
93114

94-
## multihash.add
115+
const hash = await sha256.digest(json.encode({ hello: 'world' }))
116+
CID.create(1, json.code, hash)
95117

96-
## multihash.hash
118+
//> CID(bagaaierasords4njcts6vs7qvdjfcvgnume4hqohf65zsfguprqphs3icwea)
119+
```
97120

98-
# multicodec
99121

100-
## multicodec.encode
101122

102-
## multicodec.decode
123+
# Implementations
103124

104-
## multicodec.add
125+
By default, no base encodings (other than base32 & base58btc), hash functions,
126+
or codec implementations are included exposed by `multiformats`, you need to
127+
import the ones you need yourself.
105128

106-
# multibase
129+
## Multibase codecs
107130

108-
## multibase.encode
131+
| bases | import | repo |
132+
--- | --- | --- |
133+
`base16` | `multiformats/bases/base16` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/bases) |
134+
`base32`, `base32pad`, `base32hex`, `base32hexpad`, `base32z` | `multiformats/bases/base32` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/bases) |
135+
`base64`, `base64pad`, `base64url`, `base64urlpad` | `multiformats/bases/base64` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/bases) |
136+
`base58btc`, `base58flick4` | `multiformats/bases/base58` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/bases) |
109137

110-
## multibase.decode
138+
## Multihash hashers
111139

112-
## multibase.add
140+
| hashes | import | repo |
141+
| --- | --- | --- |
142+
| `sha2-256`, `sha2-512` | `multiformats/hashes/sha2` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/hashes) |
143+
| `sha3-224`, `sha3-256`, `sha3-384`,`sha3-512`, `shake-128`, `shake-256`, `keccak-224`, `keccak-256`, `keccak-384`, `keccak-512` | `@multiformats/sha3` | [multiformats/js-sha3](https://github.com/multiformats/js-sha3) |
144+
| `murmur3-128`, `murmur3-32` | `@multiformats/murmur3` | [multiformats/js-murmur3](https://github.com/multiformats/js-murmur3) |
113145

114-
## multiformats/bases
146+
## Codec Implementations (multicodec)
115147

116-
```js
117-
import { create } from 'multiformats'
118-
import base16 from 'multiformats/bases/base16'
119-
import base32 from 'multiformats/bases/base32'
120-
import base58 from 'multiformats/bases/base58'
121-
import base64 from 'multiformats/bases/base64'
122-
const multiformats = create()
123-
multiformats.add([base16, base32, base58, base64])
124-
```
148+
| codec | import | repo |
149+
| --- | --- | --- |
150+
| `raw` | `multiformats/codecs/raw` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/codecs) |
151+
| `json` | `multiformats/codecs/json` | [multiformats/js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/codecs) |
152+
| `dag-cbor` | `@ipld/dag-cbor` | [ipld/js-dag-cbor](https://github.com/ipld/js-dag-cbor) |
153+
| `dag-json` | `@ipld/dag-json` | [ipld/js-dag-json](https://github.com/ipld/js-dag-json) |
125154

126-
## multiformats/hashes
127-
128-
## multiformats/codecs
129-
130-
# CID
131-
132-
Changes from `cids`:
133-
134-
* All base encodings are cached indefinitely.
135-
* CID's can be created without any multiformat data.
136-
* The new API is entirely based on parsing the varints
137-
so it doesn't need the table metadata in order to associate
138-
string names.
139-
140-
There are also numerous deprecations. These deprecations all stem from the
141-
fact that we no longer know the full set of available multicodec information.
142-
It's actually quite possible to provide a CID interface without this, you can
143-
still do everything you used to do, you just need to use ints instead of strings
144-
and do some of the fancier V0 coercions outside this library.
145-
146-
Deprecation List:
147-
* the multibase encoding is no longer cached during instantiation.
148-
* this being indeterministic was causing some nasty problems downstream
149-
since `toString()` needs to be used as a cache key and it's not possible
150-
to encode V0 into anything but base58btc. this means that you can't have
151-
deterministic hash keys without also requiring base58btc support, so we
152-
removed this feature.
153-
* no more .toBaseEncodedString(), just toString()
154-
* no more .multibaseName
155-
* no more .prefix()
156-
* no more .codec
157-
* new property ".code" is the multiformat integer.
158-
* this is going to be a painful transition but we have to get off of using
159-
the string if we ever want to drop the full table. while the DX for this is
160-
nice it forces into bloating the bundle and makes using new codecs very
161-
painful.

‎package.json

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,23 @@
66
"type": "module",
77
"scripts": {
88
"build": "npm_config_yes=true npx ipjs@latest build --tests",
9+
"build:vendor": "npm run build:vendor:varint && npm run build:vendor:base-x",
10+
"build:vendor:varint": "npx brrp -x varint > vendor/varint.js",
11+
"build:vendor:base-x": "npx brrp -x @multiformats/base-x > vendor/base-x.js",
912
"publish": "npm_config_yes=true npx ipjs@latest publish",
1013
"lint": "standard",
1114
"test:cjs": "npm run build && mocha dist/cjs/node-test/test-*.js && npm run test:cjs:browser",
1215
"test:node": "hundreds mocha test/test-*.js",
1316
"test:cjs:browser": "polendina --cleanup dist/cjs/browser-test/test-*.js",
1417
"test": "npm run lint && npm run test:node && npm run test:cjs",
1518
"test:node-v12": "mocha test/test-*.js && npm run test:cjs",
16-
"coverage": "c8 --reporter=html mocha test/test-*.js && npm_config_yes=true npx st -d coverage -p 8080"
19+
"coverage": "c8 --exclude vendor --reporter=html mocha test/test-*.js && npm_config_yes=true npx st -d coverage -p 8080"
20+
},
21+
"c8": {
22+
"exclude": [
23+
"test/**",
24+
"vendor/**"
25+
]
1726
},
1827
"keywords": [],
1928
"author": "Mikeal Rogers <[email protected]> (https://www.mikealrogers.com/)",
@@ -22,10 +31,6 @@
2231
".": {
2332
"import": "./src/index.js"
2433
},
25-
"./basics": {
26-
"import": "./src/basics-import.js",
27-
"browser": "./src/basics-browser.js"
28-
},
2934
"./cid": {
3035
"import": "./src/cid.js"
3136
},
@@ -45,10 +50,19 @@
4550
"import": "./src/bases/base64-import.js",
4651
"browser": "./src/bases/base64-browser.js"
4752
},
53+
"./hashes/hasher": {
54+
"import": "./src/hashes/hasher.js"
55+
},
56+
"./hashes/digest": {
57+
"import": "./src/hashes/digest.js"
58+
},
4859
"./hashes/sha2": {
4960
"browser": "./src/hashes/sha2-browser.js",
5061
"import": "./src/hashes/sha2.js"
5162
},
63+
"./codecs/codec": {
64+
"import": "./src/codecs/codec.js"
65+
},
5266
"./codecs/json": {
5367
"import": "./src/codecs/json.js"
5468
},
@@ -69,10 +83,8 @@
6983
]
7084
},
7185
"dependencies": {
72-
"base-x": "^3.0.8",
7386
"buffer": "^5.6.0",
74-
"cids": "^1.0.0",
75-
"varint": "^5.0.0"
87+
"cids": "^1.0.0"
7688
},
7789
"directories": {
7890
"test": "test"

0 commit comments

Comments
 (0)