Skip to content

Commit b2b0c4f

Browse files
AljoschaMeyerAljoschaMeyer
authored
Merge pull request #177 from earthstar-project/drop_bab
Make drop format bab-aware, remove mandatory encryption, adds willow25 parameters for the drop format. Also makes the URI spec official.
2 parents 28f5b9c + 2f170d2 commit b2b0c4f

File tree

5 files changed

+348
-188
lines changed

5 files changed

+348
-188
lines changed

‎src/pages/specs.tsx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,9 @@ export const specs = (
7979
<Li>
8080
<R n="lcmux">Multiplexing and Flow Control</R>
8181
</Li>
82+
<Li>
83+
<R n="uris">Willow URIs</R>
84+
</Li>
8285
</Ul>
8386
</Nav>
8487
</PageTemplate>

‎src/pages/specs/drop_format.tsx‎

Lines changed: 182 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ import { M } from "macromania-katex";
1717
import { PreviewScope } from "macromania-previews";
1818
import { DefVariant } from "macromania-rustic";
1919
import {
20+
Bitfield,
2021
bitfieldIff,
2122
C64Encoding,
2223
c64Tag,
@@ -119,26 +120,23 @@ export const drop_format = (
119120
infrastructure users already have.
120121
</P>
121122

122-
<P>
123-
Finally, given that this protocol cannot interactively authorise users
124-
(e.g. via{" "}
125-
<R n="private_interest_overlap">
126-
private interest intersection
127-
</R>), drops are always fully encrypted.
128-
</P>
129-
130123
<Hsection n="drop_format_parameters" title="Parameters">
131124
<PreviewScope>
132125
<P>
133126
<Marginale>
134-
See <R n="willow25" />{" "}
127+
See <R n="willow25_drop_format" />{" "}
135128
for a default recommendation of parameters.
136129
</Marginale>
137130
In order to use the drop format, one must first specify a full
138131
suite of instantiations of the{" "}
139132
<R n="willow_parameters">
140133
parameters of the core Willow data model
141-
</R>. In addition to this, the drop format requires the following:
134+
</R>. The <R n="hash_payload" />{" "}
135+
parameter of the data model must be a member of the{" "}
136+
<AE href="https://worm-blossom.github.io/bab/">
137+
Bab family of hash functions
138+
</AE>. In addition to this, the drop format requires the
139+
following:
142140
</P>
143141

144142
<Ul>
@@ -210,34 +208,33 @@ export const drop_format = (
210208
<R n="sl_default_payload_digest" />)
211209
</Code>, and
212210
</Li>
213-
<Li>
214-
a function <DefFunction n="encrypt" />{" "}
215-
which encrypts the final compiled encoding.
216-
</Li>
217211
</Ul>
218212
</PreviewScope>
219213
</Hsection>
220214

221215
<Hsection n="drop_format_desc" title="Format">
216+
<P>
217+
A <R n="drop" />{" "}
218+
is a collection of entries, together with verifiable subslices of
219+
their payloads. The payload slice verification relies on{" "}
220+
<AE href="https://worm-blossom.github.io/bab/">Bab</AE>;{" "}
221+
<Rs n="drop" /> store the{" "}
222+
<AE href="https://worm-blossom.github.io/bab/#baseline_slice">
223+
baseline verifiable slice streams
224+
</AE>{" "}
225+
of payload slices.
226+
</P>
227+
222228
<PreviewScope>
223229
<P>
224230
Let <DefValue n="drop_entries" r="entries" /> a sequence of{" "}
225231
pairs of a <Rs n="PossiblyAuthorisedEntry" />{" "}
226-
and either a full corresponding{" "}
227-
<Sidenote
228-
note={
229-
<>
230-
That is, a <R n="Payload" /> whose length is the{" "}
231-
<R n="entry_payload_length" /> of the{" "}
232-
<R n="Entry" />, and which hashes to the{" "}
233-
<R n="entry_payload_digest" /> of the <R n="Entry" />.
234-
</>
235-
}
236-
>
237-
<R n="Payload" />
238-
</Sidenote>{" "}
239-
or the marker value{" "}
240-
<DefVariant n="sl_none" r="none" />, all from a single{" "}
232+
and a sequence of non-overlapping slices of payload{" "}
233+
<AE href="https://worm-blossom.github.io/bab/#chunk">chunks</AE>
234+
{" "}
235+
with stricly increasing start offsets. The{" "}
236+
<Rs n="PossiblyAuthorisedEntry" /> must all come from a single
237+
{" "}
241238
<R n="namespace" /> with <R n="NamespaceId" />{" "}
242239
<DefValue n="sl_namespace" r="namespace" />. The Sideloading
243240
Protocol then defines how to encode this sequence as a single
@@ -264,18 +261,7 @@ export const drop_format = (
264261
</PreviewScope>
265262

266263
<P>
267-
Then,{" "}
268-
<DefValue
269-
n="side_contents"
270-
r="contents"
271-
preview={
272-
<>
273-
The unencrypted encoding of a sequence of{" "}
274-
<Rs n="AuthorisedEntry" /> and optionally their{" "}
275-
<Rs n="Payload" />, as used in a <R n="drop" />.
276-
</>
277-
}
278-
/>{" "}
264+
Then, the actual <R n="drop" />{" "}
279265
is the concatenation of the following:
280266
</P>
281267

@@ -308,7 +294,10 @@ export const drop_format = (
308294
]}
309295
/>,
310296
<M>
311-
<DefValue n="sl_payload" r="optional\_payload_i" />
297+
<DefValue
298+
n="sl_payload_slices"
299+
r="payload\_slices_i"
300+
/>
312301
</M>,
313302
]}
314303
/>
@@ -323,13 +312,6 @@ export const drop_format = (
323312
<Encoding
324313
idPrefix="sl_enc_nested"
325314
bitfields={[
326-
bitfieldIff(
327-
<>
328-
<Code>
329-
<R n="sl_payload" /> != <R n="sl_none" />
330-
</Code>
331-
</>,
332-
),
333315
bitfieldIff(
334316
<>
335317
<Code>
@@ -386,10 +368,6 @@ export const drop_format = (
386368
</>
387369
),
388370
},
389-
{
390-
count: 1,
391-
content: <>arbitrary</>,
392-
},
393371
c64Tag(
394372
"timediff",
395373
2,
@@ -420,6 +398,41 @@ export const drop_format = (
420398
</Code>
421399
</>,
422400
),
401+
{
402+
count: 2,
403+
id: "slice_mode",
404+
content: (
405+
<>
406+
Two bits, selected as follows:
407+
408+
<Ul>
409+
<Li>
410+
If <R n="sl_payload_slices" />{" "}
411+
is empty, these two bits may be set to{" "}
412+
<Code>00</Code>.
413+
</Li>
414+
<Li>
415+
If <R n="sl_payload_slices" />{" "}
416+
consists of a single slice, and that slice
417+
constitutes the full payload of{" "}
418+
<R n="sl_e" />, these two bits may be set to
419+
{" "}
420+
<Code>01</Code>.
421+
</Li>
422+
<Li>
423+
If <R n="sl_payload_slices" />{" "}
424+
consists of a single slice, and that slice
425+
starts at chunk index zero, these two bits may
426+
be set to <Code>10</Code>.
427+
</Li>
428+
<Li>
429+
These two bits may always be set to{" "}
430+
<Code>11</Code>.
431+
</Li>
432+
</Ul>
433+
</>
434+
),
435+
},
423436
]}
424437
contents={[
425438
<EncConditional
@@ -470,7 +483,7 @@ export const drop_format = (
470483
</CodeFor>,
471484
<CodeFor
472485
enc="SideloadingEncodeAuthorisationToken"
473-
notStandalone
486+
// notStandalone
474487
relativeTo={
475488
<>
476489
the pair of (<R n="sl_e">
@@ -487,41 +500,132 @@ export const drop_format = (
487500
>
488501
<R n="sl_a" />
489502
</CodeFor>,
490-
<EncConditional
491-
condition={
492-
<>
493-
<Code>
494-
<R n="sl_payload" /> != <R n="sl_none" />
495-
</Code>
496-
</>
497-
}
498-
>
499-
<RawBytes lowercase noPeriod>
500-
<R n="sl_payload" />
501-
</RawBytes>
502-
</EncConditional>,
503+
<Ul>
504+
<Li>
505+
If bits <Bitfield id={"slice_mode"} /> are{" "}
506+
<Code>00</Code>: the empty string.
507+
</Li>
508+
<Li>
509+
If bits <Bitfield id={"slice_mode"} /> are{" "}
510+
<Code>01</Code>:{" "}
511+
<RawBytes lowercase noPeriod>
512+
the payload of <R n="sl_e" />
513+
</RawBytes>.
514+
</Li>
515+
<Li>
516+
If bits <Bitfield id={"slice_mode"} /> are{" "}
517+
<Code>10</Code>: the concatenation of{" "}
518+
<Ul>
519+
<Li>
520+
<C64Standalone notStandalone>
521+
the length of the slice in chunks
522+
</C64Standalone>, and
523+
</Li>
524+
<Li>
525+
<RawBytes lowercase noPeriod>
526+
the{" "}
527+
<AE href="https://worm-blossom.github.io/bab/#kgrouped_baseline">
528+
64-grouped
529+
</AE>{" "}
530+
<AE href="https://worm-blossom.github.io/bab/#baseline_slice">
531+
baseline verifiable slice stream
532+
</AE>{" "}
533+
of the single slice in{" "}
534+
<R n="sl_payload_slices" />
535+
</RawBytes>.
536+
</Li>
537+
</Ul>
538+
</Li>
539+
<Li>
540+
If bits <Bitfield id={"slice_mode"} /> are{" "}
541+
<Code>11</Code>: the concatenation of{" "}
542+
<Ul>
543+
<Li>
544+
<C64Standalone notStandalone>
545+
the number in slices in{" "}
546+
<R n="sl_payload_slices" />
547+
</C64Standalone>,
548+
</Li>
549+
<Li>
550+
<C64Standalone notStandalone>
551+
the final chunk index (inclusive) of the final
552+
slice in <R n="sl_payload_slices" />
553+
</C64Standalone>, and
554+
</Li>
555+
<Li>
556+
<EncIterator
557+
val={
558+
<>
559+
<M>
560+
<DefValue n="drop_slice_j" r="slice_j" />
561+
</M>
562+
</>
563+
}
564+
iter={
565+
<>
566+
<R n="sl_payload_slices" />
567+
</>
568+
}
569+
>
570+
<Encoding
571+
idPrefix="sl_enc_nested_nested"
572+
bitfields={[]}
573+
contents={[
574+
<C64Standalone>
575+
the start chunk index of{" "}
576+
<R n="drop_slice_j" />{" "}
577+
minus the end chunk index of the
578+
previously encoded slice for this entry
579+
(or zero for the first slice of the entry)
580+
</C64Standalone>,
581+
<C64Standalone>
582+
the length of <R n="drop_slice_j" />{" "}
583+
in chunks
584+
</C64Standalone>,
585+
<RawBytes>
586+
the{" "}
587+
<AE href="https://worm-blossom.github.io/bab/#kgrouped_baseline">
588+
64-grouped
589+
</AE>{" "}
590+
<AE href="https://worm-blossom.github.io/bab/#baseline_slice">
591+
baseline verifiable slice stream
592+
</AE>{" "}
593+
of{" "}
594+
<R n="drop_slice_j" />, with the maximal
595+
{" "}
596+
<AE href="https://worm-blossom.github.io/bab/#left_skip">
597+
left-skip
598+
</AE>{" "}
599+
such that the only omitted node data has
600+
been included in the encoding of an
601+
earlier slice in{" "}
602+
<R n="sl_payload_slices" />
603+
</RawBytes>,
604+
]}
605+
/>
606+
</EncIterator>
607+
</Li>
608+
</Ul>
609+
</Li>
610+
</Ul>,
503611
]}
504612
/>
505613
</EncIterator>,
506614
]}
507615
/>
508616
</PreviewScope>
617+
</Hsection>
509618

619+
<Hsection n="sideload_transport" title="Transport">
510620
<P>
511-
The the <R n="drop" /> corresponding to the sequence{" "}
512-
<R n="drop_entries" /> is the result of applying <R n="encrypt" />
513-
{" "}
514-
to <R n="side_contents" />.
621+
Once created, a <R n="drop" />{" "}
622+
can be transported by whatever means a single bytestring can be
623+
transferred, so that the decoded <R n="drop_entries" />{" "}
624+
can be ingested by its recipients.
515625
</P>
516-
</Hsection>
517626

518-
<Hsection n="sideload_transport" title="Transport">
519627
<P>
520-
Once created, a{" "}
521-
<R n="drop" />" can be transported by whatever means a single
522-
bytestring can be transferred, to be decrypted and the recovered
523-
{" "}
524-
<R n="drop_entries" /> ingested by its intended recipient.
628+
We <Em>highly</Em> recommend encrypting drops for transport.
525629
</P>
526630
</Hsection>
527631

‎src/pages/specs/encodings.tsx‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1245,7 +1245,7 @@ export const encodings = (
12451245
<Code>
12461246
<AccessStruct field="TimeRangeStart">
12471247
<ValAccess field="AreaTime" />
1248-
</AccessStruct>
1248+
</AccessStruct>{" "}
12491249
-{" "}
12501250
<AccessStruct field="TimeRangeStart">
12511251
<RelAccess field="AreaTime" />
@@ -1272,7 +1272,7 @@ export const encodings = (
12721272
<Code>
12731273
<AccessStruct field="TimeRangeEnd">
12741274
<ValAccess field="AreaTime" />
1275-
</AccessStruct>
1275+
</AccessStruct>{" "}
12761276
-{" "}
12771277
<AccessStruct field="TimeRangeStart">
12781278
<RelAccess field="AreaTime" />
@@ -2022,7 +2022,8 @@ export const encodings = (
20222022
<M>
20232023
<DefValue n="ccrpia_area_base" r="area_{-1}" />
20242024
</M>{" "}
2025-
be the <R n="subspace_area" /> of <ValAccess field="communal_cap_user" />.
2025+
be the <R n="subspace_area" /> of{" "}
2026+
<ValAccess field="communal_cap_user" />.
20262027
</P>
20272028
}
20282029
bitfields={[

0 commit comments

Comments
 (0)