Skip to content

Commit bec1bd6

Browse files
committed
[NFC][ntuple] fix spec of sharded clusters / cluster summary flags
1 parent a7f8615 commit bec1bd6

File tree

1 file changed

+9
-9
lines changed

1 file changed

+9
-9
lines changed

tree/ntuple/v7/doc/specifications.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -684,7 +684,7 @@ Followed by the page list envelope link.
684684

685685
To compute the minimum entry number, take first entry number from all clusters in the cluster group,
686686
and take the minimum among these numbers.
687-
The entry span is the number of entries that are (partially for sharded clusters) covered by this cluster group.
687+
The entry span is the number of entries that are covered by this cluster group.
688688
The entry range allows for finding the right page list for random access requests to entries.
689689
The number of clusters information allows for using consistent cluster IDs even if cluster groups are accessed non-sequentially.
690690

@@ -709,19 +709,19 @@ The cluster summary record frame contains the entry range of a cluster:
709709
| |
710710
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
711711
| Number of Entries |
712-
+ +-+-+-+-+
713-
| | Flags |
712+
+ +-+-+-+-+-+-+-+-+
713+
| | Flags |
714714
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
715715
```
716716

717-
If flag 0x01 (sharded cluster) is set,
718-
an additional 32bit integer containing the column group ID follows the flags field.
719-
If flags is zero, the cluster stores the entry range of _all_ the original columns
720-
_including_ the columns from extension headers.
721-
722717
The order of the cluster summaries defines the cluster IDs,
723718
starting from the first cluster ID of the cluster group that corresponds to the page list.
724719

720+
Flag 0x01 is reserved for a future specification version that will support sharded clusters.
721+
The future use of sharded clusters will break forward compatibility and thus introduce a corresponding feature flag.
722+
For now, readers should abort when this flag is set.
723+
Other flags should be ignored.
724+
725725
#### Page Locations
726726

727727
The page locations are stored in a nested list frame as follows.
@@ -1019,7 +1019,7 @@ The limits refer to a single RNTuple and do not consider combinations/joins such
10191019
| Maximum number of cluster groups | 4B (foreseen: <10k) | List frame limits |
10201020
| Maximum number of clusters per group | 4B (foreseen: <10k) | List frame limits, cluster group summary encoding |
10211021
| Maximum number of pages per cluster per column | 4B | List frame limits |
1022-
| Maximum number of entries per cluster | 2^60 | Cluster summary encoding |
1022+
| Maximum number of entries per cluster | 2^56 | Cluster summary encoding |
10231023
| Maximum string length (meta-data) | 4GB | String encoding |
10241024
| Maximum RBlob size | 128 PiB | 1GiB / 8B * 1GiB (with maxKeySize=1GiB, offsetSize=8B) |
10251025

0 commit comments

Comments
 (0)