add grids

Author: alimanfoo

slides/v3-update-20190619.html

@@ -39,6 +39,9 @@
                         .reveal h1, .reveal h2, .reveal h3, .reveal h4, .reveal h5 {
                                 text-transform: none;
                         }
+			.reveal p, .reveal li {
+			        font-size: 0.9em;
+			}
                 </style>
 
 	</head>

slides/v3-update-20190619.md

@@ -22,6 +22,35 @@ Alistair Miles ([@alimanfoo](https://github.com/alimanfoo))
 
 ====
 
+### Design principles
+
+1. Hackable
+2. Parallel
+3. Distributed
+
+===
+
+### Hackable
+
+* Easy to implement
+* Easy to extend with new functionality
+* Easy to inspect and manipulate data with generic tools
+
+===
+
+### Parallel
+
+* Think "what happens if two workers do X at the same time"?
+* Avoid race conditions
+
+===
+
+### Distributed
+
+* Accommodate eventual consistency
+
+====
+
 ### Modular spec architecture
 
 * [Core protocol spec](https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/protocol/core/v3.0.html)
@@ -150,14 +179,15 @@ document. Store.*
 
 ### Node names - restrictions
 
-* Node paths are used by users to access nodes and navigate a
+* Node paths are used by users to access nodes and explore/navigate a
   hierarchy.
 
 * N.B., node paths are also used to form storage keys (see later).
 
 * To try and ensure compatibility with a variety of storage systems,
-  the core protocol currently states fairly heavy restrictions on node
-  names.
+  the core protocol currently states fairly heavy
+  [restrictions](https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/protocol/core/v3.0.html#node-names)
+  on node names.
 
   * Includes restriction to ASCII alpha-numeric characters, "-", "_"
     and ".".
@@ -193,15 +223,121 @@ document. Store.*
 
 ====
 
-### Core protocol - data types
+### [Core protocol - data types](https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/protocol/core/v3.0.html#id5)
 
-@@TODO
+* Boolean (single byte)
+
+* Integer (signed or unsigned; 1, 2, 4, 8 bytes; little- or
+  big-endian)
+
+* Float (2, 4, 8 bytes; little- or big-endian)
+
+* Any other data type can be defined via a protocol extension
+
+  * E.g., [datetime data
+    types](https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/protocol/extensions/datetime-dtypes/v1.0.html)
+
+===
+
+### Data types - identifiers
+
+* Each data type needs an identifier for use in metadata documents.
+
+* E.g., "bool", "i1", "<i4", ">u8", "<f2", etc.
 
 ====
 
 ### Core protocol - chunk grids
 
-@@TODO
+* A chunk grid defines a set of chunks which contain the elements of an array.
+
+* The chunks of a grid form a tessellation of the array space, which
+  is a space defined by the dimensionality and shape of the array.
+
+  * => Every element of the array is a member of one chunk, and there
+    are no gaps or overlaps between chunks.
+
+===
+
+### Grid types
+
+* In general there are several different possible types of grid.
+
+* The core protocol defines [regular
+  grids](https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/protocol/core/v3.0.html#regular-grids).
+  
+* Other grid types can be defined via protocol extensions.
+
+* Any grid type must define:
+
+  * How the array space is divided into chunks.
+  
+  * A unique identifier for each chunk in the grid (used to form
+    storage keys, see later).
+
+===
+
+### [Regular grids](https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/protocol/core/v3.0.html#regular-grids)
+
+* Grid type where each chunk is a (hyper)rectangle of the same shape.
+
+* I.e., grid type used in HDF5 and Zarr v2.
+
+* Each chunk has a grid index, which is a tuple of grid coordinates.
+
+  * E.g., grid index (0, 3, 7) means first chunk along first
+    dimension, fourth chunk along second dimension, 8th chunk along
+    third dimension.
+
+===
+
+### Regular grids - chunk identifiers
+
+* Chunk identifier is formed from grid index.
+
+  * E.g., chunk at grid index (0, 3, 7) has identifier "0.3.7".
+  
+* Default separator is "." but can be changed (e.g., to "/") in array
+  metadata (see later).
+
+===
+
+### Regular grids - edge chunks
+
+* All chunks have the same shape.
+
+* If the length of any array dimension is not perfectly divisible by
+  the chunk length along the same dimension, the grid will overhang
+  the edge of the array space.
+  
+* Spec currently doesn't say any more about how to handle edge chunks,
+  maybe it should?
+
+  * E.g., suggest using array fill value to fill contents of edge
+    chunks beyond the array space.
+	
+* Other approaches (e.g., truncated edge chunks) could be defined as a
+  different grid type via a protocol extension.
+
+===
+
+### Regular grids - resizing arrays
+
+* Regular grid supports growing and shrinking an array along any
+  dimension.
+
+  * Growing only requires change to array metadata (update array
+    shape), no chunk data needs to be added or modified.
+	
+  * Shrinking requires change to array metadata (update array shape)
+    plus delete any chunks now completely outside the array space.
+	
+* Regular grid does not support growing an array in "negative"
+  direction, i.e., prepending.
+  
+  * But could define a grid type that does support this via a protocol
+    extension.
+
 
 ====