Merge pull request #20 from MSanKeys963/main

Add meeting notes for 2022-11-03, TL;DRs and text blurb for ZEP meetings on homepage

Author: sanketverma1704

index.markdown

@@ -32,16 +32,20 @@ We intend ZEPs to be the primary mechanism for evolving the spec, collecting
 community input on major issues and documenting the design decision that has
 gone into Zarr.
 
+### ZEP Meetings 🧑🏻‍💻
+
+We hold bi-weekly ZEPs meetings to propose, discuss, review and finalize discussions around current ZEPs and Zarr Specification. More info available here: [https://zarr.dev/zeps/meetings/](https://zarr.dev/zeps/meetings/)
+
 ---
 
-### Contributing
+### Contributing 🤝🏻
 
 If you wish to contribute to Zarr's codebase, propose a new ZEP(s), website, blog
 posts or in any way, please visit Zarr's GitHub [here](https://github.com/zarr-developers/).
 You can discuss the change you want to see by opening an issue in the appropriate
 repository, or if the issue is already present, feel free to submit a pull request.
 
-### Code of Conduct
+### Code of Conduct ⚖️
 
 ZEPs are governed by Zarr Community's
 [CODE OF CONDUCT](https://github.com/zarr-developers/.github/blob/main/CODE_OF_CONDUCT.md).

meetings/2022-09-08.md

@@ -46,7 +46,7 @@ This was the first ZEP meeting ever. Representatives from the [Zarr Implementati
       - JS: in this cycle and not limited iterations is just the limited time.
         - but for now, trying to make batched changes
 
-**Agenda:**
+**Meeting Minutes:**
 - JS: **review of memory order decision number-16 from list**
   - zarr's goal is interoperability. therefore propose to keep C & F (benefit for community)
   - could support read only, even with a transpose (if too slow, add a warning?)

meetings/2022-09-22.md

@@ -10,7 +10,11 @@ nav_order: 2
 
 **Attending:** Ward Fisher (WF), Josh Moore (JM), Ryan Abernathey (RA), Jeremy Maitin-Shepard (JMS), Dennis Heimbigner (DH)
 
-**Agenda:**
+## TL;DR
+
+Consolidate metadata needs an extension for V3, which might result in a new ZEP. Next, JMS shared a document titled ‘Optionally-cooperative distributed b-tree for Tensorstore’. The participants discussed the document after that. After that, JM initiated the discussion on codecs-registry, which was built by one of the GSoC students this summer. The meeting ended with a discussion on the path to the metadata files.
+
+**Meeting Minutes:**
 
 - Java/NetCDF side:
   - JM: Sanket met people 

meetings/2022-10-06.md

@@ -10,9 +10,12 @@ nav_order: 3
 
 **Attending:** Ward Fisher (WF), Josh Moore (JM), Jeremy Maitin-Shepard (JMS), Greg Lee (GL), Jonathan Striebel (JS)
 
-**Agenda**:
+## TL;DR
+
+JM shared that there were some good conversations around OME-Zarr yesterday. The summary is available [here](https://forum.image.sc/t/ome-ngff-community-call-transforms-and-tables/71792/10). WF shared that Kitware is looking for partners and a link to the sign-up form. GL shared that during the CZI Open-Science Summit 2022, he worked on writing tests for Xarray. After this, there was an extensive discussion on URL syntax initiated by JMS.
+
+**Updates:**
 
-* Misc
   * miscellaneous reading before the meeting (JM)
     - [https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/](https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/)
     - [https://github.com/kaitai-io/kaitai_struct/issues/125](https://github.com/kaitai-io/kaitai_struct/issues/125)
@@ -25,6 +28,9 @@ nav_order: 3
   * xarray test (GL)
     - during czi conference.
     - release of 2.13 hopefully fixed it all :tada:
+
+**Meeting Minutes:**
+
 * URL syntax? (JMS)
   - helps to figure out the metadata location.
   - Josh: great idea. have several ongoing discussions at the NGFF level

meetings/2022-10-20.md

@@ -6,16 +6,24 @@ parent: ZEP meetings
 nav_order: 4
 ---
 
-## 2022-10-20
+# 2022-10-20
 
 **Attending:** Ward Fisher (WF), John Kirkham (JK), Jeremy Maitin-Shepard (JMS)
 
-* Misc
-    * (WF) Working on maintenance netcdf release candidate (`v4.9.1-rc1`). No new features, just bug fixes and improvements.
-    * (JMS) Added CMake support to TensorStore
-    * Discussion about CMake, dependency management
-        * https://cmake.org/cmake/help/book/mastering-cmake/chapter/CDash.html
-        * https://github.com/cpm-cmake/CPM.cmake
+## TL;DR
+
+WF is working on the maintenance NetCDF release candidate (`v4.9.1-rc1`), and JMS added CMake support to TensorStore. After this, JMS initiated a discussion on Path structure and was stretched for the remaining meeting.
+
+**Updates:**
+
+- (WF) Working on maintenance netcdf release candidate (`v4.9.1-rc1`). No new features, just bug fixes and improvements.
+- (JMS) Added CMake support to TensorStore
+- Discussion about CMake, dependency management
+    - https://cmake.org/cmake/help/book/mastering-cmake/chapter/CDash.html
+    - https://github.com/cpm-cmake/CPM.cmake
+
+**Meeting Minutes:**
+
 * (JMS) Path structure
     * Require or encourage root directory to end in .zarr
     * How to name all the metadata files?
@@ -61,4 +69,4 @@ nav_order: 4
             * (JMS) Has a group of files for mesh
             * (JK) Maybe ignore specific paths?
             * (WF) Having mixed media is valuable though can be logisticially tricky
-            * (WF) What defines Zarr as a data model? At least need to say some behavior is undefined (mixed media). Ideally ignores mixed media files.
+            * (WF) What defines Zarr as a data model? At least need to say some behavior is undefined (mixed media). Ideally ignores mixed media files.

meetings/2022-11-03.md

@@ -0,0 +1,54 @@
+---
+layout: default
+title: 3rd November
+description: ZEPs Meeting Notes for 2022-11-03
+parent: ZEP meetings
+nav_order: 5
+---
+
+# 2022-11-03
+
+**Attending:** Josh Moore (JM), Jonathan Striebel (JS), Jeremy Maitin-Shepherd (JMS), Sanket Verma (SV)
+
+## TL;DR
+
+Discussions were held on how to move forward with ZEP1 quickly. The summary can be viewed [here](https://github.com/zarr-developers/zarr-specs/pull/149#issuecomment-1302440391). Then the attendees discussed extensions in V3, and JMS is considering trying with non-zero origin. SV joined the meeting after 30 mins. After that, JS mentioned some high-level issues looming around V3 spec.
+
+**Meeting Minutes:**
+
+* JMS: number of PRs that could be merged into the working draft
+  - JS: don't want to just close it
+  - JM: can we cross link e.g. JMS' PR? Yes.
+  - ==> once all cross-linked close PR.
+* JS: when to merge?
+  - JM: when it matches the consensus?
+  - JS: ok, but don't have merge rights.
+  - ==> Let's merge proactively.
+* see: https://github.com/zarr-developers/zarr-specs/pull/149#issuecomment-1302440391
+* extensions
+  - JMS: thinking of trying with non-0-origin
+  - JM: think that's a general principle we should try for all issues/PR is "could it be an extension"
+  - JMS: thinking of extensions as plugins? Not exactly.
+  - JS: how to influence if an implenentation adopts an extension? if there's a concrete implementation / clear interface
+  - JMS: agreed and some obvious ones (codecs) but not clear there will be a broader abstraction
+    - JS: "index transformer" _perhaps_
+    - or as transformer _if_ multiple of chunking
+    - JMS: unfortunate limitation
+  - JMS: re: transformers - it doesn't make sense to compose a different storage transform _before_ sharding
+     - JS: depends. cache of chunks or shards? also checksum
+     - JM: codec is similar
+     - JMS: caching enabled in code, but not in zarr metadata
+     - JS: that's in spec, yes. "runtime-only" but still before or after
+     - JMS: when implementing sharding, would check if it's first
+       - want to be able to tell the user "this is the graularity to write"
+       - JS: good that it's flexible. like c/f order.
+       - JS: mention in implementation "sharding must be first"
+     - JMS: composing makes for useful extension point
+     - JS: most important point: are we sure enough about the extension points?
+- _Sanket joins_ 🧑🏻‍💻
+- Jonathan: high-level issues looming
+  - paths/URL discussion (needs an issue)
+  - global transformers
+  - variable chunk length (possibly origin offset)
+  - indexing more abstract
+  - upgrade path! (`{“extension”: [“@v2-layout”]}`)