Merge pull request #53 from GraphBLAS/wma/custom-format-keyword

add a keyword for custom formats

Author: willow-ahrens

README.md

@@ -15,10 +15,10 @@ Here is a table listing the current tensor frameworks that support the format:
 
 | Language | Framework | Status | Notes |
 | -------- | ------ | ------ | ----- |
-| C | [binsparse-reference-c](https://github.com/GraphBLAS/binsparse-reference-c) | under development | converts between binsparse V1.0 and custom in-memory sparse matrices |
-| C++ | [binsparse-reference-impl](https://github.com/GraphBLAS/binsparse-reference-impl) | under development | converts between binsparse V1.0 and custom in-memory sparse matrices | 
-| Julia | [Finch.jl](https://willowahrens.io/Finch.jl/dev/fileio/) | under development | converts between binsparse V1.0 and V2.0 and Finch matrices and tensors |
-| Python | [binsparse-python](https://github.com/ivirshup/binsparse-python) | under development | converts between binsparse V1.0 and scipy.sparse matrices |
+| C | [binsparse-reference-c](https://github.com/GraphBLAS/binsparse-reference-c) | under development | converts between binsparse and in-memory sparse matrices |
+| C++ | [binsparse-reference-impl](https://github.com/GraphBLAS/binsparse-reference-impl) | under development | converts between binsparse and in-memory sparse matrices | 
+| Julia | [Finch.jl](https://willowahrens.io/Finch.jl/dev/fileio/) | under development | converts between binsparse and Finch matrices and tensors, supports custom format spec |
+| Python | [binsparse-python](https://github.com/ivirshup/binsparse-python) | under development | converts between binsparse and scipy.sparse matrices |
 
 ### Editing
 

spec/latest/index.bs

@@ -287,10 +287,10 @@ Pairs must not be duplicated.
 
 Coordinate format is an alias for [[#coor_format]] format.
 
-### Version 2.0 only: Custom Formats ### {#custom_formats}
+### Custom Formats ### {#custom_formats}
 
-The contents of this section will be finalized with the release of Binsparse
-V2.0, and are subject to change until then.
+The contents of this section are optional for all parsers, but enable
+customizable sparse formats for matrices and tensors.
 
 Binsparse describes custom multidimensional formats hierarchically.  We can
 understand these formats as arrays of arrays, where the parent array and
@@ -318,7 +318,7 @@ and tensors. For example, a dense vector of sparse vectors is equivalent to the
 CSR matrix format, and a sparse vector of sparse vectors is equivalent to the
 hypersparse DCSR matrix format.
 
-When defining a custom format, the outermost `subformat` key is defined as the
+When defining a custom format, the outermost `level` key is defined as the
 root level descriptor (a level which will only hold one array). If a level holds
 many different arrays, we refer to the `p`th array as the array in position `p`.
 
@@ -329,24 +329,25 @@ to represent is `A` and the tensor described by the format descriptor is `B`,
 then `A[i_1, ..., i_n] = B[i_(transpose[1]), ..., i_(transpose[n])]`. `transpose` must
 be a permutation.
 
-If the format key is a dictionary, the `level` key must be present and shall
-describe the storage format of the level used to represent the sparse array.
+If the `custom` key is present, it holds a dictionary for custom formats. The
+root level is stored under the `level` key. Each level mush have a `level_desc`
+attribute which describe the storage format of the level.
 
 The level descriptors are dictionaries defined as follows:
 
 #### Element #### {#element_level}
 
-If the level key is "element", the level represents zero or more scalars.
+If the level descriptor is "element", the level represents zero or more scalars.
 
 : values
 :: Array of size `number_of_positions` whose `p`th element holds the value of the scalar at position `p`.
 
 #### Dense #### {#dense_level}
 
-If the level key is "dense", the `subformat` key must be present.  The `rank`
+If the level descriptor is "dense", the `level` key must be present.  The `rank`
 key must be present, and set to an integer `r` greater than or equal to 1.  The
 dense level represents zero or more r-dimensional dense arrays whose elements
-are themselves arrays specified by `subformat`. For example, a dense level
+are themselves arrays specified by `level`. For example, a dense level
 of
 rank 2 represents a collection of dense matrices of subarrays. 
 
@@ -363,10 +364,10 @@ of the sublevel.
 
 #### Sparse #### {#sparse_level}
 
-If the level key is "sparse", the `subformat` key must be present.  The
+If the level descriptor is "sparse", the `level` key must be present.  The
 `rank` key must be present, and set to an integer `r` greater than or equal to
 `1`.  The sparse level represents zero or more `r`-dimensional sparse arrays
-whose non-implicit elements are themselves arrays specified by `subformat`. For
+whose non-implicit elements are themselves arrays specified by `level`. For
 example, a sparse level of rank 1 represents a collection of sparse vectors of
 subarrays.
 
@@ -387,17 +388,17 @@ length of any of the `indices` arrays in this level.
 
 ### Equivalent Formats ### {#equivalent_formats}
 
-The following formats are equivalent
+The following formats are equivalent. Parsers which support custom formats should also write `format` aliases when appropriate.
 
 #### DVEC #### {#dvec_format_equiv}
 
 ```json
-"format": {
-  "subformat": {
-    "level": "dense",
+"custom": {
+  "level": {
+    "level_desc": "dense",
     "rank": 1,
-    "subformat": {
-      "level": "element",
+    "level": {
+      "level_desc": "element",
     }
   }
 }
@@ -406,15 +407,15 @@ The following formats are equivalent
 #### DMATR #### {#dmatr_format_equiv}
 
 ```json
-"format": {
-  "subformat": {
-    "level": "dense",
+"custom": {
+  "level": {
+    "level_desc": "dense",
     "rank": 1,
-    "subformat": {
-      "level": "dense",
+    "level": {
+      "level_desc": "dense",
       "rank": 1,
-      "subformat": {
-        "level": "element",
+      "level": {
+        "level_desc": "element",
       }
     }
   }
@@ -424,16 +425,16 @@ The following formats are equivalent
 #### DMATC #### {#dmatr_format_equiv}
 
 ```json
-"format": {
+"custom": {
   "transpose": [1, 0],
-  "subformat": {
-    "level": "dense",
+  "level": {
+    "level_desc": "dense",
     "rank": 1,
-    "subformat": {
-      "level": "dense",
+    "level": {
+      "level_desc": "dense",
       "rank": 1,
-      "subformat": {
-        "level": "element",
+      "level": {
+        "level_desc": "element",
       }
     }
   }
@@ -443,12 +444,12 @@ The following formats are equivalent
 #### CVEC #### {#cvec_format_equiv}
 
 ```json
-"format": {
-  "subformat": {
-    "level": "sparse",
+"custom": {
+  "level": {
+    "level_desc": "sparse",
     "rank": 1,
-    "subformat": {
-      "level": "element",
+    "level": {
+      "level_desc": "element",
     }
   }
 }
@@ -457,15 +458,15 @@ The following formats are equivalent
 #### CSR #### {#csr_format_equiv}
 
 ```json
-"format": {
-  "subformat": {
-    "level": "dense",
+"custom": {
+  "level": {
+    "level_desc": "dense",
     "rank": 1,
-    "subformat": {
-      "level": "sparse",
+    "level": {
+      "level_desc": "sparse",
       "rank": 1,
-      "subformat": {
-        "level": "element",
+      "level": {
+        "level_desc": "element",
       }
     }
   }
@@ -475,16 +476,16 @@ The following formats are equivalent
 #### CSC #### {#csc_format_equiv}
 
 ```json
-"format": {
+"custom": {
   "transpose": [1, 0],
-  "subformat": {
-    "level": "dense",
+  "level": {
+    "level_desc": "dense",
     "rank": 1,
-    "subformat": {
-      "level": "sparse",
+    "level": {
+      "level_desc": "sparse",
       "rank": 1,
-      "subformat": {
-        "level": "element",
+      "level": {
+        "level_desc": "element",
       }
     }
   }
@@ -494,15 +495,15 @@ The following formats are equivalent
 #### DCSR #### {#dcsr_format_equiv}
 
 ```json
-"format": {
-  "subformat": {
-    "level": "sparse",
+"custom": {
+  "level": {
+    "level_desc": "sparse",
     "rank": 1,
-    "subformat": {
-      "level": "sparse",
+    "level": {
+      "level_desc": "sparse",
       "rank": 1,
-      "subformat": {
-        "level": "element",
+      "level": {
+        "level_desc": "element",
       }
     }
   }
@@ -512,16 +513,16 @@ The following formats are equivalent
 #### DCSC #### {#dcsc_format_equiv}
 
 ```json
-"format": {
+"custom": {
   "transpose": [1, 0],
-  "subformat": {
-    "level": "sparse",
+  "level": {
+    "level_desc": "sparse",
     "rank": 1,
-    "subformat": {
-      "level": "sparse",
+    "level": {
+      "level_desc": "sparse",
       "rank": 1,
-      "subformat": {
-        "level": "element",
+      "level": {
+        "level_desc": "element",
       }
     }
   }
@@ -531,12 +532,12 @@ The following formats are equivalent
 #### COOR #### {#coor_format_equiv}
 
 ```json
-"format": {
-  "subformat": {
-    "level": "sparse",
+"custom": {
+  "level": {
+    "level_desc": "sparse",
     "rank": 2,
-    "subformat": {
-      "level": "element",
+    "level": {
+      "level_desc": "element",
     }
   }
 }
@@ -547,13 +548,13 @@ The following formats are equivalent
 Column-wise Coordinate format
 
 ```json
-"format": {
+"custom": {
   "transpose": [1, 0],
-  "subformat": {
-    "level": "sparse",
+  "level": {
+    "level_desc": "sparse",
     "rank": 2,
-    "subformat": {
-      "level": "element",
+    "level": {
+      "level_desc": "element",
     }
   }
 }