Merge pull request #2096 from oesteban/enh/better-tsv-rendering

FIX+ENH: Beautify tabular data fences

Author: effigies

mkdocs.yml

@@ -101,15 +101,21 @@ extra:
 
 extra_javascript:
   - js/jquery-3.6.0.min.js
+extra_css:
+  - css/tsv.css
 markdown_extensions:
   - toc:
       anchorlink: true
+  - attr_list
   - pymdownx.superfences:
       preserve_tabs: true
       custom_fences:
         - name: tsv
           class: tsv
           format: !!python/name:bidsschematools.render.tsv.fence
+        - name: tsvgz
+          class: tsv
+          format: !!python/name:bidsschematools.render.tsv.fence
   - admonition
   - pymdownx.details
 plugins:

src/common-principles.md

@@ -505,18 +505,18 @@ TSV files MUST be in UTF-8 encoding.
 
 Example:
 
-```Text
-onset   duration    response_time   trial_type        trial_extra
-200     20.0        15.8            word              中国人
-240     5.0         17.34e-1        visual            n/a
+```tsv {linenums="1"}
+onset	duration	response_time	trial_type	trial_extra
+200	20.0	15.8	word	中国人
+240	5.0	17.34e-1	visual	n/a
 ```
 
 !!! warning "Attention"
 
     The TSV examples in this document (like the one above this note) are occasionally
-    formatted using space characters instead of tabs to improve human readability.
-    Directly copying and then pasting these examples from the specification
-    for use in new BIDS datasets can lead to errors and is discouraged.
+    formatted with the addition of the row indices as first column.
+    Those indices are presented for visual reference and
+    are not part of the tabular data file's content.
 
 Tabular files MAY be optionally accompanied by a simple data dictionary
 in the form of a JSON [object](https://www.json.org/json-en.html)
@@ -627,6 +627,13 @@ Rules for formatting plain-text tabular files apply to TSVGZ files with three ex
     (for example, FSL, or PNM), and to facilitate the support for other file formats
     in the future.
 
+The above example, if stored as a TSVGZ file would have the following decompressed content:
+
+```tsvgz {linenums="1"}
+200	20.0	15.8	word	中国人
+240	5.0	17.34e-1	visual	n/a
+```
+
 ### Key-value files (dictionaries)
 
 JavaScript Object Notation (JSON) files MUST be used for storing key-value

src/css/tsv.css

@@ -0,0 +1,58 @@
+.tsv-table {
+    font-feature-settings:"kern";
+    min-width: 100%;
+    border-collapse: collapse;
+    overflow: auto;
+    display: block;
+}
+
+.tsv-table th {
+    color: var(--md-code-fg-color);
+    font-family:var(--md-code-font-family);
+    font-weight: 800;
+    background-color: var(--md-code-bg-color);
+    border-top: 1px solid var(--md-default-fg-color--light);
+    font-size: 0.8em;
+    padding: 0 10px 0 10px;
+}
+
+.tsv-table.index tr th:first-child {
+    background-color: var(--md-text-bg-color);
+    border-top: 0;
+    user-select: none;
+}
+
+.tsv-table tr:last-child td {
+    border-bottom: 1px solid var(--md-default-fg-color--light);
+}
+
+.tsv-table tr:first-child td {
+    border-top: 1px solid var(--md-default-fg-color--light);
+}
+
+.tsv-table td {
+    font-family:var(--md-code-font-family);
+    font-size: 0.8em;
+    text-align: center;
+    vertical-align: middle;
+    color: var(--md-code-fg-color);
+    background-color: var(--md-code-bg-color);
+    padding: 0 10px 0 10px;
+}
+
+.tsv-table.index td:first-child {
+    color:var(--md-default-fg-color--light);
+    font-family:var(--md-text-font-family);
+    font-size: 0.7em;
+    background-color: var(--md-text-bg-color);
+    padding-right: 15px;
+    text-align: right;
+    min-width: 25px;
+    width: 35px;
+    border-top: 0;
+    user-select: none;
+}
+
+.tsv-table.index tr:last-child td:first-child {
+    border-bottom: 0;
+}

src/derivatives/imaging.md

@@ -524,21 +524,21 @@ and a guide for using macros can be found at
 
 An example, custom `dseg.tsv` that defines three labels:
 
-```Text
-index   name            abbreviation    color       mapping
-100     Gray Matter     GM              #ff53bb     1
-101     White Matter    WM              #2f8bbe     2
-102     Brainstem       BS              #36de72     11
+```tsv
+index	name	abbreviation	color	mapping
+100	Gray Matter	GM	#ff53bb	1
+101	White Matter	WM	#2f8bbe	2
+102	Brainstem	BS	#36de72	11
 ```
 
 The following example `dseg.tsv` defines regions that are not part of the
 standard BIDS labels:
 
-```Text
-index   name                abbreviation
-137     pars opercularis    IFGop
-138     pars triangularis   IFGtr
-139     pars orbitalis      IFGor
+```tsv
+index	name	abbreviation
+137	pars	opercularis	IFGop
+138	pars	triangularis	IFGtr
+139	pars	orbitalis	IFGor
 ```
 
 <!-- Link Definitions -->

src/modality-specific-files/microscopy.md

@@ -418,10 +418,10 @@ when applicable.
 Additional optional columns MAY be used to describe other subjects' attributes.
 
 `participants.tsv` example:
-```Text
-participant_id species strain strain_rrid
-sub-01 mus musculus C57BL/6J RRID:IMSR_JAX:000664
-sub-02 mus musculus C57BL/6J RRID:IMSR_JAX:000664
+```tsv
+participant_id	species	strain	strain_rrid
+sub-01	mus	musculus	C57BL/6J	RRID:IMSR_JAX:000664
+sub-02	mus	musculus	C57BL/6J	RRID:IMSR_JAX:000664
 ```
 
 `participants.json` example:

src/modality-specific-files/near-infrared-spectroscopy.md

@@ -294,7 +294,7 @@ Note that upper-case is REQUIRED.
 ### Example `*_channels.tsv`
 
 ```tsv
-Name	type	source	detector	wavelength_nominal	units
+name	type	source	detector	wavelength_nominal	units
 S1-D1	NIRSCWAMPLITUDE	A1	Fz	760	V
 S1-D1	NIRSCWAMPLITUDE	A1	Fz	850	V
 S1-D2	NIRSCWAMPLITUDE	A1	Cz	760	V

src/modality-specific-files/physiological-recordings.md

@@ -84,7 +84,7 @@ A guide for using macros can be found at
 
 (after decompression)
 
-```tsv
+```tsvgz {linenums="1"}
 34	110	0
 44	112	0
 23	100	1

src/modality-specific-files/positron-emission-tomography.md

@@ -318,22 +318,22 @@ If recorded once per participant, these data SHOULD be included in the
 
 For example:
 
-```Text
-participant_id body_weight
-sub-01 58
-sub-02 96
-sub-03 72
+```tsv
+participant_id	body_weight
+sub-01	58
+sub-02	96
+sub-03	72
 ```
 
 If multiple measurements are made, these data SHOULD be included in the
 [Sessions file](../modality-agnostic-files.md#sessions-file).
 
 For example:
 
-```Text
-session_id body_weight
-ses-01 58
-ses-02 59
+```tsv
+session_id	body_weight
+ses-01	58
+ses-02	59
 ```
 
 ## Blood recording data

src/modality-specific-files/task-events.md

@@ -206,11 +206,11 @@ A guide for using macros can be found at
 
 Example of the content of the TSV file:
 
-```Text
-onset duration  trial_type  identifier  database  response_time
-1.2 0.6 afraid  AF01AFAF  kdef  1.435
-5.6 0.6 angry AM01AFAN  kdef  1.739
-5.6 0.6 sad AF01ANSA  kdef  1.739
+```tsv
+onset	duration	trial_type	identifier	database	response_time
+1.2	0.6	afraid	AF01AFAF	kdef	1.435
+5.6	0.6	angry	AM01AFAN	kdef	1.739
+5.6	0.6	sad	AF01ANSA	kdef	1.739
 ```
 
 The `trial_type` and `identifier` columns from the `events.tsv` files might be described

tools/schemacode/src/bidsschematools/render/tsv.py

@@ -1,15 +1,54 @@
+from __future__ import annotations
+
 import io
+import re
 
 import pandas as pd
 from markdown_it import MarkdownIt
 from tabulate import tabulate
 
 
-def fence(source: str, language: str, css_class: str, options: dict, md, **kwargs) -> str:
+def fence(
+    source: str,
+    language: str,
+    css_class: str,
+    options: dict,
+    md,
+    classes: list[str],
+    id_value="",
+    attrs: dict | None = None,
+    **kwargs,
+) -> str:
+    attrs = attrs or {}
+    linenums = attrs.get("linenums", "0") == "1"
+
+    if language == "tsvgz" and "header" not in attrs:
+        attrs["noheader"] = True
+
+    classes[:0] = ["tsv-table", "index" if linenums else "noindex"]
+
     try:
-        df = pd.read_csv(io.StringIO(source), sep="\t", dtype=str, keep_default_na=False)
-        md_table = tabulate(df, headers="keys", tablefmt="github", showindex=False)  # type: ignore
+        df = pd.read_csv(
+            io.StringIO(source),
+            sep="\t",
+            dtype=str,
+            index_col=False,
+            keep_default_na=False,
+            header=None if "noheader" in attrs else "infer",
+        )
+        md_table = tabulate(
+            df,  # type: ignore
+            tablefmt="github",
+            showindex=linenums,
+            headers="keys",
+            numalign="right",
+        )
         html = MarkdownIt("commonmark").enable("table").render(md_table)
+        if "noheader" in attrs:
+            html = re.sub("<thead>.+</thead>", "", html, flags=re.DOTALL)
+
+        html = html.replace("<table>", f'<table class="{" ".join(classes)}">')
+
         # Remove newlines from HTML to prevent copy-paste from inserting spaces
         return html.replace("\n", "")
     except Exception: