Discordances in enforcement of Inheritance Principle #217
Description
Relates to bids-standard/bids-specification#2155.
In the construction of a large number of new example BIDS datasets for evaluating the compatibility between the Inheritance Principle and the surrounding ecosystem (bids-standard/bids-examples#504), I have identified particular motifs of dataset structures where the current outcomes of the validator do not match my own a priori expectations of what those outcomes should be.
To reproduce the discrepancies, see: https://github.com/Lestropie/IP-freely/blob/0.1.0/README.md?plain=1#L72-L80
1. Changes to permissible metadata file paths in 1.7.0
In #946 for 1.7.0, the Inheritance Principle was codified. The ruleset did however change ever-so-slightly, and this means that two datasets of equivalent content that differ only in the reported BIDSVersion should change whether they pass or fail validation.
1.1. Subject-specific file in root directory
In example dataset "ip112badmetapathe1", file "sub-01_T1w.json" is defined at the root of the dataset. This should fail for versions prior to 1.7.0, given:
Files for a particular participant can exist only at participant level directory, i.e
/dataset/sub-*[/ses-*]/sub-*_T1w.json.
1.7.0 however doesn't say such explicitly; rather it says:
- A metadata file MUST NOT have a filename that would be otherwise applicable to some data file based on rules 2.b and 2.c but is made inapplicable based on its location in the directory structure as per rule 2.a.
The fact that the metadata file contains entity "sub-01" means that it is inapplicable to any of the content of sub-02/, and the dataset therefore does not violate any written rule.
(Whether it should be in violation of a rule is a discussion for the BIDS specification repository)
1.2. Subject-specific file in session directory
In example dataset "ip170badrelpath", file "sub-01_T1w.json" is placed in directory sub-01/ses-01/anat/, despite the existence of file "sub-01/ses-02/anat/sub-01_ses-02_T1w.nii.gz". Pre-1.7.0, there wasn't anything technically precluding this. But from 1.7.0 onwards this should violate rule 3 quoted above. Neither validator correctly identifies this as a violation.
2. Exclusive-match, non-sidecar, data file - metadata file pairs
In example dataset "ipexclnonsc", there is one data file and one metadata file, and the latter is applicable to the former via the Inheritance Principle. But the two are not an exact match in terms of their entities. While this is highly unusual and unconventional, and should arguably result in a warning / error in future versions of the specification, I've not found anything in any existing version of the specification that explicitly precludes this.
3. Multiple non-JSON metadata files within a single parent directory
The schema-based validator appears to be uniquely capable of identifying cases where there are multiple applicable JSON metadata files at a single filesystem level. But the Inheritance Principle applies to non-JSON metadata also.
In example dataset "ipdwi003", there are two pairs of FSL .bvec/.bval diffusion gradient table files within directory sub-01/dwi/.
Any metadata file (.json, .bvec, .tsv, etc.) may be defined at any directory level, but no more than one applicable file may be defined at a given level (Example 1).
From version 1.7.0 onwards:
- There MUST NOT be multiple metadata files applicable to a data file at one level of the directory hierarchy.
By my reading, this example dataset should fail validation under both BIDS versions; however both validators deem this dataset compliant.
4. "Loose" metadata
Example dataset "iploosemeta" contains file sub-01/anat/sub-01_T2w.json despite there being no T2-weighted NIfTI image in the dataset. While it would seem sensible that this should be reported as an error, as it's likely that there is some problem with the dataset, I've failed to find any statement in the specification that explicitly precludes this.