New datasets for Inheritance Principle validation

[Lestropie]

This content is being posted as a draft Pull Request on a forked repository
in order to demonstrate the volume of changes generated
during the OHBM 2025 Hackathon.
There is further work to be done in augmenting / adding more datasets
before it can be submitted as a Pull Request to the bids-examples repository
within the bids-standard organisation:

• Add key-value metadata to more datasets

  Particularly for those complex examples intended to demonstrate the correct order of metadata association,
  it would be advantageous to add metadata fields to the JSONs
  (even if some of those examples were first documeted elsewhere without any explicit metadata)
  to make sure that metadata precedence rules are being upheld.
  Ensure that for any datasets where such data is added, the exported override information is also consistent with manual expectation.

• Add override metadata to applicable datasets

  In datasets that explicitly demonstrate the effects of key-value overloading,
  manually generate JSON files containing the expected overloads,
  then make sure that the output of the -o option matches those expectations.

• Expand list of datasets

  The set of example datasets constructed thus far focus on the handling of JSON key-value metadata.
  JSON key-value files are however not the only form of metadata associated with data files in BIDS.
  Additional datasets should be constructed to demonstrate validate functionality such as:

  • Multiple applicable metadata files of a type other than .json

    (eg. bvec/bval files present both at BIDS root and as sidecars)

  • Metadata files of all different metadata extensions all present in a single dataset.

  • A metadata file that would be deemed applicable to some data file
    based exclusively on the entities & suffix of its file name,
    but that is ineligible for inheritance based on its location in the file system.

  • More realistic example of complex inheritance

    An appealing choice here is complex multi-echo fMRI data:
    Most metadata fields will be identical across all files;
    however "EchoTime" will vary between "_echo-#" files,
    and "Units" will vary between "_part-mag" and "_part-phase".

  • Tied inheritance for metadata type where only a unique nearest can be selected.

    I'm thinking of a scenario where a DWI does not have sidecar .bvec / .bval,
    there are two applicable .bvec / .bval pairs via inheritance,
    but they are in the same directory and have the same number of entities
    and therefore cannot be disambiguated.
    (The case where they differ in entity count and therefore can be disambiguated
    should also be included).

• Use .bids-validator-config.json / .SKIP_VALIDATION to stop the BIDS Validator from complaining
  about presence of unrecognised metadata fields / illegitimate filesystem structures.

  While the examples could technically be modified to use valid BIDS fields,
  in some of these circumstances the metadata key names are used to communicate purpose
  in the context of Inheritance Principle demonstration / validation.
  The downside of this however is that attempting to run the BIDS validator on these datasets,
  including through the run_tests.sh command, will fail;
  unless this file can be specified in such a way that these Inheritance Principle-specific datasets
  don't otherwise upset the validator.

[Lestropie]

One possible concern here is the shear number of new datasets being proposed for addition. If the relevant tool under investigation were to undergo the requisite augmentations to be able to process each subject directory individually, then there would likely be several "datasets" that could be merged together. It however needs to be possible to process each scenario individually to ensure comprehensiveness of testing given that many tests involve checking for non-conformity.

[Lestropie]

Should run both pybids and validator against these new datasets, modifying BIDSVersion to simulate all versions of the specification. Any discrepancies between their outcomes and the expectations as defined in the test site here should be reported forward.

[Lestropie]

Deprecated, now posted as PR to origin: bids-standard#504