On "entity-linked file collections" #90
Description
Relates to bids-standard/bids-specification#2155.
The BIDS Specification has the concept of "entity-linked file collections". These are files that:
... are related to each other based on a repetitive acquisition of sequential data by changing acquisition parameters one (or multiple) at a time or by being inherent components of the same data.
It is suggested that entities "echo", "flip", "inv", "mt" and "part" may be the distinguishing factors between files that are a part of a collection. It is also defined that all files must possess the same suffix to be considered a part of an ELFC.
The way I have tended to think of an ELFC is that it is a group of files that "bear a much stronger
relationship to one another than to the rest of the dataset", but for which it is not possible to encode all such data into a single data file given the limitations of the data file format.
I have multiple criticisms of the way in which this concept is currently defined:
-
These kinds of relationships can intersect, covering multiple distinct entities or potentially even files with different suffixes. The first example proving the point is complex multi-echo data (see "
ipmultielfcv2" in New datasets for demonstrating / validating the Inheritance principle bids-examples#504): it is difficult to simultaneously claim that the two complex parts of one echo are an ELFC, and that the data across echoes are also an ELFC. -
It explicitly excludes data files with different suffixes from being part of an ELFC. Such data can however be highly prevalent, particularly when it comes to scanner-generated derivatives. Consider eg. the various files produced by MP2RAGE, DWI derivatives (New dataset "dwi_deriv" for scanner-generated DWI derivatives bids-examples#452), any kind of relaxometry where the scanner generates parameter estimates. The inability to explicitly establish these as "collections" may arise from the stipulation that key-value metadata files must contain a suffix, whereas if this was not the case (Suffixes mandatory for data but optional for metadata? #75) then the mutual information / acquisition source of such data could be more explicit.
Indeed prior discussion on suffixes RE "model fit parameters" vs. "model-derived parameters" (too lazy to link) were about establishing a strong ELFC among the former separate from the latter. In reality, MFP are more strongly linked to one another than they are to MDP, but they are not entirely unlinked to MDP.
-
It sets a false dichotomy between "these are an ELFC" versus "these are not". To me, this is a continuous spectrum: different data files may have stronger or weaker relationships with one another, depending on whether they come from the same subject / session / modality / sequence execution.
Whether or not any particular relationship between such data should be consequential for processing will depend on the particular processing to take place.
There is an alternative way to conceptualise these kinds of relationships. These relationships, and other comparable relationships between data files, will typically share a lot of metadata, with only subtle differences (eg. having units for the phase part, or different echo times for multi-echo data). The extent of such shared metadata can become highly visible with utilisation of the Inheritance Principle (technically it's present in the dataset without IP exploitation, it's just less obvious). A metadata file that expresses a large volume of mutual metadata between two or more data files therefore encodes these relationships in both strength (data files for which almost all of the metadata is shared form a "strong" ELFC) and which entities the relationship is defined across.
Having made an attempt to get my stream of consciousness onto screen, the question remains of what might be done for BIDS 2.0.
What I'm spitballing here is:
A metadata file inherited by multiple data files defines an Entity-Linked File Collection.
This can't be achieved in BIDS 1.x because the Inheritance Principle is under-powered.
Whether that thought actually progresses BIDS 2.0 in any way, regarding the Inheritance Principle or otherwise, I've no idea.
Thoughts welcome.