Discuss BEP021 derivatives for electrophys

[robertoostenveld]

This continues the issue started here bids-standard/bids-specification#733. This BEP021 is a better place to continue the discussion, as we can also use projects and share files here which would not fit under bids-specification directly.

[robertoostenveld]

On 12 February 2021 we had a Zoom call to discuss the progress on BEP021 for which the draft is on http://bids.neuroimaging.io/bep021 on google docs. Some of the people that attended were @arnodelorme, @dorahermes, @guiomar, @jasmainak, @agramfort, but do not know the github handle of all. Please help to attend the others with a github presence by mentioning them here.

[robertoostenveld]

@jasmainak mentioned

The example dataset:

bids-standard/bids-examples#171
bids-standard/bids-examples#161

I haven't looked too closely but I see that it was merged by Robert :) Perhaps a dataset to use for discussion?

[robertoostenveld]

The pull requests bids-standard/bids-examples#171 and bids-standard/bids-examples#161 refer to “derivatives”, but looking at the dataset at https://github.com/bids-standard/bids-examples/tree/master/eeg_face13 there is nothing that sets it apart from a regular raw BIDS EEG dataset.

An example dataset that is a proper derivatives is https://github.com/bids-standard/bids-examples/tree/master/ds000001-fmriprep. Note, however, that there are quite some files in that derived dataset that have to be ignored as they are not standardized (yet).

Just some general pointers that relate to some topics we discussed:

On https://bids-specification.readthedocs.io/en/stable/03-modality-agnostic-files.html#derived-dataset-and-pipeline-description it is specified that a derived dataset is also a BIDS dataset. The derived dataset has some required and some recommended extra elements in the dataset_description. When you prepare an example/draft derived ieeg/eeg/meg dataset, please keep these in mind.

Also, on https://bids-specification.readthedocs.io/en/stable/05-derivatives/01-introduction.html it states "Derivatives are outputs of common processing pipelines, capturing data and meta-data sufficient for a researcher to understand and (critically) reuse those outputs in subsequent processing.” That lines up with what Scott and I were saying that a (derived) dataset should be (re)usable in its own right.

[robertoostenveld]

ping @adam2392 @hoechenberger

[robertoostenveld]

@guimar wrote:

Hi! I copy here the email too :)

Thanks everyone for the meeting today!

We thought that for the next meeting some of us could present some practical examples so we can have a starting point to discuss.
Preferably we could share the datasets in advance so everyone that has time can go over them and focus the discussion in the more problematic things.

Here's the link to the common derivatives specs merged in 1.4.0:
https://bids-specification.readthedocs.io/en/stable/05-derivatives/01-introduction.html
There, "derivative" is defined as:

Derivatives are outputs of common processing pipelines, capturing data and meta-data sufficient for a researcher to understand and (critically) reuse those outputs in subsequent processing.

In line with what we have discussed.

Also note that there are some metadata fields to point to source data:
https://bids-specification.readthedocs.io/en/stable/05-derivatives/02-common-data-types.html

It's important to distinguish between two types of derivatives:

pre-processed (in essence they are similar to source data - datatype remains unchanged)
and processed (they are substantially different to source data - datatype changed)
So I think pre-processed derivatives can be easily accessible at this point (e.g. detrending, filtering, downsampling, re-referencing, etc).

In this logic I don't know where annotations may exactly fall. But they are also an important step (and I think that can be also interesting for other modalities and extensions).

Talk very soon!

[robertoostenveld]

and @sappelhoff wrote that

As far as I heard, @smakeig (Scott Makeig) @nucleuscub (Ramon Martinez-Cancino) also attended

[arnodelorme]

Thank you, Robert, for this summary. We will generate test datasets as discussed and then we should reconvene.

[guiomar]

Hi @sappelhoff @robertoostenveld @arnodelorme, @dorahermes, @jasmainak, @agramfort @hoechenberger @smakeig @nucleuscub @CPernet !

I want to retake the effort on ephys derivatives.

Having a new look at the document, I see there are 3 main blocks:

1. Annotations
2. Preprocessing (derivatives that doesn't change datatype)
3. Procesing (new datatype)

I would incline to divide the work into independent lines, to avoid getting stuck due to the big amount of work ahead.
And I would start by dealing with: 2) pre-processing.
We have almost all the details needed to takle this one. I'm preparing some examples.

Would you like to meet and move this part forward?
https://www.when2meet.com/?13797922-eLdhm

[hoechenberger]

And I would start by dealing with: 2) pre-processing.

Preprocessing sounds good to me, I'm not sure about the point in the parens though ("derivatives that doesn't change datatype"). To me, preprocessing also includes epoching the data, which will create new data types too. But I'm also happy to just limit the next discussion to continuous data only (Maxwell-filtering, frequency filtering, …)

I will share some datasets we process using our BIDS pipeline shortly so you all can see how we're currently approaching things.

Cheers,
Richard

cc @agramfort

[guiomar]

Thanks @hoechenberger !!

I think this definition comes from the generic derivatives specification, but I'm not able to find it anymore.
Still it makes sense to differenciate between the two at this first approach, until we agree on the data format to be used when the datatype from the raw source is changed, to avoid getting blocked by this decision.
The annotations part is also more prone to debate. So let's focus on the more objective and straight forward!

This sounds awesome Richard! Thanks!

[adam2392]

I have read over the BEP021 derivatives and also added my availability, although I'll be in California at the time, so could possibly be hard to overlap.

I also utilized the "annotations" derivatives framework to create a dataset of event markings found in the iEEG via an automated algorithm. Specifically, these are "high-frequency oscillation" (HFO) onset/duration/channels stored as a tsv file. It works well for my use-case, but the tsv file does "explode" in length. Dataset is in dropbox link.

https://www.dropbox.com/sh/5ih5ay9fvo3q12s/AADBY5eDc_SmszHGyC3Mn6QJa?dl=0

[jasmainak]

I'm hammered that week. Unlikely to be able to join :(

I agree about the "explosion" issue that @adam2392 pointed out. I think I've seen it before. Probably it might help to limit the vocabulary of annotations so it is machine-readable not just machine-writeable.

[guiomar]

Hi @adam2392! Awesome! We can review the annotations with your example as well :)

[guiomar]

It seems the most preferred day to meet is Wed 15th Dec from 5 to 6pm CET.
I'll send you calendar invites with hangout link :)