[ENH] microelectrode electrophysiology specification (BEP032) #1705

yarikoptic · 2024-02-14T15:24:13Z

Replaces #1352 submitted from a fork outside of bids-specification.

Add specification for microelectrode electrohpysiology datasets based on the BEP032 proposal. old google doc

Note

We meet regularly and everyone is welcome

Next meeting: insert date on URL to join

Communication channel: https://framalistes.org/sympa/info/neuroscience-data-structure

Tip

HTML preview of this BEP

To use this WiP schema on sample datasets, see https://deno.land/x/[email protected]#modifying-and-building-a-new-schema on how to use "stock" bids-validator with a custom schema. (attn @TheChymera)

DONEs

TODOs

Issues this PR would likely to address

Fixes run node tests in github actions legacy-validator#1375

Issues to see being addressed while working on this BEP (likely to move above) or not (moved below):

Other issues which relate but not in scope here and provided for reference/backreference

Spreadsheet with correspondence to ProbeInterface: https://docs.google.com/spreadsheets/d/1O0bZzD-n4MjR68r1GlcH3d2JLXBLAU1PfsDceD3IPeo/edit?usp=sharing

… _acq-

src/schema/rules/files/raw/channels.yaml

TheChymera · 2024-03-06T16:39:46Z

src/schema/objects/columns.yaml

    for privacy purposes.
  type: number
  unit: year
+alpha_rotation:


I'd really recommend renaming the rotation axes to yaw, roll, and pitch (that would be the analogous angle order). There was no consensus either way on the google docs discussion. Someone said both are confusing, which I guess might be expected, but alpha, beta, gamma, are just more confusing...

I agree -- very adhoc. Let's discuss in google doc

https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

Could you chime in? I think the other guy commenting might be amenable to accepting this as well.

I agree that both are confusing, but at least alpha, beta and gamma are SO confusing that everyone realizes that additional specification is needed to define them properly. With roll, yaw and pitch it seems at first that all is clear, until you have a number of different people go through different use cases. See the more challenging examples that I posted on the google doc under https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

@robertoostenveld sorry, I only now saw the update, reply there. I don't think confusion on purpose is good in this case, because the documentation is very eclectic so we might be sending people down rabbit holes. Wiki, where people will invariably go first, does a particularly poor job explaining both euler and TB angles for the casual non-mathematics-versed user. The only thing that wiki has going for it here are the aircraft animations on the TB page. Yaw, Pitch, Roll, will be intuited correctly as long as we specify the starting postion. That we can do (1) as (I think, it's pretty vague) is currently proposed, aligning the implant with the world coordinate system (meaning most implants will be at yaw 0 pitch -90 roll 0) or (2) relative to the implantation stereotax normal (meaning most implants will be at 0 0 0).

For comparison, Euler commonly has the normal pointing up so most implants will be.... 0 -180 0 🤔

in the last BEP032 meeting we discussed further and wanted to follow this approach now (based on the Allen Inst. standard and IBL standard):

Assumption: x,y,z is posterior, ventral, right (unit needs to be specified).

Translational origin (0,0,0) needs to be defined (typically Bregma for rodents).

Rotational origin (0,0,0) is the probe facing up with the tip facing forward. Rotations are all clockwise in degrees and around the tip. For multi-shank probes, the “tip” of the probe is defined as the end of the left shank if you are looking at the electrodes.

yaw: clockwise when looking down

pitch: In the direction of the electrode face

roll: clockwise when looking down at the probe

The depth (unit needs to be specified) is a translation in the direction that the tip is pointing.

We need to add a “probe_model”, which references probeinterface_library

versioning of probe library files (raised here)

annotation of “tip” position (raised here)

NOTE: We need to change the electrode x,y,z.
X,y,z in BIDS refers to location in brain, not on probe.

src/schema/objects/columns.yaml

TheChymera · 2024-03-12T01:33:27Z

Also relevant if you'd like to comment. → https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAABIzHGpUU

TheChymera · 2024-03-12T01:57:39Z

Also I forgot to link to this when I wrote it → https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAABIGPAMOw

src/schema/rules/files/raw/ephys.yaml

…t was missing

src/schema/objects/datatypes.yaml

src/schema/objects/modalities.yaml

src/schema/objects/suffixes.yaml

…ication into bep032

bendichter · 2025-10-07T22:59:43Z

src/schema/objects/columns.yaml

+    Point of the probe that is described by the probe coordinates and
+    on which the yaw, pitch, and roll rotations are applied.


Suggested change

Point of the probe that is described by the probe coordinates and

on which the yaw, pitch, and roll rotations are applied.

Point of the skull that acts as the origin of the surgical coordinate system. If AP, DV, and ML are present and this field is omitted, it as assumed to be "bregma."

reviewing where/how coordinate_reference_point, it is consistent with existing wording here is that it is intended to describe reference point for electrode positions ON the probe. And you point to this name being misleading. In the text https://github.com/bids-standard/bids-specification/pull/1705/files#r2440584806 we proposed to add anatomical_reference_point to make it clearer.

And overall, with probes geometries to be described via probeinterface library specs, we might want to get rid of coordinate_reference_point completely??

src/schema/objects/entities.yaml

… BIDS conventions

ree-gupta · 2025-10-14T12:18:17Z

I think any of these is probably fine, ultimately.

If I understand it properly, the idea is for the files in /probes/ to be (in principle) uploadable to a remote store as-is and removed from the dataset. If that's the case, then keeping them as standalone documents and not nested inside a probes.json makes more sense to me.

In terms of connecting them to your probes.tsv file through JSON, you can still use TermURL along with BIDS URIs:
"model": {
    "Levels": {
        "neuronexus1": {
            "Description": "A 1-shank probe",
            "TermURL": "https://raw.githubusercontent.com/SpikeInterface/probeinterface_library/refs/heads/main/neuronexus/A1x32-Poly3-10mm-50-177/A1x32-Poly3-10mm-50-177.json"
        },
        "neuronexus2": {
            "Description": "new experimental probe",
            "TermURL": "bids::probes/neuronexus2.json"
        },
    }
}
bids:: prefix means the file is inside the current dataset, and probes/neuronexus2.json is the root-relative path to the file.

Does that fill in the remaining gaps?

This looks good to me too! I had just a small question about how should the probes directory at root be correctly recognized by the schema/validator? Would you suggest adding it to src.schema.objects.files.yaml? like how stimuli directory is specified?

…a little

yarikoptic · 2025-10-15T14:21:03Z

bids:: prefix means the file is inside the current dataset, and probes/neuronexus2.json is the root-relative path to the file.

@effigies so you suggest to establish yet another folder with structured data but which will not be formalized by BIDS in terms of schema/structure under it? or should we strive to come up with some consistent with BIDS overall and/or like e.g. BEP044 approaches to formalize under stimuli/ - https://bids.neuroimaging.io/extensions/beps/bep_044.html and consider making it probes/probe-<label>_probeinterface.json so we

have it explicitely stating to be in probeinterface format
have probe-<label> to be able to say probe_id (probe-<label>) eventually (and <label> here I think matches the "name")

effigies · 2025-10-15T15:00:29Z

@ree-gupta Apologies for the delay. I had unsubscribed so that I would see mentions but not other activity on this PR, so I did not see your response. Please @ me in the future, as I will be doing so again after today.

I had just a small question about how should the probes directory at root be correctly recognized by the schema/validator? Would you suggest adding it to src.schema.objects.files.yaml? like how stimuli directory is specified?

Yes, that or phenotype are the most direct parallels.

@yarikoptic

so you suggest to establish yet another folder with structured data but which will not be formalized by BIDS in terms of schema/structure under it?

The probes are a separate standard with their own contents and file-naming conventions. The intended progression appears to be:

Stick new probe definitions into /probes and refer to them with bids::probes/*.
Upload probes as-is to the global probe namespace.
Replace the bids::probes/ prefix with a full URL prefix.

Given that they already have a BIDS-incompatible naming convention (there are hyphens in the names), it does not seem particularly appealing to me to put an effort to come up with a mapping. Any tool that wants to find the probe definitions is going to need to resolve a URI in any case, so querying using BIDS entities doesn't seem like it adds much utility.

We can probably check that the file name follows probe interface conventions as a schema check, if not a normal filename rule.

That said, if making the directory more BIDSy feels like a good use of people's time, I'm not here to stop you.

yarikoptic

some recommendations

src/modality-specific-files/microelectrode-electrophysiology.md

yarikoptic · 2025-10-15T15:11:28Z

src/modality-specific-files/microelectrode-electrophysiology.md

+
+The Microelectrode Electrophysiology modality encompasses recordings made with micrometer-scale electrodes,
+distinguishing it from related BIDS modalities (EEG, MEG, iEEG) that use larger electrodes.
+This modality is primarily used in animal research.


TODO here -- add an image describing relation between probes electrodes and channels.

smth like

based on what is in the spikeinterfaces docs

src/modality-specific-files/microelectrode-electrophysiology.md

yarikoptic · 2025-10-15T15:23:27Z

src/modality-specific-files/microelectrode-electrophysiology.md

+**Intracellular electrophysiology example:**
+
+```tsv
+name	type	units	sampling_frequency	recording_mode	gain	ground	status


in addition to recording_mode we need to describe how clamping is done following e.g. types described in OpenMINDS at https://github.com/openMetadataInitiative/openMINDS_instances/tree/main/instances/latest/terminologies/patchClampVariation which we can name patch_variation and describe permitted values. (might need extra value for jaxtracellulular patching... loose patch recording according to google search by @apdavison)

I would also vote to define all the values allowed explicitly so could be validated.

yarikoptic · 2025-10-15T15:25:55Z

src/modality-specific-files/microelectrode-electrophysiology.md

+### Recommended Channel Type Values
+
+For the `type` column we recommend to use the following terms (adapted from [iEEG](intracranial-electroencephalography.md#channels-description-_channelstsv))
+


TODO: make a macro from the schema! Tireless @ree-gupta promised to do that in a separate PR and adopt here to avoid hardcoding

yarikoptic · 2025-10-17T14:28:05Z

src/modality-specific-files/microelectrode-electrophysiology.md

+**Extracellular electrophysiology example:**
+
+```tsv
+probe_name	type	AP	ML	DV	AP_angle	ML_angle	rotation_angle	hemisphere	manufacturer	device_serial_number	electrode_count	width	height	depth	dimension_unit	coordinate_reference_point	associated_brain_region	associated_brain_region_id	reference_atlas	material


note: at our meeting Oct 15th 2025 we briefly touched on a possible topic of introducing probe-<label> entity! It would reflect that actual recordings directly from hardware do ATM store each probe separately and it could be that there are different manufacturers/types of probes. That would then mandate pretty much to have probe_id (probe-<label>) as the leading column in the *probes.tsv file (which would exist then to summarize across probes, potentially even may be benefitting from inheritance principle for common metadata?).

Here probe_name came from analogy to name of electrodes.tsv inherited from EEG. But the question remains whether we might want to standardize this right away. WDYT @effigies ? Should we may be right away introduce probe entity?

Sorry, I don't think I have enough to go on from this comment and the immediately surrounding text. I would probably need to read the BEP in some detail to form an opinion.

yarikoptic · 2025-10-17T16:15:05Z

@effigies

so you suggest to establish yet another folder with structured data but which will not be formalized by BIDS in terms of schema/structure under it?

The probes are a separate standard with their own contents and file-naming conventions.

I do not think there is any particular filename convention outside of their internal library. Here we are to provide files in an alien to that library location, so then tools could load using their "standard" https://probeinterface.readthedocs.io/en/main/api.html#probeinterface.io.read_probeinterface function (correct @bendichter ?) . Then the question of filename sanitization is also not a problem, we could e.g. replace all - with now allowed +.

yarikoptic · 2025-10-17T16:32:14Z

src/modality-specific-files/microelectrode-electrophysiology.md

+### Default probe-relative coordinate systems
+
+When no [`space-<label>`](../appendices/entities.md#space) entity is specified in the filename,
+electrode positions in `*_electrodes.tsv` are **probe-relative coordinates**:


chatting with @CodyCBakerPhD who raises a good point (I think) that if we are to describe probe geometries , either standard, or custom under /probes, then why do we want this yet another way to describe coordinates for the electrodes on the probes, and in the _probes.tsv table in particular? they would not be different for the same probe manufacturer/model, so why to repeat across the files? if very custom, a file under /probes could still be used to describe it.

yarikoptic · 2025-10-17T16:42:06Z

src/modality-specific-files/microelectrode-electrophysiology.md

+
+**Lambda**: the meeting point of the sagittal suture (between left and right parietal bones) and the lambdoid suture (between parietal and occipital bones).
+
+Both points serve as standard reference points for stereotaxic coordinates in neuroscience research. `(0,0,0)` is assumed to be Bregma when working with rodents. It may optionally be defined differently using `coordinate_reference_point`, and MUST be defined for other species.


coordinate_reference_point, so far, describes ref point for electrodes on the probe. Here it is used for a different purpose, as @bendichter trying also to "fixup" in columns.yaml. Should we decide to allow to override here, we need to create a new column name, e.g.

Suggested change

Both points serve as standard reference points for stereotaxic coordinates in neuroscience research. `(0,0,0)` is assumed to be Bregma when working with rodents. It may optionally be defined differently using `coordinate_reference_point`, and MUST be defined for other species.

Both points serve as standard reference points for stereotaxic coordinates in neuroscience research. `(0,0,0)` is assumed to be Bregma when working with rodents. It may optionally be defined differently using `anatomical_reference_point`, and MUST be defined for other species.

Co-authored-by: Ben Dichter <[email protected]>

Add first BEP032 objects

7ec5d5e

yarikoptic requested review from erdalkaraca and tsalo as code owners February 14, 2024 15:24

yarikoptic mentioned this pull request Feb 14, 2024

[ENH] Add BEP032 (microelectrode electrophysiology) specification #1352

Closed

3 tasks

Add ephys modality, start adding _channels but ran into a question on…

2f393ca

… _acq-

yarikoptic commented Feb 14, 2024

View reviewed changes

src/schema/rules/files/raw/channels.yaml Show resolved Hide resolved

tsalo added the BEP label Feb 15, 2024

yarikoptic mentioned this pull request Feb 29, 2024

Allow for using an arbitrary suffix and/or arbitrary entity bids-standard/bids-2-devel#63

Open

yarikoptic and others added 2 commits March 5, 2024 19:56

Add _acq- to ephys, add _channels and _coords

648781a

Draft for probes TSV file

d3c9996

TheChymera reviewed Mar 6, 2024

View reviewed changes

src/schema/objects/columns.yaml Show resolved Hide resolved

TheChymera closed this Mar 7, 2024

TheChymera deleted the bep032 branch March 7, 2024 22:13

TheChymera restored the bep032 branch March 7, 2024 22:15

TheChymera reopened this Mar 11, 2024

Fix preferably typo

08c92cd

yarikoptic mentioned this pull request Mar 15, 2024

[SCHEMA] Remove stale test comment -- test no longer fails! #1727

Merged

yarikoptic commented Mar 15, 2024

View reviewed changes

src/schema/rules/files/raw/ephys.yaml Outdated Show resolved Hide resolved

robertoostenveld mentioned this pull request Mar 21, 2024

Draft examples for BEP032 on animal electrophysiology bids-standard/bids-examples#430

Draft

yarikoptic mentioned this pull request Apr 2, 2024

Automatic generation of extra-fields template forms from yaml-based schema (BIDS standard) elabftw/elabftw#5023

Closed

Comment out not yet defined iephys and add ephys to datatypes where i…

e2030ac

…t was missing

yarikoptic commented Apr 17, 2024

View reviewed changes

src/schema/objects/datatypes.yaml Outdated Show resolved Hide resolved

dorahermes reviewed Apr 17, 2024

View reviewed changes

src/schema/objects/modalities.yaml Outdated Show resolved Hide resolved

dorahermes reviewed Apr 17, 2024

View reviewed changes

src/schema/objects/suffixes.yaml Outdated Show resolved Hide resolved

TheChymera force-pushed the bep032 branch from cde80a6 to e2030ac Compare April 17, 2024 21:26

Added probes file

53f1087

Remi-Gau changed the title ~~[ENH] Add BEP032 (microelectrode electrophysiology) specification~~ [ENH] microelectrode electrophysiology specification (BEP032) Apr 18, 2024

bendichter added 3 commits October 7, 2025 13:21

[MISC] Replace image syntax with HTML for anatomical reference diagrams

551f5de

Merge branch 'bep032' of https://github.com/bids-standard/bids-specif…

40e7a5d

…ication into bep032

Fix formatting in example *_electrodes.tsv for improved readability

cc7fe98

bendichter reviewed Oct 7, 2025

View reviewed changes

Apply suggestion from @bendichter

ad41192

bendichter reviewed Oct 7, 2025

View reviewed changes

src/schema/objects/entities.yaml Outdated Show resolved Hide resolved

ree-gupta and others added 4 commits October 14, 2025 13:45

Remove channel_id def and use name__channels same as ieeg

da05a28

Update microephys electrode.tsv to match ieeg

d9266b7

Changed probe_id to probe_name to avoid use of the concept ID against…

4fc8ec8

… BIDS conventions

Merge branch 'bids-standard:bep032' into bep032

8a933ec

ree-gupta and others added 11 commits October 14, 2025 14:34

Add missing probe_height def and rewrite desc to match current markdown

79f1059

Fixed minor typo failing schema validation

40993d2

Fix remark linting issues

41b301d

Added and modified texts for coordsystem requirements

4c95199

Added icephys and ecephys distinction and restructured first section …

05d19bf

…a little

Fixed initial_column to match AP, ML, DV

a127f57

Update mock TSV and JSON and fix heading levels to match ieeg

4c776e5

Updated toy data section and links to example to match current spec

d428dc8

Minor formatting and style issues

73d3975

Fix mock data tsv and JSON

7fb3a28

Merge branch 'master' into bep032

8b39a2d

yarikoptic commented Oct 15, 2025

View reviewed changes

yarikoptic commented Oct 17, 2025

View reviewed changes

Apply various wording suggestions from code review

1cfa706

Co-authored-by: Ben Dichter <[email protected]>

		Point of the probe that is described by the probe coordinates and
		on which the yaw, pitch, and roll rotations are applied.

	Point of the probe that is described by the probe coordinates and
	on which the yaw, pitch, and roll rotations are applied.
	Point of the skull that acts as the origin of the surgical coordinate system. If AP, DV, and ML are present and this field is omitted, it as assumed to be "bregma."

		### Recommended Channel Type Values

		For the `type` column we recommend to use the following terms (adapted from [iEEG](intracranial-electroencephalography.md#channels-description-_channelstsv))


		Lambda: the meeting point of the sagittal suture (between left and right parietal bones) and the lambdoid suture (between parietal and occipital bones).

		Both points serve as standard reference points for stereotaxic coordinates in neuroscience research. `(0,0,0)` is assumed to be Bregma when working with rodents. It may optionally be defined differently using `coordinate_reference_point`, and MUST be defined for other species.

[ENH] microelectrode electrophysiology specification (BEP032) #1705

Are you sure you want to change the base?

[ENH] microelectrode electrophysiology specification (BEP032) #1705

Uh oh!

Conversation

yarikoptic commented Feb 14, 2024 • edited by CodyCBakerPhD Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODOs

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TheChymera Apr 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

TheChymera commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TheChymera commented Mar 12, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bendichter Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ree-gupta commented Oct 14, 2025

Uh oh!

yarikoptic commented Oct 15, 2025

Uh oh!

effigies commented Oct 15, 2025

Uh oh!

yarikoptic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yarikoptic Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yarikoptic commented Oct 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

14 participants

yarikoptic commented Feb 14, 2024 •

edited by CodyCBakerPhD

Loading

TheChymera Apr 17, 2024 •

edited

Loading

TheChymera commented Mar 12, 2024 •

edited

Loading

bendichter Oct 7, 2025 •

edited

Loading

yarikoptic Oct 17, 2025 •

edited

Loading