Converting OpenEphys binary data to NWB format with multiple streams

[RuthKassahun]

Hi all,

I'm currently working on converting OpenEphys binary data into NWB format. Initially, I tried to use the script below.

However, I encountered an issue due to having multiple streams of data in my folder structure. The code I used required specifying a specific stream.
Is there a workaround or an alternative approach that you might suggest for converting this data into NWB format while handling multiple streams? Any advice would be greatly appreciated.

Thank you!

from dateutil import tz
from pathlib import Path
from neuroconv import ConverterPipe

from neuroconv.datainterfaces import OpenEphysRecordingInterface ,VideoInterface


folder_path = './2021-04-09_20-36-07'
interface_ap = OpenEphysRecordingInterface(folder_path=folder_path,stream_name='Neuropix-PXI-100.0', verbose=False)
interface_lfp = OpenEphysRecordingInterface(folder_path=folder_path,stream_name='Neuropix-PXI-100.1', verbose=False)
interface_nidaq = OpenEphysRecordingInterface(folder_path=folder_path,stream_name='NI-DAQmx-104.0', verbose=False)

video_file_path = './video_record.mp4"
interface_video = VideoInterface(file_paths=[video_file_path], verbose=False)

converter = ConverterPipe(data_interfaces=[interface_ap, interface_lfp, interface_nidaq,interface_video], verbose=False)

metadata = converter.get_metadata()

folder_name = Path(folder_path).name

session_start_time = datetime.strptime(folder_name, "%Y-%m-%d_%H-%M-%S")
metadata["NWBFile"].update(session_start_time=session_start_time)

nwbfile_path = './allcombine.nwb'
converter.run_conversion(nwbfile_path=nwbfile_path,  metadata=metadata)

Here's how my folder structure looks:

2021-04-09_20-36-07/
├── video_record.mp4
├── settings.xml
└── experiment
    ├── recording
    │   ├──structure.oebin
    │   ├──continuous
    │   │   ├── Neuropix-PXI-100.0
    │   │         ├── continuous.dat
    │   │         ├── timestamps.npy
    │   │   ├── Neuropix-PXI-100.1
    │   │         ├── continuous.dat
    │   │         ├── timestamps.npy
    │   │   ├── NI-DAQmx-104.0
    │   │         ├── continuous.dat
    │   │         ├── timestamps.npy
    │   └── events
    │       ├── Message_Center-904.0
    │       │         ├── TEXT_group_1
    │       │               ├── channels.npy
    │       │               ├── text.npy
    │       │               ├──timestamps.npy
    │       ├──Neuropix-PXI-100.0
    │       │         ├── TTL_1
    │       │               ├── channel_states.npy
    │       │               ├── channels.npy
    │       │               ├── full_words
    │       │               ├──timestamps.npy
    │       └── NI-DAQmx-104.0
    │       │         ├── TTL_1
    │       │               ├── channel_states.npy
    │       │               ├── channels.npy
    │       │               ├── full_words
    │       │               ├──timestamps.npy
    

[oruebel]

@RuthKassahun I'm CC'ing @CodyCBakerPhD @bendichter since this questions is related to neuroconv

[RuthKassahun]

Thank you @oruebel

[bendichter]

@RuthKassahun this script looks correct. Is this not meeting your needs? Does it throw an error? If so, could you copy/paste your entire stack traceback?

[RuthKassahun]

@bendichter thank you for your response. The error I got is like this:

Metadata is valid!
conversion_options is valid!
Traceback (most recent call last):

  File ~\.conda\envs\neuroconv_environment\Lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File c:\users\gonzalez lab\documents\allstreamscombinedneuroconv.py:35
    converter.run_conversion(nwbfile_path=nwbfile_path,  metadata=metadata)

  File ~\Documents\neuroconv\neuroconv\src\neuroconv\nwbconverter.py:160 in run_conversion
    self.add_to_nwbfile(nwbfile_out, metadata, conversion_options)

  File ~\Documents\neuroconv\neuroconv\src\neuroconv\nwbconverter.py:114 in add_to_nwbfile
    data_interface.add_to_nwbfile(

  File ~\Documents\neuroconv\neuroconv\src\neuroconv\datainterfaces\ecephys\baserecordingextractorinterface.py:335 in add_to_nwbfile
    add_recording(

  File ~\Documents\neuroconv\neuroconv\src\neuroconv\tools\spikeinterface\spikeinterface.py:753 in add_recording
    add_electrical_series(

  File ~\Documents\neuroconv\neuroconv\src\neuroconv\tools\spikeinterface\spikeinterface.py:681 in add_electrical_series
    nwbfile.add_acquisition(es)

  File ~\.conda\envs\neuroconv_environment\Lib\site-packages\hdmf\utils.py:644 in func_call
    return func(args[0], **pargs)

  File ~\.conda\envs\neuroconv_environment\Lib\site-packages\pynwb\file.py:854 in add_acquisition
    self._add_acquisition_internal(nwbdata)

  File ~\.conda\envs\neuroconv_environment\Lib\site-packages\hdmf\utils.py:644 in func_call
    return func(args[0], **pargs)

  File ~\.conda\envs\neuroconv_environment\Lib\site-packages\hdmf\container.py:1031 in _func
    raise ValueError(msg)

ValueError: 'ElectricalSeries' already exists in NWBFile 'root'

[CodyCBakerPhD]

@RuthKassahun Looks like you need to specify different names for the multiple ElectricalSeries

This can be done via the es_key argument to each interface

Modified code would be something like

interface_ap = OpenEphysRecordingInterface(folder_path=folder_path, stream_name='Neuropix-PXI-100.0', verbose=False, es_key="ElectricalSeriesAP")
interface_lfp = OpenEphysRecordingInterface(folder_path=folder_path, stream_name='Neuropix-PXI-100.1', verbose=False, es_key="ElectricalSeriesLF")
interface_nidaq = OpenEphysRecordingInterface(folder_path=folder_path, stream_name='NI-DAQmx-104.0', verbose=False, es_key="ElectricalSeriesNIDQ")