Comments on stream implementation and its implications. (#970)

* Move constants into class so accessible for testing.

* Test of building NcsBlocks.

* Handle old files with truncated frequency in header and test.

* Change interface on parse versus build.

* Tests for PRE4 type and code corrections.

Tests for v5.5.1 still failing.

* Fix initializer, update loop vars.

* Add additional tests on v5.5.1 with 2 blocks

* Tests of block construction for incomplete blocks

* Fix up single block case.

* Remove unneeded classes. Clean up style.

* Use private dtype by new private name.

* Add test of side effects of read_ncs_files

* Use NcsBlocksFactory and logical or.

* Fix off by one in range for list. Comments.

* Use standard time calculation for last time of block.

* Remove test with tolerance over whole length. Fix microsPerSampUsed assignement.

Using a tolerance over a longer experiment is not sensitive enough to detect blocks where perhaps a large amount of samples are dropped and there is a small gap afterwards.

* Tests of raw io for incomplete records multiple block case.

* Update stop times to include time for samples in partially filled records.

* PEP and style cleanup. Corrected gap comment.

* Line shortening for PEP8.

* More PEP8 items.

* Correct column error in data tested.

Strange this hasn’t caused issues before with other file sets.

* Update comment on PRE4 file limitations.

* Tests for PRE4 file type in Cheetah v4.0.2.

* PEP8 indent corrections.

* Small PEP8 correctin.

* Proper time opened header for BML and use of frequency.

* Download files and activate tests for BML style files.

* record start and end times in NcsBlocks for verification later

* Code to handle digitallynx with cheetah v1

Like files from Dr. Florian Mormann’s lab.

* Fixes to handle split block.

* Tests NcsBlockFactory with split blocks and unfilled records.

* Tests of data read in ncs files with partial records and split blocks.

* Handle testing less data than available in all segments.

* Read and test with partial blocks. Test all types in test_neuralyxio.

* Read and test with partial blocks. Test all types in test_neuralyxio.

* PEP8 fix ups.

* Language and function renaming.

# Conflicts:
#	neo/rawio/neuralynxrawio.py

* Language and function renaming.

* Move constants into class so accessible for testing.

* Test of building NcsBlocks.

* Handle old files with truncated frequency in header and test.

* Change interface on parse versus build.

# Conflicts:
#	neo/rawio/neuralynxrawio.py

* Tests for PRE4 type and code corrections.

Tests for v5.5.1 still failing.

# Conflicts:
#	neo/rawio/neuralynxrawio.py

* Fix initializer, update loop vars.

* Add additional tests on v5.5.1 with 2 blocks

* Tests of block construction for incomplete blocks

* Fix up single block case.

* Remove unneeded classes. Clean up style.

# Conflicts:
#	neo/rawio/neuralynxrawio.py

* Add test of side effects of read_ncs_files

* Use NcsBlocksFactory and logical or.

# Conflicts:
#	neo/rawio/neuralynxrawio.py

* Fix off by one in range for list. Comments.

* Use standard time calculation for last time of block.

* Remove test with tolerance over whole length. Fix microsPerSampUsed assignement.

Using a tolerance over a longer experiment is not sensitive enough to detect blocks where perhaps a large amount of samples are dropped and there is a small gap afterwards.

* Tests of raw io for incomplete records multiple block case.

* Update stop times to include time for samples in partially filled records.

* PEP and style cleanup. Corrected gap comment.

* Line shortening for PEP8.

* More PEP8 items.

# Conflicts:
#	neo/test/rawiotest/test_neuralynxrawio.py

* Remove conflict markers.

* More conflict resolution for rebase.

* verify block function and test

* Call renamed functions.

* use _verifyBlockStructure to ensure same structure

This is hopefully a bit faster than building NcsBlocks for each file.

* Properly limit samples compared.

* Temporarily remove data test.

This matches apdavison’s from commit 2ac9596. I can’t find and revert the commit where I undid that change.

* Remove duplication of some older tests.

* Comments on structure of segment defining fields.

* Factor out NcsBlock object.

This to facilitate temporal comparisions needed for segment definitions when channels may have different recording lengths.

* Change factory methods to use NcsBlock.

* Rewrite tests for NcsBlocks refactoring.

* More refactoring of NcsBlocks.

* Gather all NcsBlocks first, then build segements.

Builds NcsBlocks for all unique structures in all files. Then sets up segment structure based on these. A precursor to more sophisticated segments for record block structures.

* change name to _scan_ncs_files rather than read

* Proper formatting of number of NcsBlock structures.

* Move neuralynxrawio to subdirectory to factor our other classes.

* Eliminate WholeTimesPositionBlock as class.

* Factor helper classes out in package.

* Define data returned by scan_ncs_files and rename.

* Return results from scan_ncs_files and unpack them into private atttributes.

* Test returned results.

* Fix for version 5.4.0 headers without :

* Allow gap which is fraction of sample time, rather than fixed micros.

This important when sampling rates are lower.

* Test changes for Cheetah v5.4.0.

* change wording about ambiguity

* Add docstring about required signature with sets of channels.

* Add comment about restrictions on selected set of channels.

* Single file mode and tests.

* Clarify use of dirname or filename, but not both.

* Start of comments to clarify streams

* More comments on streams, ids, indexes and effects on ephyviewer.

* Add comments for recording data model used and relationship to Neo IO.

* Comments regarding Neuralynx headers and how this handles Ncs files.

* Comment on slicing and lazy reading.

* Update comment on stream index and channel_index

* Remove todo to clarify.

* Clarify types of get_analogsignal_chunk.

* Assign to returned argument.

Co-authored-by: Peter N. Steinmetz <[email protected]>
Co-authored-by: Julia Sprenger <[email protected]>

Author: 3 people