Add read_openephys_binary and catalogue-based probe construction for OpenEphys#406
Open
h-mayorquin wants to merge 5 commits intoSpikeInterface:mainfrom
Open
Add read_openephys_binary and catalogue-based probe construction for OpenEphys#406h-mayorquin wants to merge 5 commits intoSpikeInterface:mainfrom
read_openephys_binary and catalogue-based probe construction for OpenEphys#406h-mayorquin wants to merge 5 commits intoSpikeInterface:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds
read_openephys_binary(settings_file, oebin_file, stream_name), a new reader for Open Ephys binary format recordings that uses bothsettings.xml(probe geometry) andstructure.oebin(binary data layout) to build probes. It also refactorsread_openephysto use catalogue-based probe construction, fixing contact_id consistency with SpikeGLX. See #405 for the broader discussion on separating probe construction from wiring.Both readers now follow the same pattern as
read_spikeglx(introduced in PR #383): build the full probe from the catalogue viabuild_neuropixels_probe, then combine it with data from the source files to determine which electrodes are active and how they map to the data. For SpikeGLX, the source file is the.metafile (IMRO table). For Open Ephys, thesettings.xmlprovides electrode positions (viaELECTRODE_XPOS/ELECTRODE_YPOS) or electrode indices (viaSELECTED_ELECTRODES), which are matched against the catalogue probe geometry to select the active contacts.read_openephys_binaryadditionally reads thestructure.oebinto setdevice_channel_indices. As proposed in SpikeInterface/spikeinterface#4435, the oebin contains per-channelelectrode_indexmetadata (added in neuropixels-pxi plugin v0.5.0, January 2023) that maps each binary file column to its electrode index. When available and non-zero,read_openephys_binaryuses this for wiring. For older recordings whereelectrode_indexis missing or all zeros, it falls back to identity wiring, which is correct because the binary.datfile is always written in channel-number order (as confirmed by Josh Siegle). The existingread_openephysis kept for backward compatibility and always sets identity wiring since it has no oebin.Because both readers now derive
contact_idsfrom the catalogue probe, this also fixes #394 (OpenEphyscontact_idsinconsistent with SpikeGLX). The previous code reverse-engineered electrode IDs from positions, which produced different IDs than SpikeGLX for probes where OpenEphys positions don't exactly match the canonical geometry. @chrishalcrow identified the affected datasets in the existing test data (OE_Neuropix-PXI-NP-Ultra,OE_6.7_enabled_disabled_Neuropix-PXI,OE_Neuropix-PXI-QuadBase). A new consistency test verifies that for these and other datasets, thecontact_idsfromread_openephysare a subset of the catalogue probe'scontact_ids.Other changes:
read_openephys_binaryis annotated withplugin_channel_key(the XML CHANNELS key, e.g. "CH0", "CH29"), distinct from neo'schannel_namewhich comes from the oebin and uses different naming conventions.settings.xml+structure.oebinfiles.