Repository navigation
Description
Right now the tests that every reader gets are the compliance checks in BaseTestRawIO.test_read_all. These are consistency checks (shapes, dtypes, different ways of reading the same data agreeing with each other) and as they are the same for every format they can't express what values a given file should contain.
However, these recent fixes were all for readers that returned the right shape with the wrong values:
- BrainVisionRawIO: read marker positions as 1-based #1933:
BrainVisionRawIOused the 1-based marker positions as 0-based sample indices and every event came out one sample late. - Fix BCI2000RawIO scaling: decimal gains were parsed as 0 and the offset was applied with the wrong convention #1932:
BCI2000RawIOparsed decimal gains as 0 and every channel came out as a constant. - Map EDF digital endpoints to their physical endpoints #1920:
EDFRawIOcomputed the gain with a+ 1in the digital range.
What these PRs share is that each one adds a separate test against specific values which means we load the data again. I think we should include this mechanism within our testing framework. Each test class would list the values it expects from the files it already tests, and test_read_all would check them on the reader it has already parsed. This way we reuse the files we already download, we don't read them twice and we have one place to test values. A test class that lists nothing doesn't change:
class BaseTestRawIO:
...
expected_values = {}
def test_read_all(self):
for entity_name in self.entities_to_test:
... # build the reader, parse the header and run the compliance checks as today
expected = self.expected_values.get(entity_name, {})
event_names = list(reader.header["event_channels"]["name"])
for event_name, timestamps in expected.get("event_timestamps", {}).items():
actual, _, _ = reader.get_event_timestamps(event_channel_index=event_names.index(event_name))
np.testing.assert_array_equal(actual, timestamps)
if "scaled_samples" in expected:
... # read the first samples, rescale and compare
class TestBrainVisionRawIO(BaseTestRawIO, unittest.TestCase):
...
expected_values = {
"brainvision/File_brainvision_3_int16.vhdr": {
# SegmentDataPoints=294 in the .vhdr, a "New Segment" marker on the first sample of each segment
"event_timestamps": {"New Segment": np.arange(34) * 294},
},
}Every expected value has to come from outside neo (the specification, the vendor's reader or the file's own header) with a comment naming the source. Values copied from neo's current output would only freeze its current bugs.
I am not married to the specifics but I think we need something like this. I do something similar in neuroconv, where the EDF test class states the session start time and the subject sex in the header of edf+C.edf and the shared metadata test checks them:
Activity
[-]Check values against an outside source in the rawio tests[/-][+]Testing: improve generic test to also test concrete expected values[/+]h-mayorquin commentedon Nov 6, 2026
Another case, from the other side.
SpikeGLXRawIO.event_counthas returned the count of channel 0 for every digital channel since 2024. The sharedread_eventscheck catches it, butspikeglx/DigitalChannelTest_g0, the only test file with digital transitions, was read in its own test and never went throughtest_read_all. With the expected values attached to the files inentities_to_test, that file would have run the shared checks as well.