Skip to content

OverflowError when reading large files #1825

@LizzyMcKay

Description

@LizzyMcKay

Describe the bug
np.unit32 is not enough to represent longer recordnigs, resulting in OverflowError when _get_analogsignal_chunk is called

To Reproduce
Tried to process and sort a .bin recording (30 minutes @ 48 kHz) from the Axona system using SpikeInterface and Mountainsort 5. Calling sortings_ms5 = ss.run_sorter( sorter_name='mountainsort5', recording=presleep01) throws the following error:

File "/home/<user>/conda/envs/mntenv2/lib/python3.14/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py", line 367, in get_traces
    raw_traces = self.neo_reader.get_analogsignal_chunk(
        block_index=self.block_index,
    ...<4 lines>...
        channel_indexes=channel_indices,
    )
  File "/home/<user>/conda/envs/mntenv2/lib/python3.14/site-packages/neo/rawio/baserawio.py", line 1006, in get_analogsignal_chunk
    raw_chunk = self._get_analogsignal_chunk(block_index, seg_index, i_start, i_stop, stream_index, channel_indexes)
  File "/home/<user>/conda/envs/mntenv2/lib/python3.14/site-packages/neo/rawio/axonarawio.py", line 390, in _get_analogsignal_chunk

sample1 = raw_samples * (bin_dict["bytes_packet"] // 2) + bin_dict["bytes_head"] // 2 + offset
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
OverflowError: Python integer 5719044312 out of bounds for uint32

Expected behaviour
The recording is loaded and then sorted.

Environment:

  • OS: Linux 6.19.6-arch1-1
  • Python version: Python 3.14.3
  • Neo version: 0.14.3
  • NumPy version: 2.4.2

Additional context
Changing line 389 in axonarawio.py to raw_samples = np.arange(num_packets_oi + 1, dtype=np.uint64) fixes the issue. Should I file a PR?

EDIT: It appears that line 403 should also be changed to raw_signals[:, i] = self._raw_signals[np.uint64(sig_ids) + chan_offset] because numpy sometimes upcasts sig_ids to float64

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions