Add DeAutozero preprocessing step#4423
Draft
chrishalcrow wants to merge 1 commit intoSpikeInterface:mainfrom
Draft
Add DeAutozero preprocessing step#4423chrishalcrow wants to merge 1 commit intoSpikeInterface:mainfrom
DeAutozero preprocessing step#4423chrishalcrow wants to merge 1 commit intoSpikeInterface:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a de-zeroing preprocessing step, designed for SiNAPS probes.
There are several levels of complexity in how you can do this. Currently, the PR implements the simplest possible algorithm on the simplest data, so that we have a reasonable starting point to make further progress.
Background: SINAPs probes "auto zero" their signal every now and again, to ensure the raw data remains in some reasonable range of values. These "auto zero" events shift the raw voltage reading on each channel by some amount, over a (very small) number of samples. Here's an example:
The strategy to "deautozero" the signal (any have a better name??):
get_traces, compute arecording_shiftarray equal to the cumulative jumps in each interval between jumps.This deals with the overall signal, but there are artifacts left behind by the jumps. This is because the jump does not occur at a single sample value. Instead, it is smeared across a few samples. An example of dezeroed data is shown below:
The jump has been taken care of, but either it's zeroed at the wrong place or there's an artifact on the few samples near the jump. I suggest that a good way of dealing with this, using existing tools, is
silence_periodson the samples affected by the zeroing (found in 1.)The lab I'm in touch with do a linear interpolation on these samples, which also sounds like a good idea. We don't have an implementation of this yet, right?
You now (hopefully) have an artifact-free, de-zeroed recording.
In practice, you compute the autozero information first (this requires scanning through the recording once):
Then use this information to make your dezeroed recording:
Many issues:
A. Finding all samples where zeroing is not straightforward and, depending on hardware, can be channel-specific. But there might only be 4 groups of channels, so maybe we could split the recording into these 4 groups with
split_by...B. The de-zeroed signal trends in one direction, and will saturate. Hence we'll need to upcast to int64(?) at some point. You could then apply other preprocessing and downcast again, but all a bit awkward.
EDIT: I'm also curious if all this might be overkill. You might just be able to silence the auto-zero periods and apply a bandpass filter...