Extend artifact detection: DetectAndRemoveArtifacts preprocessor and signed saturation

[alejoe91]

This PR introduces a DetectAndRemoveArtifactsRecording class so that it can be used in the preprocessing pipeline.
Users can pass a recording_to_detect: in this case, saturation/artifacts are detected on that recording (e.g., raw) and applied to the preprocessor parent (preprocessed)

Also extended saturation to optionally provide the "sign" of the saturation events. This can be useful to further inspect saturation issues and track down experimental causes to the saturation.

EDIT (Chris): new preprocessing pipeline docs: https://spikeinterface--4539.org.readthedocs.build/en/4539/modules/preprocessing.html#the-preprocessing-pipeline

[chrishalcrow]

Hello, this is great. Part of (my) pipeline philosophy is that you can load a pipeline from a dumped json/pickle:

spikeinterface/src/spikeinterface/preprocessing/pipeline.py

Line 252 in a3f8db3

def _make_pipeline_dict_from_recording_dict(recording_dict):

I don't know how this will work when one argument is a recording object? Might just work?

And I also like the idea that you can specify a pipeline externally without reference to the recording, which this breaks. But more philosophical than practical... I can't really see a way around this.

[alejoe91]

Hello, this is great. Part of (my) pipeline philosophy is that you can load a pipeline from a dumped json/pickle:

spikeinterface/src/spikeinterface/preprocessing/pipeline.py

Line 252 in a3f8db3

def _make_pipeline_dict_from_recording_dict(recording_dict):

I don't know how this will work when one argument is a recording object? Might just work?

And I also like the idea that you can specify a pipeline externally without reference to the recording, which this breaks. But more philosophical than practical... I can't really see a way around this.

I have a suggestion. We can have some "magic" string:

• "pipeline['raw']": tells the function to use the input recording
• "pipeline['{preprocessing_step}']: grabs recording at a specific step.

What do you think?

[oliche]

It's good to go, maybe we could state that this apodization is a number of samples ?

Thanks a lot for looking at this !

[alejoe91]

It's good to go, maybe we could state that this apodization is a number of samples ?

Thanks a lot for looking at this !

Renamed apodization_samples ;)

[alejoe91]

When it's apodization, we should get traces with margin because the events could be on the edge. In that casem different start/end frames could give different results if we don't take margins into account

[alejoe91]

@oliche do you agree?

[alejoe91]

@samuelgarcia I implemented the margin logic and added a proper test. For zeros and noise, margin is set to 0, so the behavior should be the same

[chrishalcrow]

I've only read the preprocessing pipeline stuff - honestly so happy with the solution!
Please don't merge until you've got an approval on the other stuff!

[alejoe91]

Also added automatic loading od saturation levels for SpikeGLX and OpenEphys, follwing SpikeInterface/probeinterface#434

When available, this sets the saturation_threshold_uV property, so saturation detection is automatic!!!