Skip to content

Fix phantom spikes in wobble under amplitude scaling, add an optional joint fit - #4951

Open
grahamfindlay wants to merge 2 commits into
SpikeInterface:mainfrom
grahamfindlay:wobble-fitted-amplitude
Open

grahamfindlay wants to merge 2 commits into
SpikeInterface:mainfrom
grahamfindlay:wobble-fitted-amplitude

Conversation

@grahamfindlay

@grahamfindlay grahamfindlay commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Problems

wobble keeps two tables per chunk: objective, from which a spike's amplitude is fitted, and objective_normalized, from which spikes are found. With amplitude scaling on, which is the default (amplitude_variance=1), subtract_spike_train takes a found spike out of objective at its fitted amplitude but out of objective_normalized at amplitude 1.

For a spike larger than its template, the difference stays in the table spikes are found from. The spike's own unit is blocked there by the refractory mask, so another unit picks the leftover up, yielding a false spike. For a spike smaller than its template, too much is removed, which can hide a real spike that overlaps it.

A second problem is with the amplitude itself. The best shift is found as a position within jitter_window, but that position is used to index scalings, which spans the whole upsampled window, and this wrong amplitude is taken.

Fixing the 2 problems gets rid of phantom spikes, but leaves 1 residual (pun intended) problem: a spike fitted alone absorbs part of a neighbor it overlaps, and that neighbor can then be missed.

Changes

Commit 1 fixes problems 1+2 and therefore the phantom spikes: subtract_spike_train now takes the spike out of both tables at its fitted amplitude, and the amplitude is read through jitter_window. There is no downside to this commit, I think.

Commit 2 fixes the third problem: It adds a joint_fit option, off by default. With joint_fit=True, each new spike's start, shift and amplitude are fitted together with the spikes it overlaps, by least squares with the amplitude prior, and those spikes are then placed and fitted again while any of them moves. A spike is fitted with at most the 8 spikes most coupled to it. A spike refitted to amplitude zero is dropped.

joint_fit needs needs numba. My initial attempt to do this in numpy was slow. Too slow to be reasonable, especially since wobble was already kinda slow. So I asked Claude to optimize a numba kernel, which makes the joint_fit runtime tolerable, and in fact it is often faster than stock wobble was on non-pathological (most) data. But since the benefits of joint_fit, though very clear on tetrode data, are less clear on neuropixel data, I have left it off by default. Plus, with the joint fit, it's a little bit more of a "wobble 2.0" than "fixed wobble".

Other

Sort of a separate but related discussion (does not need to be resolved here): should amplitude_variance=1 remain the default? It is kind of extreme (e.g. see dartsort's choice of 0.01).

Also, not in this PR, but worth mentioning: The output's amplitude field is still the trace value at the spike's largest channel, not the fitted amplitude.

@samuelgarcia @yger

With scale_amplitudes on, a detected spike was removed from the
objective at its fitted amplitude but from the normalized objective at
amplitude 1. For a spike larger or smaller than its template, the
difference stayed in the normalized objective and could be detected
again as a spike of another unit. Both objectives now use the fitted
amplitude.

The fitted amplitude itself was read from the wrong row. The best
jitter is a position within the jitter window, but it was used to index
the scalings array, which spans the whole upsampled window. It now goes
through jitter_window first.

Add a test with three noiseless spikes at 2, 4 and 0.6 times their
templates, which must come back as exactly those three spikes.
Wobble fits each spike's amplitude once, when it is found. If a second
spike overlapping it is found later, the first amplitude has already
absorbed part of the second spike, which is then missed or misplaced.

With the new parameter joint_fit, each new spike's start, jitter and
amplitude are fitted together with the spikes it overlaps, by least
squares with the amplitude prior. Those spikes are then placed and
fitted again while any of them moves. A spike is fitted with at most 8
others, the ones most coupled to it, which bounds the work in dense
stretches. A spike whose amplitude is refitted to zero is dropped.

joint_fit is off by default. It needs amplitude scaling
(amplitude_variance > 0) and numba; the kernels are in _wobble_numba.py.

test_wobble_scaled_spikes now also runs with joint_fit, with two
overlapping spikes added.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant