Skip to content

Reorder spike vectors with a flexible-dtype numba kernel - #4784

Open
grahamfindlay wants to merge 4 commits into
SpikeInterface:mainfrom
grahamfindlay:perf/reorder-record-kernel
Open

grahamfindlay wants to merge 4 commits into
SpikeInterface:mainfrom
grahamfindlay:perf/reorder-record-kernel

Conversation

@grahamfindlay

Copy link
Copy Markdown
Collaborator

Follows @samuelgarcia 's comment on #4695 and #4781.

The numba kernel from #4695 moved rows through a flat (N, num_fields) int64 view and addressed unit_index and segment_index as columns 1 and 2, assuming minimum_spike_dtype.

Samuel suggest that we might want to change the spike vector dtype in the future, so this is a more flexible variant that can take narrow index dtypes (e.g. int8).

Also updated the numpy fallback to work for narrow index fields.

Measured on ~400M spikes, 342 units:

  • 11.6 -> 12.0 s (3x int64 fields)
  • 13.1 -> 13.0 s (4x int64 fields)
  • 9.1 s (1 each of int64/int32/int8 fields)

@alejoe91 alejoe91 added core Changes to core module performance Performance issues/improvements labels Sep 16, 2026
@grahamfindlay

Copy link
Copy Markdown
Collaborator Author

The 2 failing tests don't appear to have anything to do with this PR. They're tolerance tests failing on test_decimate_antialias_by_chunks which funnily enough passed in its own PR (#4621) with the same scipy and numpy versions, so they must be nondeterministic. Maybe create_sinusoidal_traces just needs a seed. It looks like they're still tiny tiny single-sample discrepancies. It must be a rounding thing on Apple Silicon.

@grahamfindlay

grahamfindlay commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator Author

Yes, I know it was kinda naughty of me to push a fix for that to this PR... forgive me.

EDIT: No longer applicable, see below.

@grahamfindlay
grahamfindlay force-pushed the perf/reorder-record-kernel branch from 62a512f to 22cd36f Compare September 17, 2026 16:42
@grahamfindlay

Copy link
Copy Markdown
Collaborator Author

Removed the fix for the unrelated failing tests since another one was already just merged in #4792

@JoeZiminski
JoeZiminski self-requested a review September 23, 2026 20:05
@grahamfindlay
grahamfindlay force-pushed the perf/reorder-record-kernel branch from 22cd36f to 89ee24f Compare October 8, 2026 17:10
@grahamfindlay

Copy link
Copy Markdown
Collaborator Author

@samuelgarcia the failing Python 3.14 tests are unrelated and should be fixed in #4845 so this is still ready for your review anytime

@grahamfindlay
grahamfindlay force-pushed the perf/reorder-record-kernel branch from 89ee24f to 98f3b8d Compare October 14, 2026 16:45
grahamfindlay and others added 3 commits October 29, 2026 15:40
The counting-sort kernel from SpikeInterface#4695 moved rows through a flat (N,
num_fields)
int64 view and addressed unit_index and segment_index as columns 1 and
2, essentially assuming minimum_spike_dtype.

Samuel suggest that we might want to change the spike vector dtype in
the future, so this is a more flexible variant that can take narrow
index dtypes (e.g. int8).

Also updated the numpy fallback to work for narrow index fields.

Measured on ~400M spikes, 342 units:
- 11.6 -> 12.0 s (3x int64 fields)
- 13.1 -> 13.0 s (4x int64 fields)
- 9.1 s (1 each of int64/int32/int8 fields)
@grahamfindlay
grahamfindlay force-pushed the perf/reorder-record-kernel branch from 98f3b8d to 99ab7c5 Compare October 29, 2026 20:57
@grahamfindlay

Copy link
Copy Markdown
Collaborator Author

@samuelgarcia Actually, nothing about this had to change, so it is all ready for your review.

@JoeZiminski JoeZiminski left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool @grahamfindlay this is very nice! Took me a while to look back and understand the provenance of these changes, in general a cool program of work. I expected this change to require much more than it did, very elegant.

No real comments feel free to ignore renaming suggestions, the only one to really consider is excluding some other dtype than numba can't handle e.g. float16

any width. Any other fields (e.g. the "channel_index" that
`to_spike_vector(main_channel_indices=...)` adds) are carried through to the output.
Anything array-like is accepted and materialised with `np.asarray`, so lazily-backed spike
vectors work, at the cost of being read into memory.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worth mentioning here that if any other fields are objects it forces the slow path?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

in_flat = np.ascontiguousarray(spike_vector).view(np.int64).reshape(num_spikes, num_fields)
out_flat = np.empty((num_spikes, num_fields), dtype=np.int64)
spike_vector = np.ascontiguousarray(spike_vector)
ordered_spikes = np.empty(num_spikes, dtype=spike_vector.dtype)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this be called ordered_spike_vector?

@jit(nopython=True, nogil=True, cache=False)
def reorder_spike_vector_numba(in_flat, unit_stride, segment_stride, num_buckets, out_flat, order, counts):
def reorder_spike_vector_numba(
spikes, unit_index, segment_index, unit_stride, segment_stride, num_buckets, out, order, counts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could spikes be spike_vector and out be output_spike_vector? (not strong on this, but renaming out would be nice)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I used ordered_spike_vector rather than output_spike_vector, to match the name in reorder_spike_vector_by_unit_and_segment, since it's the same array.

Stable counting-sort of a structured spike vector by (unit, segment).

`unit_index` and `segment_index` are the field views (i.e., zero-copy, strided) of `spikes`.
They are widened to int64 so narrow fields can't overflow the bucket arithmetic.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could mention here 'They can be of arbitrary integer dtype, and so are widened…"

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


`unit_index` and `segment_index` are the field views (i.e., zero-copy, strided) of `spikes`.
They are widened to int64 so narrow fields can't overflow the bucket arithmetic.
Rows move as whole records (`out[pos] = spikes[i]`), so any extra fields travel with their spike.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious why this doesn't work if the extra fields are objects now

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is essential because numba compiles the kernel to machine code that runs without the Python interpreter. Each object field holds reference(s) to Python objects, and Python counts the references to each object so it knows that object can be freed. To copy a reference safely, something needs to update that count, and AFAIK only the interpreter can do that. So numba refuses to compile for arrays with object fields at all.



NARROW_INDEX_DTYPES = {
"int64/int32/int8": np.dtype([("sample_index", "int64"), ("unit_index", "int32"), ("segment_index", "int8")]),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this is done elsewhere, is it also worth testing the all int64 array too?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya it's already covered by test_reorder_spike_vector_by_unit_and_segment and test_reorder_spike_vector_by_unit_and_segment_extra_fields, which both use all-int64 dtypes, and each runs on both the numba and numpy paths.*

*the core CI job doesn't install numba, so the numba cases are skipped on PRs. Just discovered this. But I think the daily full-tests job still runs them. And of course I ran them locally.

"""
names = dtype.names
if names is None or names[:3] != tuple(name for name, _ in minimum_spike_dtype):
if names is None or dtype.hasobject:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could also check:

  • dtype is not float16 which numba cannot support, as well as some extended precision float and complex dtype
  • dtype.isnative ensures endianness is same as machine in use

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a good catch. float16 actually crashed the kernel instead of falling back to the numpy path like it was supposed to. So what I did was, rather than list everything numba can't handle, I'm now passing each record to the kernel as just generic/opague/mystery bytes (a view with a single uint8 array field), since all the kernel has to do with a record is move it. So any extra field like float16 can now take the numba path, even though numba doesn't support float16. Only object fields (see comment above) and unit_index/segment_index fields that aren't native-order integers (that's where isnative went) fall back to numpy. Speed is unchanged as far as I can tell.

Fields numba can't compile crashed the kernel instead of falling back to numpy. Since the kernel only moves whole records, it now gets them as opaque bytes, so only object fields and non-native or non-integer index fields use numpy. Also rename and docstring clarifications from review.
@grahamfindlay

Copy link
Copy Markdown
Collaborator Author

@JoeZiminski Thank you for the very good and helpful review! Should be all set to go now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Changes to core module performance Performance issues/improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants