Skip to content

Parallelize coherence-based bad channel detection - #4750

Open
JESUSROYETH wants to merge 5 commits into
SpikeInterface:mainfrom
JESUSROYETH:radar/fix-spikeinterface-2869
Open

Parallelize coherence-based bad channel detection#4750
JESUSROYETH wants to merge 5 commits into
SpikeInterface:mainfrom
JESUSROYETH:radar/fix-spikeinterface-2869

Conversation

@JESUSROYETH

Copy link
Copy Markdown
Contributor

Description

The default coherence+psd method first materialises 100 scaled chunks and then processes them in serial. On a 384-channel recording, profiling shows that the Welch and median computations dominate the call, while extracting the random traces takes a small part of the total time.

This change sends the same random slices through TimeSeriesChunkExecutor. The default stays at n_jobs=1, but the chunks are streamed instead of kept together in memory. A final job_kwargs parameter lets callers select a thread or process pool without changing positional arguments.

I measured the default 100 × 0.3 s chunk path on an Intel Core i9-13900HX, with numerical thread pools pinned to one thread. The generated row uses the 30 s / 384-channel shape reported in the issue. The second row uses real 384-channel SpikeGLX traces from the public Noise4Sam fixture; the public file is short, it was repeated in memory to cover the same chunk path.

Workload Before After (n_jobs=8, processes) Improvement
Generated 30 s recording 8.454 s (8.384–8.479) 2.193 s (2.118–2.236) 3.86x
Noise4Sam real traces 12.836 s (12.749–12.964) 2.984 s (2.973–3.014) 4.30x

The channel ids and labels matched exactly between both implementations in each run. On the serial real-trace path, three fresh-process peak RSS measurements gave a median of 1,688,604 KiB before (range 1,688,500-1,689,352) and 353,452 KiB after (range 352,984-354,896), around 79% less. This comes from processing one random slice at a time instead of retaining the scaled chunks.

Validation:

  • pytest -q src/spikeinterface/preprocessing/tests/test_detect_bad_channels.py: 11 passed.
  • The regression tests cover a thread pool, a spawned process pool, and a two-segment recording.
  • A second spawned-process test uses a recording that is not pre-marked as filtered, so the
    highpass_filter wrapper detect_bad_channels builds internally also gets exercised across a process boundary; it matches the serial result on the same data.
  • Both new tests fail on the matching main revision and pass with this patch.
  • black and the trailing-whitespace/end-of-file checks are clean.

Fixes #2869.

@alejoe91 alejoe91 added preprocessing Related to preprocessing module performance Performance issues/improvements labels Sep 3, 2026
Comment thread src/spikeinterface/preprocessing/detect_bad_channels.py Outdated

@alejoe91 alejoe91 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @JESUSROYETH

I just have one comment but this looks great! :)

Comment thread src/spikeinterface/preprocessing/detect_bad_channels.py Outdated
JESUSROYETH and others added 2 commits September 3, 2026 08:04
Co-authored-by: Alessio Buccino <alejoe9187@gmail.com>
@JESUSROYETH

Copy link
Copy Markdown
Contributor Author

Thanks @JESUSROYETH

I just have one comment but this looks great! :)

Thanks for the review, @alejoe91! I applied both suggestions.

@alejoe91
alejoe91 requested a review from JoeZiminski September 3, 2026 15:22
@JoeZiminski

Copy link
Copy Markdown
Contributor

Thanks @JESUSROYETH! This looks great will check it out tomorrow

@alejoe91 alejoe91 added this to the 0.105.0 milestone Sep 4, 2026

@JoeZiminski JoeZiminski left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @JESUSROYETH thanks for this. The speedup is nice and the implementation is simple. I think this is a nice change.

@samuelgarcia or @alejoe91 may want to weight in as they know the parallel stuff better than me, but I cant immediately see any ways this could cause an issue.

I'm just commenting as have not played around locally or looked properly at the tests, will do this early next week!

Note to self: also check #4434 when playing around and close it.

Comment thread src/spikeinterface/preprocessing/detect_bad_channels.py
Comment thread src/spikeinterface/preprocessing/detect_bad_channels.py
@JoeZiminski

Copy link
Copy Markdown
Contributor

sorry @JESUSROYETH I did not think, that most recent refactor makes it hard to read the diff. Please revert it while others take a look, maybe it can be refactored as a follow up PR.

@JESUSROYETH

Copy link
Copy Markdown
Contributor Author

sorry @JESUSROYETH I did not think, that most recent refactor makes it hard to read the diff. Please revert it while others take a look, maybe it can be refactored as a follow up PR.

@JoeZiminski

No worries, you are right — the refactor makes the diff harder to read. I reverted it, and the PR is back to the previous version. We can revisit the refactor in a follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance issues/improvements preprocessing Related to preprocessing module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

detect_bad_channels is really slow

3 participants