Skip to content

fix(diarization): honor configurable speaker caps for long recordings - #3794

Open
phoenixray2000 wants to merge 2 commits into
modelscope:mainfrom
phoenixray2000:codex-configurable-speaker-cap
Open

phoenixray2000 wants to merge 2 commits into
modelscope:mainfrom
phoenixray2000:codex-configurable-speaker-cap

Conversation

@phoenixray2000

@phoenixray2000 phoenixray2000 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Automatic speaker counting switches from spectral clustering to UMAP/HDBSCAN at 2048 speaker embeddings. The density-based branch ignores the spectral speaker limit, so a configured maximum of 5 can still produce 8 speaker groups.

Expose a construction-time top-level AutoModel(..., max_num_spks=5) argument, defaulting to 15. Forward that value explicitly to ClusterBackend so both automatic clustering paths obey the same cap. max_num_spks inside spk_kwargs or spk_kwargs["cb_kwargs"] raises TypeError, including when a top-level value is also supplied. This leaves one AutoModel configuration entry point without alias or precedence rules. Other backend options, such as merge_thr, continue to use the existing component configuration.

Excess UMAP/HDBSCAN clusters are greedily merged by centroid cosine similarity. Noise labels are preserved and excluded from the count; pair selection excludes the diagonal and handles nonpositive similarities and zero centroids. An explicit preset_spk_num/oracle_num takes precedence on the known-count spectral and KMeans paths.

The default limit of 15 now also applies to automatic UMAP/HDBSCAN results. Callers can increase it for larger meetings. This is an upper bound, not an exact count: setting it below the actual number of speakers can merge different people. The English and Chinese SDK guides document the top-level construction argument, defaults, validation and examples.

Validation:

  • The 2048-embedding limit regression fails against the original implementation for caps 1, 3 and 5 with NumPy and Torch inputs. Top-level routing and nested-location rejection regressions also fail before the API change; all pass afterwards.
  • Core clustering and SDK documentation checks: 66 passed, 22 subtests passed. Coverage includes top-level caps 1/3/16, the default of 15, invalid values, both nested locations alone or alongside equal/conflicting top-level values, preservation of other backend options and caller dictionaries, known-count precedence, the 2047/2048 boundary, noise, closest-centroid selection and zero/nonpositive similarities. The existing real UMAP/HDBSCAN wrapper test also passes.
  • NumPy workflow selection plus SDK, component-device and timestamp regressions: 284 passed, 42 subtests passed, 1 deselected locally. The deselected MP3 decoder case reopens a still-open NamedTemporaryFile on Windows; it remains selected in Linux CI.
  • Black 24.4.0 with the repository's 100-character line length and git diff --check pass.
  • tests/test_cluster_backend.py is included in the path filters and test selection of the NumPy 1.26.4 / 2.4.0 CI matrix.

The existing product-site browser environment has a separate incompatibility: its ubuntu-latest runner uses Ubuntu 26.04 with Playwright 1.55.0. The previous browser run fails at Install Chromium, before browser tests run, with Error: ERROR: Playwright does not support chromium on ubuntu26.04-x64.

Latest head 7bec62b9763bee22c513a156ddfcc35c55d91774: NumPy 1.26.4, NumPy 2.4.0, KWS output, model contracts, and legacy document links pass (NumPy, KWS, model contracts). The current product-site build/validation is still in progress; its browser stage is pending.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant