Skip to content

search_graph BM25: label filter applies after the FTS5 candidate window #2386

Open

Description

@DeusData

The label filter on the BM25 path is applied after the 2000-row FTS5 candidate window (BM25_INNER_LIMIT, src/mcp/mcp.c ~3778; inner query ~4073), not inside candidate selection.

Effect: when a query token is common but the requested label is rare among its matches, the matching rows of that label can fall outside the window. The response then reports total_relation: gte / candidate_window_saturated, but can contain 0 rows, even though matching nodes exist.

Possible fix: filter on the label column of nodes_fts inside the FTS MATCH itself (or pre-select candidates per label), so the window is filled with rows of the requested label only.

Test: index a fixture where one token matches more than 2000 nodes of one label and a few of another; query with the rare label and assert those rows come back.

Found while reconciling #2159 / #2148 (fixed in 1a34255).

Activity

  1. harshitaajoshi commented on Sep 28, 2026

    @harshitaajoshi
    Contributor

    Hi, I've opened #2423 for this.

    The label now goes into the FTS5 MATCH itself ((<tokens>) AND label : "<Label>"), so the 2000 row candidate window only fills with rows of the requested label. The existing n.label = ? check stays as the exact filter, and the count query and saturation probe use the same match string, so total and candidate_window_saturated describe the filtered window.

    Two regression tests are included, one of which reproduces the issue (2001 Functions crowding out 3 Classes, which returns 0 rows on current main). Full suite and lint-ci pass locally. The PR description has details on a small ranking safeguard.

    Happy to adjust anything.

  2. added
    bugSomething isn't working
    priority/highNeeds near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker.
    on Oct 19, 2026
  3. DeusData commented on Oct 19, 2026

    @DeusData
    OwnerAuthor

    Thank you for opening #2423 and for the rare-label crowding regression. The count and saturation semantics are useful parts of the proposed change, alongside the returned rows. I have no new verified review or landing status to report in this issue.

  4. DeusData commented on Oct 25, 2026

    @DeusData
    OwnerAuthor

    Thank you for your patience and the filtered-window regression in #2423. Count/saturation behavior remains part of the maintainer review. Sorry for the delay; I’ll follow up soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingparsing/qualityGraph extraction bugs, false positives, missing edgespriority/highNeeds near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions