Skip to content

fix(rust): fence a QN on the function's own cfg predicate, balanced - #2391

Closed
BobbieBarker wants to merge 2 commits into
DeusData:mainfrom
BobbieBarker:fix/rust-cfg-attr-doc-fence
Closed

BobbieBarker wants to merge 2 commits into
DeusData:mainfrom
BobbieBarker:fix/rust-cfg-attr-doc-fence

Conversation

@BobbieBarker

@BobbieBarker BobbieBarker commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

This PR was written by an AI agent working on my behalf. I certify the DCO sign-off on the commit and answer for the change.

Fixes #2390

Filed at your request in #2370. Looking at the cfg_attr fencing turned up two more misreadings in the same function, and one of them re-opens #495, so all three are here as one claim: the fence should be the function's own cfg predicate, balanced, and nothing else. #2390 carries the reproducer and the evidence for each.

The search now anchors on #[cfg( rather than a bare cfg(, so a cfg_attr carrying a nested doc(cfg(...)) stops fencing a function that is compiled unconditionally. The copy then takes the balanced span, stopping at the paren that closes cfg( and dropping newlines along with the other whitespace, which keeps the attribute's own ] out of the QN and stops a multi-line predicate being cut at its first line break. A predicate longer than the 256-byte buffer still truncates, as before, and stays distinct over its first 255 characters.

Measured

ripgrep 14.1.1 4649aa97, tokio 1.40.0 ea6d652a, rust-analyzer 2024-09-30 822644d9, indexed with a binary from 13ce2c0c and with this branch:

corpus QNs changed stop being fenced nodes added or lost
ripgrep 131 0 0
tokio 176 7 0
rust-analyzer 28 0 0

The 7 are the cfg_attr-only functions, which now carry their plain QN. tokio's multi-line predicates come back whole: main#cfg(all(tokio_unstable,tokio_taskdump,target_os=linux,any(target_arch=aarch64,target_arch=x86,target_arch=x86_64))).

Index format version

CBM_INDEX_FORMAT_VERSION goes 1 to 2, so the next index rebuilds from scratch. The deciding case is the incremental path: incr_capture_inbound_edge snapshots an inbound cross-file edge by its target's qualified_name string, and the re-link after the purge resolves that string again. A caller in an unchanged file would otherwise lose its edge into a re-extracted fenced function, with nothing reporting the loss.

Tests

Three in tests/test_extraction.c, one per misreading, and the first coverage this function has in the gating suite. tests/repro/repro_issue495.c exercises it today, but Makefile.cbm deliberately keeps that file out of make test and it asserts only that two QNs differ and that one contains not(, never the string minted. That is why a trailing ] on every fenced QN went unnoticed.

Each fails without the commit, for the reason it names: on 13ce2c0c the extraction suite goes 387 passed, 3 failed with extract_defs.c reverted to main, and 390 passed with the commit.

Full suite on the rebased branch, 13ce2c0c plus this commit: 144 of 144 suites, 8,239 passed, 0 failed, 10 skipped. clang-format clean; the four violations my local clang-format-20 reports in tests/test_extraction.c are present on a pristine tree too and are not in the lines this PR adds.

@github-actions

Copy link
Copy Markdown

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

  • It will not be closed for inactivity. No stale bot touches pull requests here.
  • It may still sit a while before a human reads it. That is on us, not on you.
  • Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

  • Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
  • Get CI green, or say which failures you believe are pre-existing.
  • Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
  • Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

@DeusData

Copy link
Copy Markdown
Owner

Thank you, @BobbieBarker. This is a really careful piece of work, and exactly what we hoped for when we asked you to file it separately. You didn't stop at the cfg_attr case. You traced all three misreadings to the QN each one mints:

You measured them on ripgrep, tokio and rust-analyzer with node counts unchanged, and you spotted that repro_issue495.c never runs in make test, which is why the ] went unnoticed. Anchoring on #[cfg( the way rust_def_is_test already does is the right call.

We checked:

On the index format version, yes please: add the CBM_INDEX_FORMAT_VERSION bump you offered. The deciding factor: on an incremental reindex, inbound cross-file edges into a changed file are snapshotted and re-linked by exact target QN (src/pipeline/pipeline_incremental.c, the block above incr_capture_inbound_edge). Without a bump, the first incremental reindex after upgrading would silently drop edges from unchanged callers into re-extracted fenced functions, until those callers are re-extracted or a full reindex runs. A one-time rebuild is the honest price for QNs that are now correct, and we'll put a line in the release notes as we did for #1104.

Tiny and optional, and pre-existing: the #495 doc comment above rust_def_is_test (~line 2041) actually describes rust_cfg_qualified_name. If you'd like to move it down while you're here, you're welcome to. Thanks again!

@BobbieBarker
BobbieBarker force-pushed the fix/rust-cfg-attr-doc-fence branch from d450275 to f297737 Compare September 28, 2026 21:09
@BobbieBarker

BobbieBarker commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Both in, rebased onto 13ce2c0c.

CBM_INDEX_FORMAT_VERSION is 2. I checked the mechanism you named before taking it: incr_capture_inbound_edge does s->target_qn = strdup(tgt->qualified_name) and the post-purge re-link resolves that string, so the dropped edge is real and silent. The commit message records it. test_index_format.c asserts the version symbolically and needed no edit.

The #495 comment moved down to rust_cfg_qualified_name.

rust_cfg_qualified_name folds a function's `#[cfg(...)]` predicate into its
qualified name so two mutually-exclusive twins get distinct QNs instead of one
overwriting the other (DeusData#495). It read the attribute three ways that the QN it
minted shows are wrong.

The suffix was copied from the `cfg(` it found to the END of the decorator text.
extract_decorators stores the bracketed form, so that appended the attribute's
own `]` to every QN: `#[cfg(test)]` minted `add#cfg(test)]`. Across ripgrep
14.1.1, tokio 1.40.0 and rust-analyzer 2024-09-30 that is 332 of 335 fenced QNs.

On a multi-line attribute the copied text also carried the newline, and
cbm_defs_push cuts a qualified name at a line break by design, pinned by
defs_push_cuts_multiline_names_and_rejects_js_literal_names. What survived was
everything before the newline, so the suffix stopped mid-predicate:
`thread_rng_n#cfg(any(`. Two twins whose predicates differ only past that
newline then computed ONE qualified name and the second upsert overwrote the
first, which is the loss this function exists to prevent. 10 tokio QNs are
truncated this way. None of them happens to collide, because the visible
prefixes stay distinct there; the collision is in the test below, which drops
from two nodes to one without this commit.

`strstr(decorators[i], "cfg(")` was also unanchored, and `cfg_attr` carries its
own nested `cfg(`. In `#[cfg_attr(docsrs, doc(cfg(feature = "rt")))]` that inner
predicate gates DOCUMENTATION, not compilation: the function is compiled
unconditionally and has no twin to be told apart from, and it was fenced anyway
as `yield_now#cfg(feature=rt)))]`. 7 tokio functions are fenced solely by a
cfg_attr. rust_def_is_test, twenty lines above, already anchors on the bracketed
path and says why; this was the one Rust decorator inspector that did not.

The fix anchors on `#[cfg(` and copies the balanced span, stopping at the
paren that closes it and dropping newlines with the other whitespace. A
predicate longer than the 256-byte buffer still truncates, as before, and stays
distinct over its first 255 characters.

Measured by indexing the three corpora above with a binary built from 5df8b04
and with this one:

  corpus          QNs changed   stop being fenced   nodes added or lost
  ripgrep                 131                   0                     0
  tokio                   176                   7                     0
  rust-analyzer            28                   0                     0

The 7 are the cfg_attr-only functions, which now carry their plain QN. Node
counts are identical in all three, and tokio's multi-line predicates are
reconstructed in full:
`main#cfg(all(tokio_unstable,tokio_taskdump,target_os=linux,any(target_arch=aarch64,target_arch=x86,target_arch=x86_64)))`.

This changes node identity for cfg-fenced Rust functions, so
CBM_INDEX_FORMAT_VERSION goes 1 -> 2 and the next index rebuilds from scratch.
The deciding case is the incremental path rather than the mixed store:
incr_capture_inbound_edge snapshots an inbound cross-file edge by its target's
qualified_name STRING, and the re-link after the purge resolves that string
again. A caller in an unchanged file would therefore lose its edge into a
re-extracted fenced function, because the QN it was snapshotted under no longer
exists, and nothing would report the loss.

The DeusData#495 comment sat above rust_def_is_test while describing this function; it
moves down with the code it documents.

tests/test_extraction.c: three tests, one per misreading, and the only coverage
this function has in the gating suite. tests/repro/repro_issue495.c exercises it
today but Makefile.cbm keeps that file out of `make test`, and it asserts only
that two QNs differ and that one contains `not(`, never the string minted. Each
of the three fails without this commit.

Signed-off-by: Chad <4307099+BobbieBarker@users.noreply.github.com>
@BobbieBarker
BobbieBarker force-pushed the fix/rust-cfg-attr-doc-fence branch from f297737 to 538ac25 Compare September 28, 2026 21:14
@DeusData DeusData added bug Something isn't working parsing/quality Graph extraction bugs, false positives, missing edges labels Oct 19, 2026
@DeusData

Copy link
Copy Markdown
Owner

Thanks for adding the requested index-format change and correcting the comment. I have read your follow-up; the revised Rust cfg identity fix remains pending the next maintainer review/merge step. No new change is requested here.

@DeusData

Copy link
Copy Markdown
Owner

Thank you, @BobbieBarker, and sorry this sat so long. Your analysis here was excellent: you traced all three misreadings in rust_cfg_qualified_name to the exact names they minted (the stray ] on 332 of 335 fenced names, the multi-line predicate cut at its first newline that quietly reopened the #495 collision, and the docs-only cfg_attr fencing a function with no twin), measured them on ripgrep, tokio and rust-analyzer, and spotted that repro_issue495.c never runs in make test. You also turned the index-format request around the same day.

We are closing this one because main has since retired the function it fixes, not because anything was wrong with your fix. Commit 42a9320 (landed with #2458 on 2026-10-07, included in v0.12.0 and v0.12.1) resolved #495 a different way: cfg-gated Rust twins no longer get a #cfg(...) suffix at all. They keep the plain qualified name and become one node whose variants property lists every twin's span, and whose calls are the union of both bodies (tests/test_conditional_variants.c, cv_rust_cfg_free_functions and cv_rust_cfg_methods). The old suffixed names left every call made inside a twin without a source node, because those calls are attributed to the plain name.

With no suffix, all three cases you found are gone on current main. We indexed your reproducers from #2390 and this PR with a build of current main: the two multi-line twin functions are one twin node whose variants keep both spans, so nothing is overwritten; #[cfg(test)] fn gated is plain gated with no bracket; and yield_now under the docs-only cfg_attr is plain yield_now. When each multi-line twin calls a different function, the one node calls both, and its caller still reaches it. rust_cfg_qualified_name no longer exists, so this branch has nothing left to apply to, and its tests assert the suffixed names main deliberately stopped minting.

Your index-format concern is covered too: the format has moved from 1 to 4 since then, so any index that still holds the old suffixed names is rebuilt in full on upgrade, and the incremental edge loss you identified cannot happen.

Thanks again for the care you put into this, and for #2370 before it.

@DeusData DeusData closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working parsing/quality Graph extraction bugs, false positives, missing edges

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A multi-line #[cfg(...)] can lose one of its twins, and a docs-only cfg_attr fences a function that has none

2 participants