Skip to content

fix(markdown): qualify a repeated heading's Section by its parent headings (#2586) - #2613

Merged
DeusData merged 9 commits into
DeusData:mainfrom
bline:fix/doc-section-hierarchy
Oct 31, 2026
Merged

DeusData merged 9 commits into
DeusData:mainfrom
bline:fix/doc-section-hierarchy

Conversation

@bline

@bline bline commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #2586. A Markdown Section's qualified name is the file's module plus
the heading text. So ## Implementation under # Service A and under
# Service B shared one qualified name. The graph kept one node (the
last), and the references under both headings went to the File node.

This changes Section qualified names, so please check the naming. A
heading whose name repeats in the file now has the names of the headings
above it in its qualified name:

heading before after
# Service A …docs.repeated.Service-A unchanged
## Implementation (under A) …docs.repeated.Implementation (shared) …docs.repeated.Service-A.Implementation
## Implementation (under B) …docs.repeated.Implementation (shared) …docs.repeated.Service-B.Implementation
## Notes (unique) …docs.repeated.Notes unchanged

Choices I made, any of which I'm happy to change:

  • Only repeated names are qualified. A heading whose name is unique in
    its file keeps its current qualified name, so existing queries and
    stored names for those Sections still work. The cost is that adding a
    second ## Implementation renames the first. Qualifying every nested
    heading would avoid that, but it would rename most Sections in every
    indexed repository.
  • The whole chain of headings above, not just the nearest. The name is
    then fixed by the document's structure alone, with no ordinal that shifts
    when a section is inserted above.
  • A qualified name that equals another heading's name gets a suffix. A
    heading text may contain dots, so # Service-A.Implementation would get
    the same name as the qualified ## Implementation under # Service A.
    All final names in a file are settled together: a heading whose name is
    unique keeps it, and a qualified name that collides takes the first free
    ~2, ~3, …. (When only qualified names collide, the first in the file
    keeps the name.) Names match only when their headings do. The comparison
    is on the list of headings, not on the dotted text. The suffix appears
    only in that collision.
  • Headings that repeat under the same parents (two ## Fixes under one
    # Changes) still share a name and keep the existing File fallback in
    doclink_md.c. doc_links_md_repeated_headings pins that and is
    unchanged.
  • name stays the heading text. Only qualified_name changes.

How. The parent headings come from the tree: tree-sitter-markdown nests
each heading's section inside the section of the heading above it. The
file's heading names are collected and sorted once per file, when the first
heading is seen. They are allocated in the scratch arena and held in a new
CBMExtractCtx.md_headings slot, like doc_memo. Diagrams find their
Section through the definition list, so a diagram under a repeated heading
now hangs off that heading's own Section.

This is Markdown only. reST and AsciiDoc sections are built elsewhere, and I
haven't checked whether they share the problem.

Evidence

doc_links_md_repeated_headings_by_parent_issue2586 in
tests/test_doc_links_md.c indexes the document from the issue, with a
Mermaid diagram under the second Implementation.
doc_links_md_repeated_heading_name_collision_issue2586 adds
# Service-A.Implementation to that document and requires five Sections,
with the unique heading keeping its name. Without the source change the first
test fails on the defect itself:

FAIL tests/test_doc_links_md.c:245: dm_count(... name = 'Implementation' ...) == 1, expected 2

With it, scripts/test.sh --suites doc_links_md,doc_links_rst,doc_links_adoc,doc_mentions,diagrams
reports 115 passed, 0 failed.

Checklist

  • Every commit is signed off (git commit -s) — required, CI rejects
    unsigned commits (DCO, see CONTRIBUTING.md)
  • Tests pass locally: scripts/test.sh (the canonical leg), 9384 passed, 0 failed, 10 skipped,
    189 suites, Linux x86-64, gcc
  • Lint passes: scripts/lint.sh --ci in test-infrastructure lint container
    (cppcheck + clang-format-20 + memory-core), "All linters passed"
  • New behavior is covered by a test (reproduce-first for bug fixes)

Written with an AI agent (Claude); I reviewed the change and certify it via
the DCO sign-off.

🤖 Generated with Claude Code

@bline
bline requested a review from DeusData as a code owner October 12, 2026 21:24
@github-actions

Copy link
Copy Markdown

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

  • It will not be closed for inactivity. No stale bot touches pull requests here.
  • It may still sit a while before a human reads it. That is on us, not on you.
  • Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

  • Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
  • Get CI green, or say which failures you believe are pre-existing.
  • Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
  • Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

…dings (DeusData#2586)

A Section's qualified name is the file's module plus the heading text, so
two `## Implementation` headings under `# Service A` and `# Service B` had
one qualified name. The graph kept one node (the last), and doclink_md sent
the references under both headings to the File node, since neither heading
owned the shared node.

A heading whose name repeats in the file now takes the names of the
headings above it into its qualified name
(`docs.repeated.Service-A.Implementation`). tree-sitter-markdown nests each
heading's `section` in the section of the heading above it, so the chain is
read from the tree. A heading whose name is unique in the file keeps its
name alone, so no other Section is renamed. Headings that repeat under the
same parents still share a name and keep the existing File fallback
(doc_links_md_repeated_headings is unchanged).

A heading text may hold dots, so a qualified name can equal another
heading's name: `# Service-A.Implementation` against the qualified
`## Implementation` under `# Service A`. All of a file's final names are
therefore settled together, once per file on its first heading, in the
scratch arena. A unique heading keeps its name, or else the first
qualified one does, and every other qualified name in the collision takes
the first free `~2`, `~3`, .... Names collide only when they come from
different headings: the comparison is on the list of heading names, not on
the dotted text.

Diagrams find their Section through the definition list, so a diagram under
a repeated heading now hangs off that heading's own Section.

Tests (tests/test_doc_links_md.c): the document from the issue, with a
Mermaid diagram under the second heading. Without the fix it fails on the
defect: one `Implementation` Section where there are two. The same document
plus `# Service-A.Implementation` requires five Sections, the unique heading
keeping its name. Without the collision handling it fails with four.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Scott Beck <296871+bline@users.noreply.github.com>
@bline
bline force-pushed the fix/doc-section-hierarchy branch from 0450ad6 to 9129898 Compare October 14, 2026 22:14
@DeusData DeusData added bug Something isn't working parsing/quality Graph extraction bugs, false positives, missing edges labels Oct 19, 2026
@DeusData

Copy link
Copy Markdown
Owner

Thanks for the careful repeated-heading examples and the explicit compatibility tradeoffs. I have read the proposal; the choice to qualify only repeated names, including its rename behavior when a duplicate is added, needs a maintainer naming decision during review. I am leaving that decision open rather than asking you to rework it now.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Thanks, @bline, for the heading hierarchy fix! I added the agreed refinement in 86fb94f: use parent qualification only where it separates duplicate headings, preserve existing bare names, and use numeric suffixes as the final fallback. The follow-up also handles Setext headings and container boundaries, avoids quadratic collision scans, and advances the semantic index version from 9 to 10 so existing indexes are refreshed.

The focused Markdown suite passed all 23 tests. Reverting the extraction changes made all seven selected regression checks fail for their expected reasons; restoring the changes passed all 23 again, with no sanitizer diagnostics. Thank you for the original implementation and tests that made this refinement possible!

@DeusData

Copy link
Copy Markdown
Owner

Thanks again, @bline! I also merged the latest main in b759b95 because the newly merged Haskell fix touched extraction code. This merged without conflicts and preserved the refinement patch ID. Required GitHub CI will validate the combined final head before merge.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Thank you again for improving Markdown section hierarchy! I merged current main into your branch in b54729e to include the shared reader-focus CI repair from #2753 and restart the required checks. The reviewed heading-naming refinement and its tests are unchanged byte-for-byte, and contributor history/sign-offs are preserved. This removes the known shared UI failure from the branch baseline without changing the Markdown behavior.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Thanks again, @bline! Main has moved since the last CI run, including changes in two files this PR touches (extract_defs.c and pipeline_internal.h). We merged current main into your branch in 0c2c8d9 (one signed-off merge commit). It merged without conflicts, and the heading-naming change and its tests are unchanged. The required checks will now run on the combined head, and we will merge once they are green. Please pull before pushing again.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Thanks again, @bline. Several other pull requests landed on main this evening, including changes in files this one also touches, so we merged current main into your branch once more (one signed-off merge commit, 32b61a8; your changes are untouched). That keeps the required checks honest against today's tree; we will merge once they are green. Please pull before pushing again.

DeusData#2305 (now on main) took semantic index version 11, so this branch's
Markdown Section change becomes version 12; its history entry is kept
after main's 10 and 11 entries.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Thank you for your patience, @bline. #2305 landed on main this evening and took semantic index version 11, which is the same line this pull request bumps, so the two conflicted. We merged current main into your branch (one merge commit, 4680954) and renumbered this change to version 12, keeping your history entry after main's 10 and 11. Your Section changes and tests are untouched. Existing indexes still rebuild once on upgrade, now under 12. We will merge once the fresh checks are green; please pull before pushing again.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Thank you for your patience, @bline. A pull request that merged into main a few minutes ago (#2769) changes internal/cbm/cbm.h and extract_defs.c, which this pull request also changes, so the CI run in progress here could no longer vouch for today's main. We merged current main into your branch once more (one merge commit, 1962c21; it merged cleanly and your own change is untouched) and started a fresh run. We will merge once the required checks are green. Please pull before pushing again.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Thank you for your patience, @bline. Main has moved since the last CI run on this branch, and some of the new commits change internal/cbm/cbm.h and extract_defs.c, which this pull request also changes, so that run can no longer vouch for today's main. We merged current main into your branch (one merge commit, 0bd442c; it merged cleanly and your own change is untouched) and started a fresh run. We will merge once the required checks are green. Please pull before pushing again.

@DeusData
DeusData merged commit 9f0fdcb into DeusData:main Oct 31, 2026
26 checks passed
@DeusData

Copy link
Copy Markdown
Owner

Merged as 9f0fdcb - thank you, Scott! A Markdown heading that repeats under different parents (two "Usage" sections, say) now gets its own Section, qualified by its parent headings, instead of collapsing into one node and misattributing the references beneath it. Thank you for the careful fix, the tests, and your patience while it waited for the index-version bump to line up with the other format changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working parsing/quality Graph extraction bugs, false positives, missing edges

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Repeated Markdown headings collapse into one Section and misattribute references

2 participants