Skip to content

docs(self-model): write the architecture document's prose as Markdown bodies with value runs - #1562

Merged
HuiJun merged 7 commits into
developfrom
docs/self-model-docgen-bodies
Oct 27, 2026
Merged

HuiJun merged 7 commits into
developfrom
docs/self-model-docgen-bodies

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What and why

examples/self-model/document.sysml wrote the architecture document's prose as one part : Paragraph per paragraph, each a string literal (:>> text = "…"), although the document generator now expands Markdown bodies in place, opens sections from headings and reads model values into prose ([](sysml:value:…)). The self-model is meant to show OpenSysML's own idioms, so this PR rewrites its document with them — the same rendered document, with the figures the model already carries read from the model instead of being retyped.

  • Every literal paragraph becomes a rep language "markdown" body at the same position, so the section structure and the tables/figures between the bodies are unchanged. The one named paragraph that was already a Markdown body (reading) stays named.
  • 12 value runs replace retyped facts: the tier and pass counts (passRegistry.tierCount/passCount), the nested-calculation budget's default and environment variable (runtime.calcDepth.defaultBound/envVar), the released and WebAssembly command counts (opensysml.releasedCommandCount/wasmCommandCount), the MCP server's tool groups, HTTP path and token variable (opensysml.mcp.toolGroups/httpPath/tokenEnv), the release asset (opensysml.wasm.releaseAsset) and the AsciiDoc parser's language and converter variable (documentPipeline.asciidocProse.markupLanguage/toolEnv).
  • 3 new [label](sysml:…) refs point the prose at the pass, rendering-kind and markup tables.
  • One stale caption corrected: "Seven budgets, seven ways a run ends early" — OpenSysMLPipeline::Runtime declares eight Budget parts and the budgetExhaustion view draws eight exhaustion states, so the caption now says eight.

Counts in document.sysml, base → head: part : Paragraph 20 → 1 (reading), string-literal paragraphs 19 → 0, Markdown bodies 2 → 21, value runs 0 → 12, tables 8 → 8, figures 23 → 23.

The AsciiDoc note

The other markup the planner accepts is exercised by a second, smaller document, OpenSysMLAsciiDoc::MarkupNote in examples/self-model/asciidoc/markup.sysml: three rep language "asciidoc" bodies carrying 9 sysml:value[…] runs (the markup languages, content kinds, run kinds and immutable-plan flag of documentPipeline.plans, the converter script, its environment variable, the Go package and the needsExternalTool flag of documentPipeline.asciidocProse, the Markdown parser's language), one sysml:ref[…], <<parsers>>/<<converter>>/<<languages>> cross-references, a [[converter]] + == The converter heading that opens a section, a bulleted list, emphasis, strong text, code spans and a NOTE: admonition, plus a Table over the MarkupLanguages query.

It lives in its own directory because an AsciiDoc body needs the external converter (scripts/asciidoc/asciidoctor-json.js, found through OPENSYSML_ASCIIDOCTOR or as asciidoctor-json on PATH) and a document with such a body does not plan without it. The architecture document, make self-model, -render-documents over examples/self-model/*.sysml and TestSelfModelDocumentRenders therefore stay converter-free — verified with the variable unset.

  • TestSelfModelAsciiDocNoteRendersInstalled (new file examples/self_model_asciidoc_test.go, 184 lines) analyses the eight self-model files plus the note with the real converter, renders it as Markdown, HTML and AsciiDoc, and pins every value run against the model's attribute, the heading-opened section, the anchors and links, the list items, emphasis/strong text and the query table. Without a converter it skips with OPENSYSML_ASCIIDOCTOR is unset and asciidoctor-json is not on PATH: run ./scripts/download-doc-pdf-toolchain.sh and point OPENSYSML_ASCIIDOCTOR at build/doc-pdf/asciidoctor/asciidoctor-json.js; with OPENSYSML_REQUIRE_PDF_TOOLCHAIN set it fails instead.
  • TestSelfModelAsciiDocNoteSource (converter-free) pins the note's shape: 3 AsciiDoc bodies, 0 Markdown bodies, 9 value runs, 1 ref, the three xrefs, the heading-opened section, and that no file matched by examples/self-model/*.sysml carries an AsciiDoc body.
  • TestExamplesAnalyseCleanly (internal/workspace/model) grouped an example's files by directory only, so it analysed the note without the self-model it layers onto and reported unresolved references. It now opens the enclosing example directories' files with a nested example — the same grouping the README's render command uses — and, when an example's only errors are a missing markup converter (document-plan-markup-tool-missing), skips with that message unless OPENSYSML_REQUIRE_PDF_TOOLCHAIN is set, in which case it fails, the policy the converter-dependent docplan and PDF tests already follow. Verified in all three modes: skip without the converter, pass with it, fail with the variable set and no converter.
  • .github/workflows/pr.yml: the full tier's PDF toolchain tests job, which already provisions the converter and sets OPENSYSML_ASCIIDOCTOR and OPENSYSML_REQUIRE_PDF_TOOLCHAIN=1, now also runs ./examples and ./internal/workspace/model with -run 'Installed|TestExamplesAnalyseCleanly', so both the converter-backed render and the example analysis run (not skip) in CI. No other shard provisions the converter.
  • examples/self-model/README.md lists the file, the install/export/render commands and a transcript of the rendered AsciiDoc; changes/unreleased/self-model-asciidoc-note.added.md records it.

Rendered-document diff (develop → head), Markdown form

./bin/sysml examples/self-model/*.sysml -render-documents <dir> at bare develop 4235cadc8 (clean worktree) and at head, diff of OpenSysMLDocument-ArchitectureDocument.md: 9 hunks, 8272 → 8278 lines, every one accounted for (identical to the diff against the original base 2c0f1750e; develop did not touch the document between the two):

Hunk Category Content
2284c value runs "Validation" paragraph now reads 4 tiers and 66 passes from passRegistry, and links the table below
2345a anchor <a id="checking-passes"></a> — the passes table became a ref target
2417c value runs "What runs" paragraph reads the calc-depth default 10000 and OPENSYSML_MAX_CALC_DEPTH from runtime.calcDepth
2432c wording fix caption "Seven budgets…" → "Eight budgets, eight ways a run ends early"
3783c value runs surfaces intro reads 5 released and 9 WebAssembly commands from opensysml
7288c value runs agents paragraph reads lsp, runtime, session, /mcp, OPENSYSML_MCP_TOKEN, sysml-wasm.wasm
7408c refs + value runs documents intro links the kinds and prose tables and reads asciidoc / OPENSYSML_ASCIIDOCTOR
7809a, 7827a anchors <a id="documents-kinds">, <a id="documents-prose"> — the two tables became ref targets

Nothing was removed: with the value runs and refs stripped, every paragraph's text is the base text verbatim (the mechanical conversion alone rendered byte-identical Markdown).

How it was verified

At head (repo root, after make build):

  • ./bin/sysml examples/self-model/*.sysml -validate — every package ✓, no errors
  • go test -count=1 ./examples/ ok
  • go test -race -count=1 ./examples/ ok
  • go test -run 'AsciiDocNote|SelfModelDocument' ./examples/ with OPENSYSML_ASCIIDOCTOR pointed at the pinned converter: 5 passed, none skipped; with it unset: the converter-backed test skips with the reason above, the rest pass
  • make self-model — 40 views into build/self-model/ (34 .mmd, as on develop; the README's "forty views" holds)
  • ./bin/sysml examples/self-model/*.sysml -render-documents build/self-model and the README's -render-document … -doc-form html -doc-toc command: every table filled, 349,882 bytes of HTML
  • PDF through the README's command with the pinned toolchain: 34 pages at head and 34 at bare develop 4235cadc8 from a clean worktree (unchanged after the e7506f764 merge) (36 at the original base 2c0f1750e) — the page count moved on develop between the two, so the README now says thirty-four
  • gofmt -l . empty, go vet ./... clean, go test ./tests/hygiene/ ok (self_model_test.go 1,789 lines, self_model_asciidoc_test.go 184, both under the cap)
  • make docs-counts current after the re-record below; python3 scripts/changelog.py check ok

New test TestSelfModelDocumentProseReadsTheModel pins the document's shape (only reading is a named Paragraph, no string-literal paragraphs, 21 Markdown bodies, 12 value runs), that the rendering has no unexpanded sysml: run, and that each run reads the model's value: the tier/pass counts and command counts from the attributes in pipeline.sysml/surfaces.sysml, the MCP strings and converter variable likewise, the calc-depth default and the AsciiDoc language from the rows of the budgets and markup tables in the same rendering.

Develop merges and baseline movement

The branch merges origin/develop twice, at 4235cadc8 (the 2026-09 pilot pin, #1529) and at e7506f764 (the re-pinned extension libraries and the tolerate filters of #1577, the restated api-json-differential.md of #1580). Each time the conflicting baselines and report were taken from develop and regenerated on the merged tree with the standard tools (oracles re-provisioned at the pin first); the only hand edits are the figure lines the referee doc tests pin to the baselines. Every movement below is against develop's own files at e7506f764.

  • pilot-apijson-diff -update (docs/project/api-json-differential-baseline.json): only the examples root moved — files 52 → 53, matched 27002 → 26937, pilot-only 15079 → 15045, ours-only 6046 unchanged, pilotLibraryElements 14020 → 15571. Two causes: the Markdown bodies (−114 matched = 6 elements × 19 literal paragraphs) and the new markup.sysml (+49 matched and the whole +1551 in pilotLibraryElements). The other eight roots are identical to develop's, byte for byte, including their input digests; totals 931 → 932 files, matched 92491 → 92426, pilot-only 37775 → 37741.
  • pilot-diff -update (docs/project/pilot-differential-baseline.json): only self-model/asciidoc/markup.sysml is new — examples root 52 → 53 files, one ours-only unresolved-reference (the typed converter-missing error the oracle run reports without a converter, line 16) and one pilot-only kind-mismatch warning (line 40, the enumeration-valued redefines root binding the pilot flags on every self-model document); totals 920 → 921 files, 518 fully agreeing unchanged, 359 → 360 ours-only, 2461 → 2462 pilot-only, 1468 → 1469 / 3570 → 3571 diagnostics. The document.sysml rows moved only in line number.
  • pilot-reject -check: no movement (349 cases, 339 both reject, 10 ours-only).
  • Corpus round-trip expectations (tests/corpus/testdata/{corpus,api_json}_roundtrip_expected.txt): committed files 52 → 53, stable self-model/asciidoc/markup.sysml.
  • docs/project/api-json-differential.md (the 49 pinned figures, the per-root table's examples row and the run totals for unkeyed references, extension annotations and unkeyable pilot elements in the prose), docs/project/pilot-differential.md (the headline, the examples and Total rows, the two examples category sentences, the movement tables) and the five testing-pilot-* skills' headline figures: numbers only, as TestRecordStatesTheBaselinesFigures, TestPilotDifferentialDocumentCountsMatchBaseline and TestW6FSkillDocumentCountsMatchBaseline pin them; make docs-counts is current.

Checklist

  • make test and make lint pass locally (the self-model gate above; CI runs the suite)
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (the self-model README figures re-verified)
  • Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
  • baselines regenerated and make docs-counts run if a gate count moved
  • No internal work-item labels in the body, docs, or changelog

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/d5353612aa7c41efb51a447c3f61bb90
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/d5353612aa7c41efb51a447c3f61bb90?variant=devin
Requested by: @HuiJun

… bodies with value runs

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration Bot and others added 4 commits October 27, 2026 01:24
…re-record the examples-root referee baselines

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…nverter

A second document, OpenSysMLAsciiDoc::MarkupNote, written as rep language
"asciidoc" bodies with sysml:value runs, <<xref>> cross-references and a
heading-opened section; its converter-gated test renders Markdown, HTML and
AsciiDoc and runs in the PDF toolchain job. Referee baselines re-recorded for
the ninth examples file.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Re-records the API-JSON and pilot differential baselines on top of the
2026-09 pilot pin: the examples root carries the self-model's Markdown
bodies and its new AsciiDoc note, every other root is develop's.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…kip prose whose converter is absent

A model in a subdirectory of an example directory layers onto the model
above it, so TestExamplesAnalyseCleanly opens the enclosing directories'
files with it. An example whose only errors are a missing markup converter
skips unless OPENSYSML_REQUIRE_PDF_TOOLCHAIN is set; the PDF toolchain job
sets it and now runs the test with the converter installed.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review October 27, 2026 07:07

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

devin-ai-integration Bot and others added 2 commits October 27, 2026 07:17
…ot table and the run totals

The per-root table's examples row and ours-missing column and the prose
totals for unkeyed references, extension annotations and unkeyable pilot
elements now state the recorded baseline's figures.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Re-records the API-JSON and pilot differential baselines on top of
develop's re-pinned libraries: the examples root carries the self-model's
Markdown bodies and its AsciiDoc note, every other root is develop's.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@HuiJun
HuiJun merged commit 17c5066 into develop Oct 27, 2026
30 checks passed
@HuiJun
HuiJun deleted the docs/self-model-docgen-bodies branch October 27, 2026 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant