Skip to content

docs(self-model): model the MCP server, Jupyter kernel, JSON-RPC commands, notebook packages, markup parsers and FMI gate - #1502

Merged
HuiJun merged 22 commits into
developfrom
docs/self-model-surfaces-refresh
Oct 26, 2026
Merged

HuiJun merged 22 commits into
developfrom
docs/self-model-surfaces-refresh

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What and why

examples/self-model/ is OpenSysML's architecture written in SysML v2, and examples/self_model_test.go reads the facts back out of the analysed model and compares them with the implementation. The facts that test pins (pass registry, budgets, rendering kinds, export formats, RPCs, LSP capabilities, analysis engines, identity and sync) were current at origin/develop = 76c98694c; the surfaces it does not pin had drifted. The tree carries nine commands (cmd/), an MCP server, a Jupyter kernel with its Python package and JupyterLab extension, four JSON-RPC commands a WebAssembly host runs, Markdown and AsciiDoc parsers behind a prose body, a CLI with four report formats and an FMI gate over the Modelica Reference-FMUs — and the model had none of them (grep -ci of the model at 76c98694c: mcp 0, jupyter 0, asciidoc 0, wasm 1).

This PR audits the model against cmd/, internal/, client/, editors/, Makefile and docs/internals/architecture.md at that commit, models every gap in the file it belongs to, pins every new fact in a second test file (examples/self_model_surfaces_test.go, so self_model_test.go stays under the 2,000-line cap of tests/hygiene/size_test.go), exposes the new parts in two new views, extends the architecture document's queries and prose, and brings the README's file table and figures back in line with real runs.

Audit

Status: ok = modelled and pinned before this PR; wrong = modelled, corrected here; missing = added here; out of scope = not modelled, with the reason.

Area Implementation at 76c98694c Model before Now
Commands COMMANDS := sysml sysml-lsp sysml-grpc sysml-jupyter-kernel sysml-mcp (5 released), WASM_COMMANDS adds sysml-engine sysml-syntax sysml-core sysml-wasm (9) sysml, sysml-lsp, sysml-grpc only; comment said "the three commands build for WebAssembly" missing → Toolchain.releasedCommandCount = 5, wasmCommandCount = 9, one part per command; pinned against the two Makefile variables (TestSelfModelCommandsMatchMakefile)
REPL, LSP, gRPC/Connect, stdio, CLI internal/frontend/{repl,lsp,grpc,stdio}, cmd/sysml ok ok
CLI report formats -format text|json|sarif|junit, internal/frontend/report/{sarif,junit} missing CommandLine.reportSetting/reportFormats/reportPackages; pinned against cmd/sysml usage and the report packages
MCP server cmd/sysml-mcp, internal/frontend/mcp (catalog groups lsp, runtime, session, -tools, -http, /mcp, OPENSYSML_MCP_TOKEN), packaging/mcp/server.json, make conformance-mcp missing McpServer part; pinned against the catalog groups, flags, path and token constants, the manifest and the Make target
Jupyter kernel cmd/sysml-jupyter-kernel, internal/frontend/jupyter (kernel sysml, display name SysML v2 (OpenSysML), nine MIME types), wraps the REPL missing JupyterKernel part wired to repl; pinned against the kernelspec constants, flags and MIME constants
JSON-RPC / WebAssembly commands cmd/sysml-syntax (Parse, Format, Tokens), cmd/sysml-engine, cmd/sysml-core, cmd/sysml-wasm (internal/frontend/{syntax,engine,core,combined} over internal/frontend/jsonrpc); release ships sysml-wasm.wasm missing (one wasm mention) JsonCommand with SyntaxCommand, EngineCommand, CoreCommand, CombinedCommand; serves pinned against each package's switch method dispatch, released/buildsForWasm against the Makefile, releaseAsset against build-release-wasm, stateless against whether the package imports the library
Generated clients (Go, Python, Java, Node, C++), Julia, MATLAB client/ ok ok
Jupyter kernel package client/jupyter-kernel (jupyter-opensysml-kernel, make jupyter-kernel-install/-test) missing JupyterKernelPackage; pinned against pyproject.toml and the Make targets
JupyterLab extension editors/jupyterlab (jupyterlab-opensysml, src/syntax.json, make jupyterlab-build/-test) missing JupyterLabExtension; pinned against package.json, the syntax table and the Make targets
VS Code extension editors/vscode — six opensysml.* commands, sysml/kerml languages, diagram webview, landing page, tools/gengrammar, make vscode-grammar partly: part existed, no commands/diagram panel/grammar generator wrong → commands, languages, webview and landing sources, grammar generator and target added; pinned against package.json, the source directories and the Makefile
Cameo, SysON integrations client/java-based ok ok
Editor pipeline (highlight, quick fixes, suggestions, edits, format, provenance) internal/semantic/highlight, internal/check/edit, … ok ok
View engine, rendering kinds internal/ir/view ok ok
Document path (query → plan → tree → Markdown/HTML/PDF) internal/doc/* ok ok
Markup parsers internal/ir/docplan/markup: Languages() = markdown, asciidoc; AsciiDoc leaves the process through OPENSYSML_ASCIIDOCTOR / scripts/asciidoc/asciidoctor-json.js missing MarkupParser with MarkdownParser, AsciiDocParser feeding DocumentPlanner (ProseBlocks item); pinned against markup.Languages(), For(), the exec.Command call and the converter script
Exporter, migrator, code generator internal/translate/* ok ok
Conformance oracles twelve modelled; tests/fmi gates the FMI integration over OPENSYSML_REQUIRE_REFERENCE_FMUS and scripts/download-reference-fmus.sh, CI sets the variable missing (FMI gate) ReferenceFmuGate — the thirteenth oracle; pinned against the test suite's constant, the script and the PR workflow
MCP conformance make conformance-mcp is one of the conformance protocols wrong — ConformanceSuite.protocols lacked mcp corrected; pinned against the suite's knownProtocols
Pipeline stages, pass registry, tiers internal/syntax, internal/semantic, internal/check/passes ok (#1484 withdraws part-usage-part-definition; not modelled here) ok
Engines (evaluator, compiled calcs, action/state executors, sweep/Monte Carlo, solver, analysis engines) internal/exec/* ok (SweepEngine covers internal/exec/sweep, pinned by analysis.Default()) ok
Identity, sync internal/semantic/identity, reposync ok ok
FMI runtime internal/exec/fmi the gate is modelled (above); the engine is reached through analysis.Default() ok
Airflow provider not in this tree (grep -ri airflow finds nothing under cmd/, internal/, client/, editors/) — out of scope: nothing to model
Source tree allocations tests/ directory held no allocation wrong tests directory added; every new part allocated

Model changes

  • surfaces.sysml: McpServer, JupyterKernel, JsonCommand + SyntaxCommand/EngineCommand/CoreCommand/CombinedCommand, JupyterKernelPackage, JupyterLabExtension, VS Code extension attributes, MarkupParser + MarkdownParser/AsciiDocParser, DocumentPlanner.markupLanguages, ReferenceFmuGate, CommandLine report attributes, ConformanceSuite.protocols, and Toolchain counts, parts and connections.
  • pipeline.sysml: item def ProseBlocks (what a markup parser hands the planner).
  • quality.sysml: tests source directory; allocations of every new part.
  • views.sysml: surfaceTable and markupParsers (forty views rendered, up from thirty-eight).
  • document.sysml: SurfacesWithProtocols and MarkupLanguages queries, a documentPipeline root, the surfaces section's prose and a table of every surface with its command and protocol, a table of the markup languages.
  • README.md: file table (eleven interfaces, thirteen oracles, forty views, the parsers and notebook packages), the "Keeping it honest" section names both test files.

Overlaps

Dry-merged this branch (head 0aa997f48, which merges origin/develop at 1c8e9e92f, 15ac97cff and 11c4699a9) against every open PR that touches examples/self-model/, the self-model tests, or a baseline / count line this PR re-recorded (gh pr list --json files + git ls-remote --heads origin):

How it was verified

All from the repo root after make build, at head 0aa997f48:

Command Result
./bin/sysml examples/self-model/*.sysml -validate every package ✓, no errors
go test -count=1 ./examples/ ok (7.5s)
go test -race -count=1 ./examples/ ok (44.4s)
make self-model 40 views and the document rendered into build/self-model/ (38 views before this PR)
./bin/sysml examples/self-model/*.sysml -render-document OpenSysMLDocument::ArchitectureDocument -doc-form html -doc-toc -o build/self-model/architecture.html written; the surface table (eleven rows), the markup-language table (two rows) and every table that was there before are filled
./scripts/download-doc-pdf-toolchain.sh then the README's PDF command (-doc-title-page -doc-toc -doc-number-sections) 36 pages (pdfinfo). The README said thirteen; the same command over origin/develop gives 34, so the figure was already stale — the README now says thirty-six
gofmt -l . empty
go vet ./... clean
go test -count=1 ./tests/hygiene/ ok — both self-model test files are under the 2,000-line cap
OPENSYSML_REQUIRE_TRAINING_CORPUS=1 OPENSYSML_REQUIRE_PILOT_CORPORA=1 OPENSYSML_REQUIRE_SPEC_EXAMPLES=1 go test -count=1 ./tests/corpus -run 'TestCorpusRoundTrip$|TestCorpusAPIJSONRoundTrip' ok (51.5s) — no movement in the eight self-model rows of either expectation file
go run -C tools ./cmd/pilot-diff -check (examples/ is one of its roots, the self-model included) before the develop merge it did not reproduce the committed baseline: the only movement was examples/self-model/document.sysml, two more pilot-only kind-mismatch warnings (examples 61 → 63, totals 2448 → 2450 only-pilot, 3557 → 3559 pilot diagnostics) on the two new tables' in redefines root = … lines — the same shape and warning as every table already there. Adjudicated as expected and re-recorded with -update; after the develop merge -check reproduces the baseline unchanged
go run -C tools ./cmd/pilot-apijson-diff -check its provenance digest over examples/ covers the self-model, so it refused the committed baseline; re-recorded with -update after the develop merge. Against develop's baseline (168c882e4, #1476) the roots that move are examples (26227 → 26982 matched, 5956 → 6046 ours-only, 14770 → 15058 pilot-only — the new parts, attributes and ports of the self-model), manual-examples (10 → 11 files: docs/manual/examples/whole-document.sysml, merged in with #1477's head) and spec-examples by three property counts (76417 → 76419 present-equal, 1553 → 1552 present-different, 74026 → 74025 ours-missing) that develop's own changes since 168c882e4 left behind its baseline. The record's headline, launch-ready gap rows and per-root table follow the baseline (the per-root rows were stale on develop too — e.g. tests/testdata said 36 files / 1414 matched where develop's own baseline had 40 / 1460)
make docs-counts the count lines it derives from those two baselines (README.md, docs/internals/architecture.md, docs/project/{pilot,api-json}-differential.md, four .agents/skills/*/SKILL.md headlines) were brought to the new figures; -check passes: "Documentation counts and refereed figures are current"

Baselines that read the self-model: tests/corpus/testdata/{api_json_roundtrip,corpus_roundtrip}_expected.txt list the eight files — TestCorpusRoundTrip and the API-JSON round trip re-run above with no movement. docs/project/api-json-differential-baseline.json and pilot-differential-baseline.json do not take examples/self-model as a root (tools/referee/diff/main.go, tools/referee/apijson), so nothing there moves.
The two edits to docs/internals/architecture.md are the generated corpus-agreement count line make docs-counts rewrites (2448 → 2450); its prose was not touched, see Follow-ups.

Follow-ups

The audit at 76c98694c found docs/internals/architecture.md stale on five points (three of nine cmd/ binaries, internal/frontend/ without mcp/jupyter/syntax/engine/core/combined, client/ with two of eight directories, no editors/ entry, no mention of the MCP server, Jupyter kernel, WebAssembly builds or the Markdown/AsciiDoc prose parsers). All five were fixed on develop by #1505 (f0733bae8, "docs(internals): list every binary, frontend, client and editor the tree has"), which this branch now carries; the only edits to that file here are the generated count lines make docs-counts rewrites.

Two automated review findings on #1477's planner code (nested heading paths as reference targets; punctuated explicit AsciiDoc ids) are addressed on this head: the first by 845e20e5c (markupOutline.sectionAt, TestCompileWholeDocumentNestedSectionPaths, a paragraph in docs/manual/authoring.md), the second by merging #1477's head 2672d357c, which fixes it.

Checklist

  • make test and make lint pass locally (gates above)
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (examples/self-model/README.md)
  • Changelog entry — none: the self-model is an example and the change is to it, its README and its tests
  • baselines regenerated and make docs-counts run if a gate count moved — none moved
  • No internal work-item labels in the body, docs, or changelog

devin-ai-integration Bot and others added 13 commits October 24, 2026 17:54
A Markdown or AsciiDoc body owned by a Document or Section nests its
headings into sections by level, names them by explicit AsciiDoc id or
a title slug unique within the document, resolves sysml: references to
them in the same and other documents, and numbers them as part-born
sections. Out-of-sequence headings are the markup-heading-level error.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
A reference cut from a markup body carries its own spacing, so the
backends must join it without inserting a separator, as for every
other run cut from the body.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…t links

The link checker resolves a Markdown link to a sibling document as a
file next to the committed rendering, which no example or golden ships;
the cross-document reference now lives in the CLI test's model.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…nto feat/document-whole-bodies

Co-Authored-By: jason.han <hanhuijun@gmail.com>

# Conflicts:
#	internal/ir/docplan/markup/asciidoc.go
#	scripts/asciidoc/asciidoctor-json.js
MkDocs builds the site in strict mode, so a rendered example under
docs/manual/examples has to appear in the navigation like its siblings.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
npm install in scripts/asciidoc leaves node_modules/ and a lockfile that
the pinned toolchain copy under build/doc-pdf makes redundant.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…cument examples

The new manual example and the self-model's markup body move the examples
and manual-examples provenance digests; the pilot-diff baseline only
shifts line numbers in the self-model document, and the API-JSON record's
scorecard and gap rows follow the re-recorded totals.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…e-bodies

Co-Authored-By: jason.han <hanhuijun@gmail.com>

# Conflicts:
#	docs/project/api-json-differential-baseline.json
#	docs/project/api-json-differential.md
…ands, notebook packages, markup parsers and FMI gate

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…model-surfaces-refresh

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration Bot and others added 3 commits October 25, 2026 05:10
…s over the enlarged self-model

The self-model's document.sysml gained two tables of the shape the existing ones have, so the pilot reports two more kind-mismatch warnings on it (2448 -> 2450 only-pilot, 3557 -> 3559 pilot diagnostics); the API-JSON differential's examples root grew with the new parts (26229 -> 26982 matched elements). No other root moved. The count lines docs-counts derives from the baselines follow.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
… ambiguous document usages

An AsciiDoc anchor is the section's name as written when it is a basic name, else slugged, and a reference spelt as the id (`<<MassBudget>>`, `<<mass-budget>>`) finds the section either way. A heading qualified by a usage typed by more than one document is an ambiguous target, as for a part.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
A later heading's explicit id never takes over an earlier heading's name, a repeated id keeps naming its first heading, and a quoted single name (`sysml:ref['power-budget']`) is looked up by its text.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration Bot and others added 6 commits October 25, 2026 17:55
…rfaces-refresh

Co-Authored-By: jason.han <hanhuijun@gmail.com>

# Conflicts:
#	README.md
#	docs/internals/architecture.md
#	docs/manual/authoring.md
#	docs/project/api-json-differential-baseline.json
#	docs/project/api-json-differential.md
#	docs/reference/diagnostics.md
#	mkdocs.yml
A heading nested under another heading or under a part : Section is now a
reference target by its path (sysml:appendix.derivation,
sysml:overview.mass_budget), in its own document and qualified by a
document from another, so the path a section renders under is also the
path a reference may spell.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ging develop

The merge brought develop's baseline, recorded before the self-model grew,
so the examples root moves as before (26227 -> 26982 matched elements); the
manual-examples root gains the whole-document example merged in from the
document-bodies branch (10 -> 11 files), and spec-examples moves by three
property counts develop's own later changes left behind. The record's
headline, gap rows and per-root rows, and the count lines docs-counts
derives, follow the new baseline.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Takes develop's README, architecture and referee baselines, then regenerates them on the merged tree: make docs-counts, pilot-diff -update (the self-model's two extra pilot-only rows on document.sysml over develop's figures), pilot-apijson-diff -update (the examples and manual-examples roots) and pilot-reject -update (no movement). The four testing-pilot skill headlines keep develop's text with the merged figures.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@HuiJun
HuiJun merged commit ce7c0b1 into develop Oct 26, 2026
33 checks passed
@HuiJun
HuiJun deleted the docs/self-model-surfaces-refresh branch October 26, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant