Repository navigation
feat(grpc): carry the notes behind a solver verdict on every surface - #1567
Merged
Merged
Conversation
A solver's verdict rests on what the query assumes and what it leaves uncovered: a feature read as one value or object while only the default 0..* multiplicity governs it (KerML 1.1 §7.4.12; SysML v2 §7.6.3 gives the implicit [1..1] to an eligible usage only), and an object-holding feature whose own assertions the question does not state. The REPL printed both; gRPC, MCP and every client saw neither. The notes are now worded once (solve.Note: kind, subject, text), printed by the REPL from that list, carried by every RPC verdict as Verdict.notes under the verdict_notes capability, printed by the MCP verify tools and read by the Go, Python, Node, Rust, Java, Julia and MATLAB clients. Co-Authored-By: jason.han <hanhuijun@gmail.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
…he MDK and SysON tests Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # client/node/src/generated/sysml_pb.ts # client/python/opensysml/proto/sysml_pb2.py # client/rust/conformance/sysml.descriptor.binpb
Co-Authored-By: jason.han <hanhuijun@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
A solver's verdict rests on two kinds of note the REPL already printed (
%check,%solve,%explain,%configure,%optimize): the features the question reads as one value or object while only the default0..*multiplicity governs them (Assuming X holds one value: it declares no multiplicity, so the default [0..*] governs it, recorded asQuery.AssumedOnesince #1553), and the object-holding features whose own assertions the question does not state (Query.Uncovered). The gRPCVerdicthad no field for them, so the assumption behind a verdict was invisible to gRPC, MCP and every client. An assumption behind a verification verdict must be visible on every surface.solve.Note(Kind,Subject,Text) andQuery.Notes()word each note once, in report order (uncovered first, then assumed). The REPL'snoteLinesprintsTextfrom that list; nothing is re-formatted downstream. The REPL text of the uncovered note loses its REPL-only remedy (%instantiate and %validate→instantiate and validate an object to check it) so the same sentence reads on every surface.Verdict.notes = 18(repeated VerdictNote),VerdictNote { kind = 1; subject = 2; text = 3 }, withkind∈assumed_one|uncovered. Stubs regenerated withmake proto(Go, Python, TypeScript, Rust, descriptor);make proto-lintpasses; JSON spellsnotes/kind/subject/text. Java message classes are generated by Maven, as before.symbolicVerdict(the one path every solver-answered verdict ofVerifyConstraint,VerifyRequirementandVerifySatisfactiongoes through) fillsNotesfrom the queries before asking the solvers, so an undecided solver answer still says what the translation assumed; an evaluation (questionempty/evaluate) and a refused translation carry none. Newverdict_notescapability (response-populated, likeverification_verdicts), appended last; MCPserver_infomaps it toverify_constraint.verdictTextprints each note under the verdict, as the REPL does.Verdict.Notes,VerdictNote,NoteAssumedOne/NoteUncovered), Python (Verdict.notes,VerdictNote,NOTE_*,CAPABILITY_VERDICT_NOTES), Node (noteson everySysMLVerdictarm,VerdictNote,CAPABILITY_VERDICT_NOTES), Rust (Verdict.notes,VerdictNote,NOTE_*,CAPABILITY_VERDICT_NOTES), Java (Verdict.notes(),VerdictNoterecord,Capabilities.VERDICT_NOTES; the conformance renderer carries them back), Julia (Verdict.notes,VerdictNote,CAPABILITY_VERDICT_NOTES), MATLAB (Verdict.notes,capabilities().verdictNotes). The Go package conformance adapter carries notes back to the wire.docs/reference/api.mdVerdict section,docs/reference/service-transports.mdcapability table, the compliance page (the multiplicity row now says the assumption reaches every gRPC verdict, a new row covers the notes on every surface, and the clients paragraph listsnotes), each client's reference page,tests/grpc/testdata/conformance/README.md. Changelog fragmentchanges/unreleased/verdict-notes.fixed.md.Specification basis
KerML 1.1 Beta 2 §7.4.12 (a feature that declares and inherits no multiplicity has the default
0..*) and SysML v2.1 Beta 2 §7.6.3 (the implicit[1..1]belongs to an eligible usage only — a barein x : Realparameter is a reference usage and keeps the default), as #1553 established. This PR changes no semantics: it carries the assumption the solver already records to the transport.docs/project/spec-compliance.md: the "feature read as one value" row's text and implementation column are updated and one row is added after the verification-questions row (🧩 Extension); no row changes status. No oracle baseline moved.How it was verified
Tests added:
internal/frontend/grpc/verify_notes_test.go: a KerMLfeature level : Realread by aninvanswersholds→violatedwith exactly oneassumed_onenote (subjectTanks::Tank::level, the REPL's sentence); the evaluation of the same constraint carries none;verdictNoteskeeps the query order (uncovered, then assumed) over several queries.internal/frontend/grpc/robustness_verdict_notes_test.go(TestGRPCRobustnessVerdictNotes): an old client — theVerdictdescriptor rebuilt without field 18 andVerdictNote— decodes a new verdict with the notes discarded andstatusintact; protojson spells the notes in camelCase and round-trips; a refused translation (**) is undecided with no notes.tests/grpc/testdata/conformance/verify_constraint_default_multiplicity.{sysml,expected.json}with a newVerifyConstraintRPC in the conformance runner (question,expected_status,expected_notes; skips without an SMT solver unlessOPENSYSML_REQUIRE_SMT).internal/frontend/mcp/runtimetools/runtimetools_test.go:TestVerificationToolsPrintTheVerdictNotes.internal/frontend/grpc/capability_test.go:verdict_notesis the newest capability.tests/test_verification_questions.py, Nodetest/verify.test.ts, Rustresults.rsunit tests, JavaResultProtosTest, Juliatest/extended_client.jl("verdict notes"), MATLABtests/test_surface_offline.m.internal/frontend/repl/configure_nested_test.go: the three uncovered-note expectations follow the reworded sentence.Gates run locally (z3 and cvc5 installed, so the solver tests ran rather than skipped):
go build ./...,go vet ./...,gofmt -l .(empty),make proto+make proto-lint.go test ./...withOPENSYSML_REQUIRE_TRAINING_CORPUS=1 OPENSYSML_REQUIRE_PILOT_CORPORA=1 OPENSYSML_REQUIRE_SPEC_EXAMPLES=1 OPENSYSML_REQUIRE_SMT=1after fetching the corpora: every packageok— 118 packages in the first run;cmd/sysml,internal/exec/runtime,internal/exec/smtandtests/codegenhit Go's 10-minute default under the parallel client suites and passed on a solo rerun with-timeout 60m(453 s, 338 s, 321 s, 492 s).make lint(staticcheck + gosec) passes;go test -C tools ./cmd/conformance/...covers the Go package conformance adapter.python3 scripts/changelog.py check,mkdocs build --strict.pytest tests/test_verification_questions.py(10 passed, 10 skipped: no live service); Nodenpm testagainst a freshbin/sysml-grpc(454 tests, 453 pass, 1 skipped, 0 fail); Rustcargo test -p opensysml --lib results::tests(16 passed) andcargo check --all-targets; Javamvn -pl opensysml-client test -Dtest=ResultProtosTest,ResultTypesTest(37 tests, 0 failures) andtest-compileof the conformance module; Juliatest/runtests.jl(exit 0,verdict notes | 5 5). MATLAB is not installed here:Verdict.mand the offline test were checked by reading only.origin/develop(f232fc5) and against chore(pilot): pin the OMG pilot implementation to release 2026-09 (0.63.0) #1529's head (9ca600a,chore/pilot-pin-2026-09) — both merge clean; on the chore(pilot): pin the OMG pilot implementation to release 2026-09 (0.63.0) #1529 merged treego test ./internal/frontend/grpc ./tests/grpc ./internal/exec/solve ./internal/frontend/replpasses. chore(pilot): pin the OMG pilot implementation to release 2026-09 (0.63.0) #1529 and this PR share onlydocs/project/spec-compliance.md, in different rows. fix(solve): read a feature's multiplicity as the model computes it, assuming one value only where the default 0..* governs #1553 and fix(runtime): judge a behavior's write count by the feature's effective multiplicity #1556 are already in develop.Not verified: the MATLAB suite (no MATLAB on this machine); a live Python/Julia run against the new service beyond the fake-transport mirror tests (the Go
tests/grpcconformance case and the MCP test cover the service side).Checklist
make testandmake lintpass locallychanges/unreleased/<slug>.<section>.md, not as an edit toCHANGELOG.mdmake docs-countsrun if a gate count moved (compliance rows need nothing: the census is counted at docs build)F4,K5) in the body, docs, or changelogLink to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/c07151afb1e540a78ec7f5fb70824aaf
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/c07151afb1e540a78ec7f5fb70824aaf?variant=devin
Requested by: @HuiJun