Skip to content

feat(api): the seven PrimitiveConstraint operators, Query.orderBy and the [2..*] composite in the Query RPC - #1541

Merged
HuiJun merged 11 commits into
developfrom
fix/query-operators-orderby
Oct 26, 2026
Merged

HuiJun merged 11 commits into
developfrom
fix/query-operators-orderby

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

What and why

The API & Services Query RPC accepted three of the standard's seven PrimitiveConstraint operators and had no orderBy, so a client written against the standard (the reproducers {"where":{"property":"multiplicityLower","operator":"<=","value":2}}, {"where":{"property":"@type","operator":"in","value":["PartUsage","AttributeUsage"]}} and {"orderBy":["name"],"where":{…"PartDefinition"}}) was refused as malformed by the Python client before anything was sent, and would have been refused by the service. Three commits, in landing order:

  1. feat(api): accept the seven PrimitiveConstraint operators in Query — PRIMITIVE_OPERATOR_LESS_EQUAL, _GREATER_EQUAL, _IN, _INSTANCE_OF in sysml.proto (stubs regenerated with make proto); validatePrimitive/validateOrdered/compareOrdered evaluate <= and >= beside < and >; in is membership of the property's value in the list (equalsAny); instanceOf applies to @type and asks the metamodel — semantics.Model.InstanceOf (new, over the existing Model.Metaclass and metaclassConformsTo) answers whether the element's metaclass conforms to a named metaclass through the library's reflective hierarchy, so a PartUsage is an instance of Usage, Feature and Element, and a doc comment of Element but not Namespace. Operands are resolved before any element is read (queryEval.resolveMetaclasses): a metaclass the library does not declare, instanceOf on another property, an empty name, or a value list under an ordered comparison fails the call as INVALID_ARGUMENT. Translated in the Python, Node, Go, Rust, Java, MATLAB and Julia clients and tools/cmd/conformance/pkgclient.go; every translator refuses a malformed operand shape before sending (ordered comparisons: one value or a one-element list; in: a non-empty list; instanceOf: non-empty names).
  2. feat(api): sort Query results by orderBy — repeated string order_by = 4 on message Query (protojson name orderBy, so the standard's JSON is unchanged); validateOrderBy refuses an unknown property as select/where do (even when nothing would be sorted); orderKeys/sortRows sort the matched elements stably by each property in turn, ascending, declaration order last. The comparison is one function, corequery.CompareOrderValues, which the OSLC oslc.orderBy path now uses too (same semantics it had: absent before present; multiplicityLower/Upper as numbers with * greatest; everything else byte-wise text, so false < true). Every property has one value type, so present-versus-absent is the only mixed comparison. Sorting by a property select leaves out does not widen the projection. The MCP query tool forwards orderBy through protojson (its schema text names it); JSON-RPC/Connect carry it as a plain field. All seven clients take orderBy / order_by.
  3. fix(api): refuse a one-member CompositeConstraint — per the maintainer's decision: constraint : Constraint [2..*] is enforced in validateComposite beside the empty-list refusal and in every client translator; TestQueryMalformedConstraintsFail gains one-member composite (top-level and nested).

Behaviour before/after, on attribute x : Real [2..*];: multiplicityLower <= 2 — before: Python QueryError: unknown primitive operator '<=', service PrimitiveConstraint on "multiplicityLower" has no operator; after: selects x. @type in [PartUsage, AttributeUsage] — before: refused; after: the usages of those two types. orderBy: ["name"] — before: Python a query has no orderBy, service ignored an unknown field; after: ascending by name. A one-member and — before: evaluated as its member; after: CompositeConstraint combines at least two constraints, got one; write the constraint on its own.

The OpenAPI question (CompositeConstraint.constraint minItems)

The schema the compliance page names is api/openapi.yaml in Systems-Modeling/SysML-v2-API-Java-Client; its HEAD is tag 2021-09 (09be58a2de0bb44fa8f4c9deb2537a0bdd822068). It carries no minItems at all — neither 2 nor 1:

    CompositeConstraint:
      properties:
        '@type':
          enum:
          - CompositeConstraint
          type: string
        constraint:
          items:
            $ref: '#/components/schemas/Constraint'
          type: array
        operator:
          enum:
          - and
          - or
          type: string
      type: object

So the schema is silent and only Figure 7 ([2..*]) speaks — hence commit 3. For the record, that schema is narrower than the PIM throughout: its PrimitiveConstraint.operator enum is =, >, < only, value is a single string, and Query has no orderBy; this PR follows §7.1.4 (Figure 7 and the createQuery(… orderBy : String [0..*]) text) as the brief directs, and says so in docs/project/spec-compliance.md.

Specification basis

SysML v2 API & Services §7.1.4 (Figure 7: PrimitiveConstraint.operator ∈ {instanceOf, =, <, <=, >, >=, in}, CompositeConstraint.constraint : Constraint [2..*], Query.orderBy: "a list of properties of Data (or its realizations) that are used for sorting the Data objects in the query response. The order of properties in the list governs the sorting order."). instanceOf conformance uses the reflective metaclass library (SysML::Systems, KerML::Root/Core/Kernel) of the pinned SysML v2.1 Beta 2 / KerML 1.1 Beta 2 standard library.

docs/project/spec-compliance.md: the Query RPC row moves ⚠️ → ✅; the PrimitiveConstraint >/< row becomes the four ordered operators and gains in and instanceOf rows; the CompositeConstraint row states [2..*] is enforced and quotes the schema finding; a Query.orderBy row is added; the malformed-query and client rows name the new refusals; the known-limitations bullets about orderBy and the missing operators are replaced by "no paging" and "ascending only". No oracle baseline (pilot-diff, pilot-exec, xpect, reject, corpus) moved: the change is confined to the Query surface.

How it was verified

  • internal/frontend/grpc: query_operators_test.go (per operator: positive, negative, malformed shape; instanceOf over Usage, Feature, Element, Namespace, Documentation, Definition, Classifier, Step, with = on Usage matching nothing), query_orderby_test.go (strings byte-wise, numbers with *, booleans, absent values, precedence list, stable tie-break, projection unchanged, unknown property), robustness_query_operators_test.go:TestGRPCRobustnessQueryOperators (eleven malformed shapes over a populated and an empty model, each INVALID_ARGUMENT naming the fault), TestQueryMalformedConstraintsFail one-member cases.
  • tests/grpc conformance: query_operators, query_order_by, query_one_member_composite (new Query RPC in the harness, README updated).
  • Clients: Python client/python/tests/test_query.py (translation per operator, operand-shape refusals, orderBy, and the real-service cases) — 52 passed; Node client/node/test/query.test.ts mirror cases — npm run build && npm test green; Go client/opensysml surface tests (new builders, OrderBy); Rust cargo test --lib query (7 passed); Java ResultProtosTest (mvn test -Dtest=ResultProtosTest, JDK 21) green; Julia test/runtests.jl green. MATLAB test_surface_offline.m gained the mirror cases but no MATLAB/Octave is available here — not run.
  • Gate: go build ./..., go vet ./..., gofmt -l . empty, python3 scripts/changelog.py check, mkdocs build --strict all pass; go test ./... with the OMG training and pilot corpora, the pilot library XMI and the PSSM suite present (122 packages ok, no failures, exit 0); the spec-examples fixtures were not fetched here, so TestSpecExamples skipped locally and is covered by CI.
  • Dry-merge: the head d3852bb4a merged onto origin/develop (e5f9c3fca) and then onto chore(pilot): pin the OMG pilot implementation to release 2026-09 (0.63.0) #1529's head chore/pilot-pin-2026-09 (9ca600a59) in a detached worktree: no conflicts; go build ./... and go test ./internal/frontend/grpc ./internal/semantic/query ./tests/grpc ./client/opensysml pass on the merged tree. chore(pilot): pin the OMG pilot implementation to release 2026-09 (0.63.0) #1529 touches none of the files here.
  • Corrections after the first CI runs, each a real commit: sysml.pb.go regenerated for the composite comment; orderBy renamed order_by in the proto for Buf's lower_snake_case rule (protojson still reads orderBy), stubs regenerated; the Node test narrows where before reading it (eslint no-unnecessary-condition); the Java conformance runner's proto→API switch covers the seven operators and carries orderBy; a rustfmt pass over the Rust capability import list; the Octave test binds the keyword-built query to a variable before reading .orderBy (Octave cannot index a package-function call result — reproduced on Octave 6.4, where the new MATLAB cases pass in isolation; the whole test_surface_offline needs Octave ≥ 7's jsonencode, which CI has).
  • Review follow-ups: Query.orderBy is advertised as the query_order_by capability (service constant and list, Service.Query refuses a sorted query with UNIMPLEMENTED when it is withheld, capability_test.go case; every client requires it before sending a sorted query and leaves unsorted queries under query alone — Python test test_sorted_query_requires_its_own_capability, Node fake service list, Rust operations.rs, Java Model.query, Julia query, MATLAB query.m; the MCP query tool maps it; docs and the changelog fragment name it). The Rust translator refuses a non-string instanceOf operand in from_json (metaclass_name) as the other clients do, with the 7 and ["Usage", 7] cases in the operand-shape test. Re-run after these: go test ./internal/frontend/grpc ./internal/frontend/mcp/... ./tests/grpc ok; Python 53 passed; Node 450 passed; Java mvn verify ok; Julia Pkg.test() all pass; cargo test --lib query 7 passed — the full cargo test --lib aborts here in encode::tests::a_value_nested_too_deep_is_refused_rather_than_overflowing (a stack overflow in a file this PR does not touch, reproduced identically on bare develop ab1a9cf9a with the same local rustc 1.97.1, so it is this machine, not the change; CI's Rust job is the gate); mkdocs build --strict and python3 scripts/changelog.py check pass.
  • Intermediate commits: the Go module builds and the query tests pass at each of the three commits (the executor wiring for orderBy and the composite refusal are added in their own commits); the shared client translators are split by hunk and are verified at the head.

devin-ai-integration Bot and others added 3 commits October 26, 2026 15:25
API & Services §7.1.4 (Figure 7) defines instanceOf, =, <, <=, >, >= and in;
the Query RPC accepted =, < and > only. Add LESS_EQUAL, GREATER_EQUAL, IN and
INSTANCE_OF to sysml.proto and the generated stubs, evaluate them in the gRPC
query executor — the inclusive comparisons beside < and >, in as membership of
the property's value in the list, instanceOf as metamodel conformance of the
element's metaclass through the library's reflective hierarchy
(semantics.Model.InstanceOf) — and translate them in the Python, Node, Go,
Rust, Java, MATLAB and Julia clients, which refuse a malformed operand shape
before sending. A metaclass the library does not declare, instanceOf on any
property but @type, or a value list under an ordered comparison fails the call
as a fault in the query.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
API & Services §7.1.4: orderBy lists the properties the response is sorted
by, the list order governing the sort. Add repeated string orderBy to
message Query and the generated stubs, sort the matched elements by each
named property in turn, ascending and stable with declaration order as the
final tie-break — an absent value before a present one, ordered properties
as numbers with * greatest, every other property byte-wise as text — through
corequery.CompareOrderValues, which the OSLC oslc.orderBy path now shares,
and refuse an unknown property as select and where refuse one. The MCP query
tool and the Python, Node, Go, Rust, Java, MATLAB and Julia clients carry
orderBy; the gRPC robustness case covers the malformed shapes of both gaps.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
API & Services §7.1.4 Figure 7 declares constraint : Constraint [2..*]; the
standard's own OpenAPI schema (SysML-v2-API-Java-Client api/openapi.yaml at
tag 2021-09) states no minItems for the array, so the figure alone speaks.
Refuse a one-member composite as malformed beside the empty one in the
executor, and before sending in every client translator; the gRPC
conformance case pins the refusal.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration Bot and others added 4 commits October 26, 2026 15:50
The one-member refusal reworded the CompositeConstraint comment in
sysml.proto after the stubs were generated; make proto-buf now reproduces
the committed sysml.pb.go.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Field names are lower_snake_case; the protojson name stays orderBy, so the
standard's JSON and every client payload are unchanged, and only the Python
keyword of the generated message moves.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
eslint's no-unnecessary-condition refused the optional chain on a value the
test had already asserted present.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…onformance runner

The runner's proto-to-public-API switch was not exhaustive once the
operator enum grew, and it dropped orderBy on the floor.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review October 26, 2026 19:02
Octave cannot index the result of a package function call directly.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration Bot and others added 3 commits October 26, 2026 19:16
…om_json

The JSON operand was stringified by compared() before check_operand saw
it, so {"value": 7} was sent and refused by the service instead of by the
client, unlike the other translators.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
A service built before the field would drop order_by as an unknown field
and answer in declaration order without any error, so the field gets its
own capability: the service advertises it and refuses a sorted query with
UNIMPLEMENTED when it is withheld, and every client requires it before
sending one, as the clients do for render_document_html.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
@HuiJun
HuiJun merged commit bd2f827 into develop Oct 26, 2026
30 checks passed
@HuiJun
HuiJun deleted the fix/query-operators-orderby branch October 26, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant