Repository navigation
test: pin the type checker's 1 sentinel and the source-dependent receipt labels - #28
Merged
Merged
Conversation
docxology
force-pushed
the
test/p3-conformance-receipt-labels
branch
from
November 12, 2026 20:24
c8c44c9 to
12bfe49
Compare
The merged fail-closed check requires `validate_gnn_files(...) is True` and labels the receipt by source path, but the suite exercised neither completely: the non-True parametrization skipped `1`, the only value that separates an identity check from `!= True`, and no test called `main()`, so the documented `repository-content-passed` / `pinned-exemplar-baseline-passed` split was unverified. - add `1` to the rejected outcomes (it is truthy and equal to True). - call `main()` on both source paths with a stand-in `gnn` and a local exemplar fetch, asserting source, result label and models count. On the pre-change script both new cases fail: `1` did not raise, and the receipt said `verified` for either source.
docxology
force-pushed
the
test/p3-conformance-receipt-labels
branch
from
November 12, 2026 20:26
12bfe49 to
1bd96b3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What the review found
#27 made the GNN conformance gate fail closed (
validate_gnn_files(...) is True) andmade the receipt
resultsource-dependent, but its tests left two of those claimsunpinned:
tests/test_gnn_conformance.pyparametrized the rejected outcomes as[2, False, None, 0]. The documented sentinels areTrue/2/False, and1was not exercised.
1is the only value where the shipped identity check and theplausible wrong variant
outcome != Truedisagree:1 != TrueisFalse, so!= Truewould accept1while rejecting2. Nothing in the suite forbade thatedit.
main(), so the documented split —.github/GNN.md:resultisrepository-content-passedfor repository sketches andpinned-exemplar-baseline-passedwhen only the pinned upstream exemplar was checked —was unverified by the gate.
Change
tests/test_gnn_conformance.pyonly:1added to the rejected-outcome parametrization.test_main_labels_repository_content_receiptandtest_main_labels_pinned_exemplar_baseline_receiptcallmain()on both sourcepaths with the existing stand-in
gnnmodule and a local stand-in forurllib.request.urlretrieve(no network). They assertsource,result,models,gnn-version, that the fetch URL ends withEXEMPLAR_PATHand carries thepin's
revision, and — by parsing the whole captured stdout as one JSON document —that the run prints exactly the one JSON line the doc promises.
No production code changed; the merged behaviour already matched the documentation.
Test behaviour before and after
Before (
3d1afa3:scripts/gnn_conformance.py, the pre-#27 script, with the new testfile):
After (
main, this branch):9 passed.The
[1]case also catches the wrong-variant fix the old test allowed: withif outcome != True,2still raises (old test green) but1does not (new casered).
Gate
./gate.sharms in a detached worktree of1031f76with its ownuv sync --locked --extra dev:linttestbuild3.3.0programsitedocsgnn{"source": "pinned-exemplar", "result": "pinned-exemplar-baseline-passed", "models": 1}aii_sidecar validate→OK788is the previous785plus these three tests; no existing test, assertion orselection was changed.
Hosted Actions remain billing-locked: every run of this branch, and of
mainat1031f76and17e9345(PR #27), concludesfailurewith an empty step list — thejobs never start. The local gate above is the evidence.
🤖 Generated with Claude Code