ci: run the reference implementation test suites - #98
Open
ucekmez wants to merge 1 commit into
Open
Conversation
Both reference stacks ship test suites — Node vitest, Python pytest with a 100% coverage gate — and no CI job ran either of them. The workflow covers every `@eep-dev/*` and `eep-*-python` package plus cross-impl, but skipped the stacks in `examples/eep-reference-implementation/`. README describes that directory as the stack "used by contributors and CI to exercise Layer 1 discovery, Layer 2 subscribe/stream, Layer 3 pulse and the gate endpoints against shared parity fixtures". CI did not use it. A subscribe handler reading `callback_url` when the schema defines `delivery_url` shipped undetected as a direct result. The Python job deliberately installs the workspace packages from source before the app's own dependencies. Without that, the job tests whatever `eep-gates` currently resolves from PyPI rather than the tree the PR changes, so a regression would surface only after a release — and a stale local copy can make the suite fail for reasons that have nothing to do with the change under review. Verified by reproducing both jobs locally from clean environments: Node 18 passed, Python 27 passed at 100% coverage. Refs: EEP audit 2026-08, follow-on from finding A1 Signed-off-by: Ugur Cekmez <ucekmez@gmail.com>
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 6 of a stacked series. Base is #97. Not for merge without review.
Both reference stacks ship test suites — Node
vitest, Pythonpytestwith a 100% coverage gate — and no CI job runs either of them..github/workflows/test.ymlcovers every@eep-dev/*package, everyeep-*-pythonpackage, andcross-impl. It skipsexamples/eep-reference-implementation/entirely.The README describes that directory as the stack
CI did not use it. The
delivery_urlbug fixed in #97 shipped undetected as a direct result — a handler reading a request field that the schema does not define, in the stack implementors are pointed at as the worked example.What changed
Two jobs:
test-reference-implementation-nodeandtest-reference-implementation-python.The Python job installs the workspace packages from source before the app's own dependencies. This matters more than it looks:
eep-gatescurrently resolves from PyPI, not the tree the PR changes — so a regression would surface only after a release.eep_gatesin a localsite-packagesmade the suite fail for reasons that had nothing to do with the change under review, and I initially mis-reported that as a parity defect. Pinning to workspace sources makes the job's verdict about the tree and nothing else.Scope
Checklist
Verification
Both jobs reproduced locally from clean environments, running the exact commands in the workflow:
test-reference-implementation-nodetest-reference-implementation-pythonapp.py164 stmts, 100% coverage (gate satisfied)YAML validated with
yaml.safe_load— 24 jobs parse.Notes for reviewers
These jobs pass only on top of #97. On
maintoday the Python suite fails, because that is the branch where thedelivery_urlfix and its regression tests live. Adding the jobs before the fix would have turnedmainred; that ordering is deliberate.I did not add a coverage gate to the Node job — the Python one has
--cov-fail-under=100from its ownpytest.ini, and the Node stack has no equivalent configured today. Raising one felt like a separate decision rather than something to slip into a CI-wiring PR.