Skip to content

ci: run the reference implementation test suites - #98

Open
ucekmez wants to merge 1 commit into
fix/reference-impl-delivery-urlfrom
ci/test-reference-implementation
Open

ci: run the reference implementation test suites#98
ucekmez wants to merge 1 commit into
fix/reference-impl-delivery-urlfrom
ci/test-reference-implementation

Conversation

@ucekmez

@ucekmez ucekmez commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Summary

PR 6 of a stacked series. Base is #97. Not for merge without review.

Both reference stacks ship test suites — Node vitest, Python pytest with a 100% coverage gate — and no CI job runs either of them.

.github/workflows/test.yml covers every @eep-dev/* package, every eep-*-python package, and cross-impl. It skips examples/eep-reference-implementation/ entirely.

The README describes that directory as the stack

used by contributors and CI to exercise Layer 1 discovery, Layer 2 subscribe/stream, Layer 3 pulse and the gate endpoints against shared parity fixtures

CI did not use it. The delivery_url bug fixed in #97 shipped undetected as a direct result — a handler reading a request field that the schema does not define, in the stack implementors are pointed at as the worked example.

What changed

Two jobs: test-reference-implementation-node and test-reference-implementation-python.

The Python job installs the workspace packages from source before the app's own dependencies. This matters more than it looks:

  • Without it, the job tests whatever eep-gates currently resolves from PyPI, not the tree the PR changes — so a regression would surface only after a release.
  • It also removes an environment-dependent failure mode I hit while working on fix(examples): reference stacks read delivery_url, the field the schema defines #97: a stale eep_gates in a local site-packages made the suite fail for reasons that had nothing to do with the change under review, and I initially mis-reported that as a parity defect. Pinning to workspace sources makes the job's verdict about the tree and nothing else.

Scope

  • Spec / schema only
  • TypeScript package(s)
  • Python package(s)
  • Tests / CI
  • Docs / examples

Checklist

  • I read CONTRIBUTING.md and CODE_OF_CONDUCT.md.
  • Tests added or updated where appropriate (this PR is the test wiring).
  • Breaking change? No. CI-only; adds two jobs, changes no product code.
  • Documentation updated for user-visible behavior.

Verification

Both jobs reproduced locally from clean environments, running the exact commands in the workflow:

Job Result
test-reference-implementation-node 18 passed
test-reference-implementation-python 27 passed, app.py 164 stmts, 100% coverage (gate satisfied)

YAML validated with yaml.safe_load — 24 jobs parse.

Notes for reviewers

These jobs pass only on top of #97. On main today the Python suite fails, because that is the branch where the delivery_url fix and its regression tests live. Adding the jobs before the fix would have turned main red; that ordering is deliberate.

I did not add a coverage gate to the Node job — the Python one has --cov-fail-under=100 from its own pytest.ini, and the Node stack has no equivalent configured today. Raising one felt like a separate decision rather than something to slip into a CI-wiring PR.

Both reference stacks ship test suites — Node vitest, Python pytest with
a 100% coverage gate — and no CI job ran either of them. The workflow
covers every `@eep-dev/*` and `eep-*-python` package plus cross-impl,
but skipped the stacks in `examples/eep-reference-implementation/`.

README describes that directory as the stack "used by contributors and
CI to exercise Layer 1 discovery, Layer 2 subscribe/stream, Layer 3
pulse and the gate endpoints against shared parity fixtures". CI did not
use it. A subscribe handler reading `callback_url` when the schema
defines `delivery_url` shipped undetected as a direct result.

The Python job deliberately installs the workspace packages from source
before the app's own dependencies. Without that, the job tests whatever
`eep-gates` currently resolves from PyPI rather than the tree the PR
changes, so a regression would surface only after a release — and a
stale local copy can make the suite fail for reasons that have nothing
to do with the change under review.

Verified by reproducing both jobs locally from clean environments:
Node 18 passed, Python 27 passed at 100% coverage.

Refs: EEP audit 2026-08, follow-on from finding A1
Signed-off-by: Ugur Cekmez <ucekmez@gmail.com>
Copilot AI lite review requested due to automatic review settings August 26, 2026 18:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants