Skip to content

feat(spec,middleware): event history, redelivery and delivery log for webhooks - #102

Open
ucekmez wants to merge 1 commit into
feat/layer1-conditional-requestsfrom
feat/webhook-backfill-redelivery
Open

feat(spec,middleware): event history, redelivery and delivery log for webhooks#102
ucekmez wants to merge 1 commit into
feat/layer1-conditional-requestsfrom
feat/webhook-backfill-redelivery

Conversation

@ucekmez

@ucekmez ucekmez commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Summary

PR 10 of a stacked series. Base is #101. Not for merge without review.

The three layers had wildly asymmetric reliability:

Layer Catch-up mechanism
Layer 2 SSE Last-Event-ID, ≥24h retention (§4.3)
Layer 3 WS system/replay from seq, history with cursor pagination (§6.3)
Layer 2 webhooks none

This is load-bearing rather than theoretical. §10 moves a subscription to paused after repeated delivery failures — precisely when the subscriber's endpoint was down and it missed the most. Nothing specified what happened to events emitted during the outage, so resuming produced a silent hole: the subscription works again and the events are simply gone.

Separately, delivery_guarantees.md §4 already mandated GET /eep/subscriptions/:id/delivery-log with 30-day retention. The normative spec never mentioned it and nothing implemented it — so a subscriber could not distinguish "the publisher never sent it" from "my endpoint rejected it", which is unanswerable from its own side.

What changed

Spec — new §5.1.2

  • GET /eep/events with source / since / until / limit, ordered by emission, retention floor matching SSE replay. next_cursor present only while events remain, so its absence is how a subscriber knows it is caught up.
  • A since outside retention MUST produce 410 Gone naming the oldest retained id. Explicitly not an empty page — an empty page is indistinguishable from "you are up to date", which would let a subscriber believe it caught up while silently losing events.
  • History is scoped to the caller and gates still apply, so catch-up cannot become a way to read events the caller could never have subscribed to.
  • POST /eep/subscriptions/{id}/redeliver. Redelivered events keep their original id, so an already-processed event is discarded by the ordinary idempotency rule rather than processed twice. Re-signed with a current timestamp; does not reset the failure counter.
  • Delivery log promoted into the normative spec.
  • §13's "Event stream history queries" quota now names a defined endpoint. It previously metered nothing (audit finding A10 — fixed here rather than separately, since it was the same missing endpoint).

Middleware

  • EventStore interface + in-memory implementation: history, redelivery lookup, delivery log, with independent retention windows for events (24h floor) and attempts (30d floor).
  • Three new routes; dispatcher records every attempt.
  • Log writes are wrapped in try/catchobservability must never break delivery.
  • undeliverable is reserved for the final attempt of an exhausted schedule; an interim failure is failed. Conflating them would make the log read as though every retry were fatal.

Scope

  • Spec / schema only
  • TypeScript package(s)
  • Python package(s)
  • Tests / CI
  • Docs / examples

Checklist

  • I read CONTRIBUTING.md and CODE_OF_CONDUCT.md.
  • Tests added or updated where appropriate.
  • Breaking change? Additive. New endpoints and a new optional eventStore option that defaults to an in-process store. Publishers that implement none of it will newly fail the §5.1.2 requirements, which is the point.
  • Documentation updated for user-visible behavior.

Verification

Suite Result
@eep-dev/middleware 186 passed (was 158)
tests/ 184 passed
@eep-dev/compliance-cli 73 passed
codegen-schema-types --check no drift

Notes for reviewers

The in-memory store is a reference, not a recommendation. It is correct for a single process and for tests; a deployment that must survive a restart backs the EventStore interface with the same storage its subscriptions live in. §5.1.2 requires the behaviour, not this class — the interface is the contract.

Two deliberate omissions, flagged rather than skipped silently:

  • No conformance probe for /eep/events yet. Meaningfully probing catch-up needs a publisher that has emitted events the runner can name, which the current probe flow does not set up.
  • No Python middleware parity for the three handlers.

I did not add a delivery-log response schema under schemas/v0.1/. The field list lives in delivery_guarantees.md §4 and I did not want to fork it into a second normative source in the same PR that promotes the endpoint. Worth doing — say the word and I'll add it with the codegen refresh.

… webhooks

SSE subscribers recover missed events with `Last-Event-ID` (§4.3, 24h
retention floor). Layer 3 clients recover with `system`/`replay` from a
`seq` (§6.3.1). Webhook subscribers had no equivalent at all.

That gap is load-bearing rather than theoretical. §10 moves a
subscription to `paused` after repeated delivery failures — precisely
when the subscriber's endpoint was down and it missed the most. Nothing
specified what happened to events emitted during the outage, so resuming
produced a silent hole: the subscription works again and the events are
simply gone.

`delivery_guarantees.md` §4 already mandated `GET
/eep/subscriptions/:id/delivery-log` with 30-day retention. The
normative spec never mentioned it and nothing implemented it, so a
subscriber could not distinguish "the publisher never sent it" from "my
endpoint rejected it" — unanswerable from its side.

Spec §5.1.2:
- `GET /eep/events` with `source` / `since` / `until` / `limit`, ordered
  by emission, retention floor matching SSE replay. `next_cursor` is
  present only while events remain, so its absence is how a subscriber
  knows it is caught up.
- A `since` outside the retention window MUST produce `410 Gone` naming
  the oldest retained id. Explicitly NOT an empty page: an empty page is
  indistinguishable from "you are up to date", which would let a
  subscriber believe it caught up while silently losing events.
- History is scoped to the caller and gates still apply, so catch-up
  cannot be used to read events the caller could never have subscribed
  to.
- `POST /eep/subscriptions/{id}/redeliver`. Redelivered events keep
  their original `id` so an already-processed event is discarded by the
  ordinary idempotency rule instead of being processed twice; they are
  re-signed with a current timestamp and do not reset the failure
  counter.
- The delivery log is promoted into the normative spec.
- §13's "Event stream history queries" quota now names a defined
  endpoint. It previously metered nothing (audit finding A10).

Middleware: `EventStore` interface plus an in-memory implementation
covering history, redelivery lookup, and the delivery log with independent
retention windows; three new routes; and the dispatcher records every
attempt. Log writes are wrapped so observability can never break
delivery, and `undeliverable` is reserved for the final attempt of an
exhausted schedule — conflating it with an interim `failed` would make
the log read as though every retry were fatal.

Refs: EEP audit 2026-08 findings B1, A10
Signed-off-by: Ugur Cekmez <ucekmez@gmail.com>
Copilot AI lite review requested due to automatic review settings August 26, 2026 19:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants