From d000020c84d1286106d938c8361a3eecceb3b1d6 Mon Sep 17 00:00:00 2001 From: Ugur Cekmez Date: Wed, 26 Aug 2026 22:51:09 +0300 Subject: [PATCH] feat(spec,middleware): CloudEvents binary content mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit EEP delivered only in CloudEvents structured mode: the whole envelope as one JSON document. Binary mode — where context attributes travel as protocol metadata and the body carries only `data` — was not offered, so every subscriber parsed a full envelope to decide whether it wanted the payload. Honest accounting, because the audit that raised this overstated it. Measured against the §4.2 example, binary mode is ~19% smaller per SSE frame and ~13% per webhook over HTTP/1.1. The earlier "62%" figure was computed by dropping `source`, `time`, `specversion` and `datacontenttype` rather than relocating them, which is not a like-for-like comparison. The body alone shrinks ~80%, and under HTTP/2 the `ce-*` header names and the attributes that do not vary between deliveries are indexed by HPACK, so the saving grows with sustained traffic. The more reliable benefit is that a subscriber can route and filter on headers without parsing the body at all. §5.2.1 says so plainly, and points publishers optimising bytes at conditional requests (§3.2.1), compression (§3.2.2) and subscription filters (§5.1.3) first — those remove whole responses and whole deliveries rather than shrinking them. This also resolves, for the binary path, the CloudEvents attribute naming divergence recorded in §7: CloudEvents excludes the underscore from context attribute names, so `ce-eep_version` is not a legal header. In binary mode the underscores are removed (`eep_version` → `ce-eepversion`). Structured mode keeps the underscored spelling that every deployed implementation emits, so nothing breaks. - `delivery_format` gains `cloudevents/v1.0-binary`; structured stays the default. - SSE binary mode relocates only the two attributes SSE natively frames (`id:`, `event:`) and stops repeating them in the body; SSE has no per-event header channel for the rest. - Binary mode relocates attributes and MUST NOT drop them. An object-valued attribute has no header representation, so the renderer keeps it in the body rather than losing it — with a test asserting every attribute survives the mode change. - Signing is unchanged: the signature covers the raw body bytes, so a binary-mode subscriber verifies exactly what it received without reassembling an envelope. Refs: EEP audit 2026-08 finding O1 Signed-off-by: Ugur Cekmez --- docs/current/SPECIFICATION.md | 78 +++++++++++- .../middleware/src/core/eep-server.ts | 3 + .../middleware/src/core/request-handler.ts | 8 ++ .../src/dispatcher/content-mode.test.ts | 114 ++++++++++++++++++ .../middleware/src/dispatcher/content-mode.ts | 68 +++++++++++ .../src/dispatcher/webhook-dispatcher.test.ts | 40 ++++++ .../src/dispatcher/webhook-dispatcher.ts | 8 +- packages/@eep-dev/middleware/src/index.ts | 5 +- schemas/v0.1/subscription.request.json | 5 +- tests/types/eep-schemas.d.ts | 4 +- 10 files changed, 324 insertions(+), 9 deletions(-) create mode 100644 packages/@eep-dev/middleware/src/dispatcher/content-mode.test.ts create mode 100644 packages/@eep-dev/middleware/src/dispatcher/content-mode.ts diff --git a/docs/current/SPECIFICATION.md b/docs/current/SPECIFICATION.md index d816022..a7ebc6b 100644 --- a/docs/current/SPECIFICATION.md +++ b/docs/current/SPECIFICATION.md @@ -602,6 +602,77 @@ EEP-Version: 0.1 } ``` +### 5.2.1 Content modes (normative) + +EEP deliveries default to CloudEvents **structured** content mode: the whole +envelope is one JSON document in the body. CloudEvents also defines **binary** +content mode, where context attributes travel as protocol metadata and the +body carries only `data`. Publishers MAY offer binary mode; subscribers select +it with `delivery_format` on the subscription request. + +**Webhooks.** Binary mode follows the CloudEvents HTTP Protocol Binding: +each attribute becomes a `ce-`-prefixed header and the body is the raw `data`. + +```http +POST /hooks/eep +ce-specversion: 1.0 +ce-id: 01HN3QK7GX-1708123456000 +ce-source: did:web:example.com:u:acme-corp +ce-type: com.example.entity.updated +ce-time: 2026-02-22T14:30:00Z +ce-eepversion: 0.1 +content-type: application/json +webhook-signature: v1,… + +{"field":"bio","previous":"Old bio","current":"New bio"} +``` + +**Attribute names in binary mode.** CloudEvents restricts context attribute +names to lowercase ASCII letters and digits — the underscore is excluded. EEP's +`eep_`-prefixed attributes therefore cannot be carried verbatim as `ce-` +headers. In binary mode the underscores are removed: `eep_version` becomes +`ce-eepversion`, `eep_subscription_id` becomes `ce-eepsubscriptionid`, and so +on. Structured mode is unchanged and keeps the underscored spelling, which is +what every deployed implementation emits. This resolves, for the binary path +only, the naming divergence recorded in §7. + +**SSE.** SSE has no per-event header channel, so only the two attributes SSE +natively frames are relocated: `id` moves to the `id:` field and `type` to the +`event:` field, and the `data:` payload omits both rather than repeating them. +Every other attribute stays in the JSON document. + +``` +id: 01HN3QK7GX-1708123456000 +event: com.example.entity.updated +data: {"specversion":"1.0","source":"did:web:example.com:u:acme-corp","time":"2026-02-22T14:30:00Z","datacontenttype":"application/json","eep_version":"0.1","data":{…}} + +``` + +**Signing is unaffected.** The signature is computed over the raw body bytes +exactly as in §5.3. In binary mode the body is the `data` document, so a +subscriber verifies what it received without reassembling an envelope first. + +**Requirements.** + +- A publisher that advertises binary mode MUST populate every attribute it + would have sent in structured mode. Binary mode relocates attributes; it + MUST NOT drop them. +- Subscribers MUST reject a binary-mode delivery missing `ce-specversion`, + `ce-id`, `ce-source` or `ce-type`. +- Publishers MUST default to structured mode when the subscriber did not ask + for binary. + +**What this is worth.** Measured against the §4.2 example, binary mode is +roughly **19% smaller** per SSE frame and **13%** per webhook over HTTP/1.1 — +useful, not transformative. The body alone shrinks ~80%, and under HTTP/2 the +`ce-*` header names and the attributes that do not vary between deliveries are +indexed by HPACK, so the saving grows with sustained traffic on a connection. +The more reliable benefit is that a subscriber can route and filter on headers +without parsing the body at all. Publishers optimising bytes should reach for +conditional requests (§3.2.1), compression (§3.2.2) and subscription filters +(§5.1.3) first: those remove whole responses and whole deliveries rather than +shrinking them. + ### 5.3 Webhook signature verification The `webhook-signature` header contains an HMAC-SHA256 signature over the concatenation of: @@ -868,8 +939,11 @@ All EEP events MUST be valid CloudEvents v1.0.2 envelopes with EEP-specific exte > excluded. The `eep_`-prefixed names above are what every deployed > implementation emits, but they do not satisfy that rule. The divergence is > harmless in structured mode and becomes load-bearing in binary content mode, -> where attributes are carried as `ce-`-prefixed HTTP headers. Tracked for -> resolution before v1.0; see the editor's note in +> where attributes are carried as `ce-`-prefixed HTTP headers. §5.2.1 resolves +> it for the binary path by removing the underscores (`eep_version` → +> `ce-eepversion`); structured mode keeps the underscored spelling that every +> deployed implementation emits. Whether to rename in structured mode too is +> still open before v1.0; see the editor's note in > [`draft-eep-protocol-core-00.md`](../standards/draft-eep-protocol-core-00.md). ### 7.1 Standard CloudEvents attributes (normative) diff --git a/packages/@eep-dev/middleware/src/core/eep-server.ts b/packages/@eep-dev/middleware/src/core/eep-server.ts index 2f65427..bb00ea6 100644 --- a/packages/@eep-dev/middleware/src/core/eep-server.ts +++ b/packages/@eep-dev/middleware/src/core/eep-server.ts @@ -430,6 +430,9 @@ export class EEPServer { failure_count: 0, expires_at: expiresAt.toISOString(), ...(filter ? { filter } : {}), + ...(body.delivery_format === "cloudevents/v1.0-binary" + ? { delivery_format: "cloudevents/v1.0-binary" as const } + : {}), delivery_secret: deliverySecret, metadata, tier, diff --git a/packages/@eep-dev/middleware/src/core/request-handler.ts b/packages/@eep-dev/middleware/src/core/request-handler.ts index 91eb8c7..257e127 100644 --- a/packages/@eep-dev/middleware/src/core/request-handler.ts +++ b/packages/@eep-dev/middleware/src/core/request-handler.ts @@ -63,6 +63,9 @@ export const DEFAULT_LEASE_SECONDS = 2_592_000; export const MIN_LEASE_SECONDS = 300; export const MAX_LEASE_SECONDS = 31_536_000; +/** CloudEvents content mode (SPECIFICATION.md §5.2.1). */ +export type DeliveryFormat = "cloudevents/v1.0" | "cloudevents/v1.0-binary"; + export type SubscriptionRecord = { subscription_id: string; source_did: string; @@ -86,6 +89,11 @@ export type SubscriptionRecord = { * `event_types` selected; never widens it. */ filter?: EventFilter; + /** + * CloudEvents content mode for this subscription (SPECIFICATION.md §5.2.1). + * Defaults to structured. + */ + delivery_format?: DeliveryFormat; /** Requested access tier; matched against gate config on delivery. */ tier?: string; created_at: string; diff --git a/packages/@eep-dev/middleware/src/dispatcher/content-mode.test.ts b/packages/@eep-dev/middleware/src/dispatcher/content-mode.test.ts new file mode 100644 index 0000000..aa3bfb3 --- /dev/null +++ b/packages/@eep-dev/middleware/src/dispatcher/content-mode.test.ts @@ -0,0 +1,114 @@ +import { describe, expect, it } from "vitest"; +import { renderDelivery, toBinaryAttributeName } from "./content-mode.js"; +import type { CloudEvent } from "../core/request-handler.js"; + +const event = (overrides: Partial = {}): CloudEvent => + ({ + specversion: "1.0", + id: "01HN3QK7GX", + source: "did:web:acme.example", + type: "com.example.entity.updated", + time: "2026-02-22T14:30:00Z", + datacontenttype: "application/json", + eep_version: "0.1", + data: { field: "bio", current: "New bio" }, + ...overrides, + }) as CloudEvent; + +describe("toBinaryAttributeName", () => { + // CloudEvents restricts context attribute names to lowercase letters and + // digits, so `ce-eep_version` is not a legal header name. + it.each([ + ["eep_version", "eepversion"], + ["eep_subscription_id", "eepsubscriptionid"], + ["specversion", "specversion"], + ["EEP_Version", "eepversion"], + ])("maps %s to %s", (input, expected) => { + expect(toBinaryAttributeName(input)).toBe(expected); + }); +}); + +describe("structured mode (default)", () => { + it("puts the whole envelope in the body", () => { + const evt = event(); + const { body, headers } = renderDelivery(evt, undefined); + expect(JSON.parse(body)).toEqual(evt); + expect(headers["content-type"]).toBe("application/json"); + }); + + it("keeps the underscored attribute spelling", () => { + const { body } = renderDelivery(event(), "cloudevents/v1.0"); + expect(JSON.parse(body)).toHaveProperty("eep_version", "0.1"); + }); + + it("emits no ce-* headers", () => { + const { headers } = renderDelivery(event(), "cloudevents/v1.0"); + expect(Object.keys(headers).filter((h) => h.startsWith("ce-"))).toEqual([]); + }); +}); + +describe("binary mode (§5.2.1)", () => { + it("carries the payload alone in the body", () => { + const { body } = renderDelivery(event(), "cloudevents/v1.0-binary"); + expect(JSON.parse(body)).toEqual({ field: "bio", current: "New bio" }); + }); + + it("relocates context attributes to ce-* headers", () => { + const { headers } = renderDelivery(event(), "cloudevents/v1.0-binary"); + expect(headers["ce-specversion"]).toBe("1.0"); + expect(headers["ce-id"]).toBe("01HN3QK7GX"); + expect(headers["ce-source"]).toBe("did:web:acme.example"); + expect(headers["ce-type"]).toBe("com.example.entity.updated"); + expect(headers["ce-time"]).toBe("2026-02-22T14:30:00Z"); + }); + + it("strips underscores from EEP attribute names", () => { + const { headers } = renderDelivery( + event({ eep_subscription_id: "sub_1" } as Partial), + "cloudevents/v1.0-binary" + ); + expect(headers["ce-eepversion"]).toBe("0.1"); + expect(headers["ce-eepsubscriptionid"]).toBe("sub_1"); + // The illegal spelling must never appear. + expect(headers["ce-eep_version"]).toBeUndefined(); + }); + + // Binary mode RELOCATES attributes; it must not drop them. An object-valued + // attribute has no header representation, so it stays in the body. + it("keeps object-valued attributes in the body rather than losing them", () => { + const { body, headers } = renderDelivery( + event({ eep_known_event_types: ["a", "b"] } as Partial), + "cloudevents/v1.0-binary" + ); + const parsed = JSON.parse(body) as Record; + expect(parsed.eep_known_event_types).toEqual(["a", "b"]); + expect(parsed.data).toEqual({ field: "bio", current: "New bio" }); + expect(headers["ce-eepknowneventtypes"]).toBeUndefined(); + }); + + it("loses no attribute across the mode change", () => { + const evt = event(); + const { body, headers } = renderDelivery(evt, "cloudevents/v1.0-binary"); + const parsed = JSON.parse(body) as Record; + for (const attribute of Object.keys(evt)) { + if (attribute === "data") continue; + const inHeader = headers[`ce-${toBinaryAttributeName(attribute)}`] !== undefined; + const inBody = Object.prototype.hasOwnProperty.call(parsed, attribute); + expect(inHeader || inBody).toBe(true); + } + }); + + it("handles an event with no data", () => { + const evt = event(); + delete (evt as { data?: unknown }).data; + const { body } = renderDelivery(evt, "cloudevents/v1.0-binary"); + expect(JSON.parse(body)).toEqual({}); + }); + + it("produces a smaller body than structured mode", () => { + const evt = event(); + const structured = renderDelivery(evt, "cloudevents/v1.0"); + const binary = renderDelivery(evt, "cloudevents/v1.0-binary"); + expect(binary.body.length).toBeLessThan(structured.body.length); + }); +}); diff --git a/packages/@eep-dev/middleware/src/dispatcher/content-mode.ts b/packages/@eep-dev/middleware/src/dispatcher/content-mode.ts new file mode 100644 index 0000000..ca9ceeb --- /dev/null +++ b/packages/@eep-dev/middleware/src/dispatcher/content-mode.ts @@ -0,0 +1,68 @@ +/** + * CloudEvents content-mode rendering (SPECIFICATION.md §5.2.1). + * + * Structured mode puts the whole envelope in the body. Binary mode relocates + * context attributes to `ce-`-prefixed headers and leaves only `data` in the + * body, so a subscriber can route and filter on headers without parsing the + * payload at all. + */ +import type { CloudEvent, DeliveryFormat } from "../core/request-handler.js"; + +/** Attributes that are part of the envelope rather than the payload. */ +const RESERVED = new Set(["data"]); + +/** + * CloudEvents restricts context attribute names to lowercase letters and + * digits — the underscore is excluded — so EEP's `eep_`-prefixed attributes + * cannot be carried verbatim as `ce-` headers. In binary mode the underscores + * are removed (`eep_version` → `ce-eepversion`). Structured mode keeps the + * underscored spelling, which is what every deployed implementation emits. + */ +export function toBinaryAttributeName(attribute: string): string { + return attribute.replace(/_/g, "").toLowerCase(); +} + +export interface RenderedDelivery { + body: string; + /** Headers the content mode contributes. Signing headers are added separately. */ + headers: Record; +} + +/** + * Render an event for delivery in the subscription's content mode. + * + * Binary mode relocates attributes; it never drops them. Any attribute that + * cannot be represented as a header value (an object or array) stays in + * structured form for that delivery rather than being silently lost. + */ +export function renderDelivery(event: CloudEvent, format: DeliveryFormat | undefined): RenderedDelivery { + if (format !== "cloudevents/v1.0-binary") { + return { + body: JSON.stringify(event), + headers: { "content-type": "application/json" }, + }; + } + + const headers: Record = { "content-type": "application/json" }; + const leftovers: Record = {}; + + for (const [attribute, value] of Object.entries(event)) { + if (RESERVED.has(attribute)) continue; + if (value === undefined || value === null) continue; + if (typeof value === "object") { + // Structured attribute values have no header representation. + // Keeping them in the body is lossless; dropping them would not be. + leftovers[attribute] = value; + continue; + } + headers[`ce-${toBinaryAttributeName(attribute)}`] = String(value); + } + + const data = (event as { data?: unknown }).data; + const body = + Object.keys(leftovers).length > 0 + ? JSON.stringify({ ...leftovers, data }) + : JSON.stringify(data ?? {}); + + return { body, headers }; +} diff --git a/packages/@eep-dev/middleware/src/dispatcher/webhook-dispatcher.test.ts b/packages/@eep-dev/middleware/src/dispatcher/webhook-dispatcher.test.ts index 6d652dc..ac582ac 100644 --- a/packages/@eep-dev/middleware/src/dispatcher/webhook-dispatcher.test.ts +++ b/packages/@eep-dev/middleware/src/dispatcher/webhook-dispatcher.test.ts @@ -59,6 +59,46 @@ describe("WebhookDispatcher", () => { expect(DEFAULT_RETRY_SCHEDULE_MS).toEqual([0, 5_000, 30_000, 120_000, 900_000, 3_600_000, 21_600_000]); }); + // SPECIFICATION.md §5.2.1 — binary content mode relocates attributes to + // ce-* headers so a subscriber can route without parsing the body. + describe("content modes (§5.2.1)", () => { + const deliverAs = async (format?: "cloudevents/v1.0" | "cloudevents/v1.0-binary") => { + const db = new InMemoryDBAdapter(); + await db.saveSubscription(subscription(format ? { delivery_format: format } : {})); + const { client, calls } = mockClient([200]); + const dispatcher = new WebhookDispatcher({ db, httpClient: client, retryScheduleMs: NO_DELAY }); + await dispatcher.dispatch(event()); + return calls[0]!; + }; + + it("defaults to structured mode", async () => { + const call = await deliverAs(); + expect(JSON.parse(call.body)).toMatchObject({ id: "evt_1", type: "entity.updated" }); + expect(call.headers["ce-id"]).toBeUndefined(); + }); + + it("delivers binary mode with ce-* headers and a payload-only body", async () => { + const call = await deliverAs("cloudevents/v1.0-binary"); + expect(call.headers["ce-id"]).toBe("evt_1"); + expect(call.headers["ce-type"]).toBe("entity.updated"); + expect(JSON.parse(call.body)).toEqual({ changed: true }); + }); + + // The signature covers the raw body bytes, so a binary-mode subscriber + // verifies exactly what it received without reassembling an envelope. + it("signs the body it actually sends in binary mode", async () => { + const call = await deliverAs("cloudevents/v1.0-binary"); + expect( + new EEPSigner(SECRET).verify( + call.headers["webhook-id"]!, + call.headers["webhook-timestamp"]!, + call.headers["webhook-signature"]!, + call.body + ) + ).toBe(true); + }); + }); + // SPECIFICATION.md §5.3.1 — HMAC proves only that someone holding the // shared secret sent the event, and the subscriber is one of them. Ed25519 // makes deliveries attributable and verifiable by third parties. diff --git a/packages/@eep-dev/middleware/src/dispatcher/webhook-dispatcher.ts b/packages/@eep-dev/middleware/src/dispatcher/webhook-dispatcher.ts index bc104b0..5bea863 100644 --- a/packages/@eep-dev/middleware/src/dispatcher/webhook-dispatcher.ts +++ b/packages/@eep-dev/middleware/src/dispatcher/webhook-dispatcher.ts @@ -3,6 +3,7 @@ import { matchesAnyPattern } from "@eep-dev/validator"; import { TEST_DELIVERY_EVENT_TYPE } from "../core/request-handler.js"; import type { EventStore } from "../core/event-store.js"; import { eventMatchesFilter } from "../core/event-filter.js"; +import { renderDelivery } from "./content-mode.js"; import type { CloudEvent, DBAdapter, @@ -314,7 +315,10 @@ export class WebhookDispatcher { return { ok: false }; } - const body = JSON.stringify(event); + // §5.2.1 — render in the subscription's content mode. The signature is + // computed over whatever body this produces, so a binary-mode subscriber + // verifies exactly what it received without reassembling an envelope. + const { body, headers: contentHeaders } = renderDelivery(event, sub.delivery_format); // Stable across retries so subscribers can deduplicate; the event `id` // is the idempotency key per delivery_guarantees.md §1. const webhookId = `msg_${event.id}`; @@ -347,7 +351,7 @@ export class WebhookDispatcher { try { const res = await this.httpClient(url, { headers: { - "content-type": "application/json", + ...contentHeaders, "webhook-id": webhookId, "webhook-timestamp": timestamp, "webhook-signature": signature, diff --git a/packages/@eep-dev/middleware/src/index.ts b/packages/@eep-dev/middleware/src/index.ts index 0e59d51..f6b5aa5 100644 --- a/packages/@eep-dev/middleware/src/index.ts +++ b/packages/@eep-dev/middleware/src/index.ts @@ -10,7 +10,8 @@ export type { RequestHandler, RouteDefinition, SubscriptionRecord, - SubscriptionUpdate + SubscriptionUpdate, + DeliveryFormat } from "./core/request-handler.js"; export { @@ -47,6 +48,8 @@ export { InMemoryEventBusAdapter } from "./event-bus/in-memory.js"; export { RedisEventBusAdapter, type RedisClientLike } from "./event-bus/redis.js"; export { KafkaEventBusAdapter, type KafkaProducerLike, type KafkaConsumerLike } from "./event-bus/kafka.js"; +export { renderDelivery, toBinaryAttributeName, type RenderedDelivery } from "./dispatcher/content-mode.js"; + export { validateFilter, eventMatchesFilter, diff --git a/schemas/v0.1/subscription.request.json b/schemas/v0.1/subscription.request.json index 87409bc..43813f1 100644 --- a/schemas/v0.1/subscription.request.json +++ b/schemas/v0.1/subscription.request.json @@ -117,9 +117,10 @@ }, "delivery_format": { "type": "string", - "description": "The event envelope format for delivery. Defaults to CloudEvents v1.0.", + "description": "The event envelope format for delivery. `cloudevents/v1.0` is structured mode: the whole envelope is one JSON document in the body. `cloudevents/v1.0-binary` is binary content mode, where context attributes travel as protocol metadata and the body carries only `data` (SPECIFICATION.md \u00a75.2.1). Defaults to structured.", "enum": [ - "cloudevents/v1.0" + "cloudevents/v1.0", + "cloudevents/v1.0-binary" ], "default": "cloudevents/v1.0" }, diff --git a/tests/types/eep-schemas.d.ts b/tests/types/eep-schemas.d.ts index 3faf703..0f5b8c7 100644 --- a/tests/types/eep-schemas.d.ts +++ b/tests/types/eep-schemas.d.ts @@ -5491,9 +5491,9 @@ export interface EEPSubscriptionRequest { */ delivery_url?: string; /** - * The event envelope format for delivery. Defaults to CloudEvents v1.0. + * The event envelope format for delivery. `cloudevents/v1.0` is structured mode: the whole envelope is one JSON document in the body. `cloudevents/v1.0-binary` is binary content mode, where context attributes travel as protocol metadata and the body carries only `data` (SPECIFICATION.md §5.2.1). Defaults to structured. */ - delivery_format?: 'cloudevents/v1.0'; + delivery_format?: 'cloudevents/v1.0' | 'cloudevents/v1.0-binary'; /** * Optional. Requested subscription lifetime in seconds, starting from successful intent verification (SPECIFICATION.md §10.2). The publisher MAY clamp this to its own policy and reports the value actually granted as `expires_at` on the subscription. Omit to accept the publisher's default lease. */