From 9412408679b75b4a7f5b5f32c502a624f3a34784 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 6 Nov 2026 10:35:32 +0000 Subject: [PATCH 1/3] fix(eql): drop ignored query_json cast and correct eql-bindings wire docs PostgreSQL ignores casts whose source type is a domain, so the json_search -> query_json cast never ran to_ste_vec_query. Remove it and document the function as the only conversion. Fix the v3 module docs: op backs _ord/_ord_ope, ob only _ord_ore. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_012Jmn7qx6btMrkTQ6rQRjDz --- .changeset/eql-drop-ignored-query-json-cast.md | 5 +++++ packages/eql/crates/eql-bindings/src/v3/mod.rs | 10 +++++----- packages/eql/docs/reference/json-support.md | 2 +- packages/eql/src/v3/json/types.sql | 9 ++++++--- .../eql/tests/sqlx/snapshots/eql_v3_public_surface.txt | 1 - 5 files changed, 17 insertions(+), 10 deletions(-) create mode 100644 .changeset/eql-drop-ignored-query-json-cast.md diff --git a/.changeset/eql-drop-ignored-query-json-cast.md b/.changeset/eql-drop-ignored-query-json-cast.md new file mode 100644 index 0000000000..c9cb12b546 --- /dev/null +++ b/.changeset/eql-drop-ignored-query-json-cast.md @@ -0,0 +1,5 @@ +--- +'@cipherstash/eql': patch +--- + +Drop the `CREATE CAST (public.eql_v3_json_search AS eql_v3.query_json)` that PostgreSQL always ignored (it warned "cast will be ignored because the source data type is a domain" on every install). `doc::eql_v3.query_json` was never converting; call `eql_v3.to_ste_vec_query(col)` instead. Also corrects the `eql-bindings` wire-format docs: `op` backs `_ord` / `_ord_ope` (and `text_ord` / `text_search` carry `hm` + `op`), `ob` only `_ord_ore`. diff --git a/packages/eql/crates/eql-bindings/src/v3/mod.rs b/packages/eql/crates/eql-bindings/src/v3/mod.rs index a230940561..7fddfd0936 100644 --- a/packages/eql/crates/eql-bindings/src/v3/mod.rs +++ b/packages/eql/crates/eql-bindings/src/v3/mod.rs @@ -23,7 +23,7 @@ //! For the generated flat-scalar families, every payload has the shared envelope //! keys (required by every scalar domain CHECK, mirroring `ENVELOPE_KEYS` in //! `eql-domains`): `v`, `i`, `c`. Then the domain's required term keys — `hm` -//! for `_eq`, `ob` for `_ord`/`_ord_ore`, `op` for `_ord_ope`, `bf` for +//! for `_eq`, `op` for `_ord`/`_ord_ope`, `ob` for `_ord_ore`, `bf` for //! `_match`, none for //! storage-only. `Option` does not appear in the generated scalar structs: the //! capability **is** the type identity. Hold a [`integer::IntegerEq`] and `hm` is @@ -31,14 +31,14 @@ //! missing term key is a deserialization error — the Rust analogue of the CHECK //! constraint. //! -//! One exception to "`ob` for `_ord`": `text`'s ordered domains carry **both** -//! `hm` and the ordering term (`text_ord`/`text_ord_ore`/`text_search` carry -//! `hm` + `ob`; `text_ord_ope` carries `hm` + `op`), where the non-text +//! One exception to "the ordering term alone": `text`'s ordered domains carry **both** +//! `hm` and the ordering term (`text_ord`/`text_ord_ope`/`text_search` carry +//! `hm` + `op`; `text_ord_ore`/`text_search_ore` carry `hm` + `ob`), where the non-text //! ordered domains carry the ordering term alone. Text routes `=`/`<>` //! through `hm` rather than the ordering term because lexicographic ORE/OPE //! over text is not equality-lossless, so equality needs the HMAC. The //! generated struct doc surfaces this structurally — its required-keys line -//! lists `hm` `ob` (or `hm` `op`) rather than just the ordering key. +//! lists `hm` `op` (or `hm` `ob`) rather than just the ordering key. //! //! The generated flat-scalar types are also **strict**: every scalar struct is //! `#[serde(deny_unknown_fields)]`, so a payload carrying keys outside the diff --git a/packages/eql/docs/reference/json-support.md b/packages/eql/docs/reference/json-support.md index aabbce99ac..1ee756c131 100644 --- a/packages/eql/docs/reference/json-support.md +++ b/packages/eql/docs/reference/json-support.md @@ -214,7 +214,7 @@ separate equality-indexed scalar column when server-side grouping is required. - **`eql_v3.jsonb_document_contains(a public.eql_v3_json_search, b public.eql_v3_json_search) RETURNS boolean`** — true if every selector in `b` is present in `a` (selector-subset containment); backs the `@>` operator. - **`eql_v3.jsonb_contains(a jsonb, b jsonb)` / `eql_v3.jsonb_contained_by(a jsonb, b jsonb)`** — function-form containment entrypoints for platforms that cannot type an operator call (Supabase PostgREST RPC, for example); same result as `@>` / `<@` (a parity test pins this). - **`eql_v3.jsonb_array(val jsonb) RETURNS jsonb[]`** — function-form array accessor for the same platforms. -- **`eql_v3.to_ste_vec_query(val public.eql_v3_json_search) RETURNS eql_v3.query_json`** — the GIN-indexable query shape `@>` inlines to. +- **`eql_v3.to_ste_vec_query(val public.eql_v3_json_search) RETURNS eql_v3.query_json`** — the GIN-indexable query shape `@>` inlines to. This is the only document-to-needle conversion: `doc::eql_v3.query_json` is **not** equivalent (PostgreSQL ignores casts from a domain, so it is a plain coercion that fails the `query_json` CHECK). - **`eql_v3.meta_data(val jsonb)`** — envelope accessor: returns `{i, v, h}` (the key header `h` included so an extracted entry can be decrypted). **`eql_v3.ciphertext(val jsonb)`** — returns the `c` field; on the ste_vec surface `c` is raw AEAD output, decryptable only together with the entry's `s` and the document `h`. **`eql_v3.selector(val jsonb)` / `(entry public.eql_v3_json_entry)`** — selector accessor. ### Path query functions diff --git a/packages/eql/src/v3/json/types.sql b/packages/eql/src/v3/json/types.sql index 0bc34904b9..32ad2daa9f 100644 --- a/packages/eql/src/v3/json/types.sql +++ b/packages/eql/src/v3/json/types.sql @@ -292,6 +292,9 @@ AS $$ )::eql_v3.query_json $$; -CREATE CAST (public.eql_v3_json_search AS eql_v3.query_json) - WITH FUNCTION eql_v3.to_ste_vec_query(public.eql_v3_json_search) - AS ASSIGNMENT; +--! @note There is deliberately no `CREATE CAST` from `public.eql_v3_json_search` +--! to `eql_v3.query_json`. PostgreSQL ignores a cast whose source type is a +--! domain (it warns "cast will be ignored because the source data type is a +--! domain"), so `doc::eql_v3.query_json` would never call +--! `to_ste_vec_query`; it is a plain binary coercion that then fails the +--! `query_json` CHECK. Call `eql_v3.to_ste_vec_query(col)` explicitly. diff --git a/packages/eql/tests/sqlx/snapshots/eql_v3_public_surface.txt b/packages/eql/tests/sqlx/snapshots/eql_v3_public_surface.txt index 85c9bfaf6d..f7e7ac8ec1 100644 --- a/packages/eql/tests/sqlx/snapshots/eql_v3_public_surface.txt +++ b/packages/eql/tests/sqlx/snapshots/eql_v3_public_surface.txt @@ -59,7 +59,6 @@ aggregate eql_v3.min(public.eql_v3_text_search_ore) aggregate eql_v3.min(public.eql_v3_timestamp_ord) aggregate eql_v3.min(public.eql_v3_timestamp_ord_ope) aggregate eql_v3.min(public.eql_v3_timestamp_ord_ore) -cast public.eql_v3_json_search -> eql_v3.query_json function eql_v3.->(e public.eql_v3_json_search, selector integer) function eql_v3.->(e public.eql_v3_json_search, selector text) function eql_v3.->>(e public.eql_v3_json_search, selector integer) From 19b8ea1e165b28944a7e0379fcabb5cbff932eee Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 6 Nov 2026 10:38:12 +0000 Subject: [PATCH 2/3] docs(eql): say why a stored document fails the query_json cast Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_012Jmn7qx6btMrkTQ6rQRjDz --- packages/eql/docs/reference/json-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/eql/docs/reference/json-support.md b/packages/eql/docs/reference/json-support.md index 1ee756c131..b7dacd0200 100644 --- a/packages/eql/docs/reference/json-support.md +++ b/packages/eql/docs/reference/json-support.md @@ -214,7 +214,7 @@ separate equality-indexed scalar column when server-side grouping is required. - **`eql_v3.jsonb_document_contains(a public.eql_v3_json_search, b public.eql_v3_json_search) RETURNS boolean`** — true if every selector in `b` is present in `a` (selector-subset containment); backs the `@>` operator. - **`eql_v3.jsonb_contains(a jsonb, b jsonb)` / `eql_v3.jsonb_contained_by(a jsonb, b jsonb)`** — function-form containment entrypoints for platforms that cannot type an operator call (Supabase PostgREST RPC, for example); same result as `@>` / `<@` (a parity test pins this). - **`eql_v3.jsonb_array(val jsonb) RETURNS jsonb[]`** — function-form array accessor for the same platforms. -- **`eql_v3.to_ste_vec_query(val public.eql_v3_json_search) RETURNS eql_v3.query_json`** — the GIN-indexable query shape `@>` inlines to. This is the only document-to-needle conversion: `doc::eql_v3.query_json` is **not** equivalent (PostgreSQL ignores casts from a domain, so it is a plain coercion that fails the `query_json` CHECK). +- **`eql_v3.to_ste_vec_query(val public.eql_v3_json_search) RETURNS eql_v3.query_json`** — the GIN-indexable query shape `@>` inlines to. This is the only document-to-needle conversion: `doc::eql_v3.query_json` is **not** equivalent (PostgreSQL ignores casts from a domain, so it is a plain coercion that fails the `query_json` CHECK for a stored document, whose `sv` entries carry ciphertext). - **`eql_v3.meta_data(val jsonb)`** — envelope accessor: returns `{i, v, h}` (the key header `h` included so an extracted entry can be decrypted). **`eql_v3.ciphertext(val jsonb)`** — returns the `c` field; on the ste_vec surface `c` is raw AEAD output, decryptable only together with the entry's `s` and the document `h`. **`eql_v3.selector(val jsonb)` / `(entry public.eql_v3_json_entry)`** — selector accessor. ### Path query functions From d25dc49beb14b28ab1e9acbd4fbdd863b08e587c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 11 Nov 2026 00:30:01 +0000 Subject: [PATCH 3/3] docs(eql): address review on query_json cast removal List the _search domains' bf term in the eql-bindings wire docs, move the no-cast note into the to_ste_vec_query doc block so Doxygen keeps it, and refresh the splinter exemption reason that still named the removed cast. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_012Jmn7qx6btMrkTQ6rQRjDz --- packages/eql/crates/eql-bindings/src/v3/mod.rs | 12 +++++++----- packages/eql/src/v3/json/types.sql | 14 +++++++------- packages/eql/tasks/test/splinter.sh | 2 +- 3 files changed, 15 insertions(+), 13 deletions(-) diff --git a/packages/eql/crates/eql-bindings/src/v3/mod.rs b/packages/eql/crates/eql-bindings/src/v3/mod.rs index 7fddfd0936..133741fab0 100644 --- a/packages/eql/crates/eql-bindings/src/v3/mod.rs +++ b/packages/eql/crates/eql-bindings/src/v3/mod.rs @@ -24,21 +24,23 @@ //! keys (required by every scalar domain CHECK, mirroring `ENVELOPE_KEYS` in //! `eql-domains`): `v`, `i`, `c`. Then the domain's required term keys — `hm` //! for `_eq`, `op` for `_ord`/`_ord_ope`, `ob` for `_ord_ore`, `bf` for -//! `_match`, none for -//! storage-only. `Option` does not appear in the generated scalar structs: the +//! `_match` (and, alongside the ordering term, for the `_search` domains), +//! none for storage-only. `Option` does not appear in the generated scalar structs: the //! capability **is** the type identity. Hold a [`integer::IntegerEq`] and `hm` is //! present, guaranteed by the Rust type and (SQL-side) by the domain CHECK. A //! missing term key is a deserialization error — the Rust analogue of the CHECK //! constraint. //! //! One exception to "the ordering term alone": `text`'s ordered domains carry **both** -//! `hm` and the ordering term (`text_ord`/`text_ord_ope`/`text_search` carry -//! `hm` + `op`; `text_ord_ore`/`text_search_ore` carry `hm` + `ob`), where the non-text +//! `hm` and the ordering term (`text_ord`/`text_ord_ope` carry `hm` + `op`; +//! `text_ord_ore` carries `hm` + `ob`; `text_search` carries `hm` + `op` + +//! `bf`; `text_search_ore` carries `hm` + `ob` + `bf`), where the non-text //! ordered domains carry the ordering term alone. Text routes `=`/`<>` //! through `hm` rather than the ordering term because lexicographic ORE/OPE //! over text is not equality-lossless, so equality needs the HMAC. The //! generated struct doc surfaces this structurally — its required-keys line -//! lists `hm` `op` (or `hm` `ob`) rather than just the ordering key. +//! lists `hm` `op` (or `hm` `ob`, plus `bf` for the `_search` domains) rather than +//! just the ordering key. //! //! The generated flat-scalar types are also **strict**: every scalar struct is //! `#[serde(deny_unknown_fields)]`, so a payload carrying keys outside the diff --git a/packages/eql/src/v3/json/types.sql b/packages/eql/src/v3/json/types.sql index 32ad2daa9f..4536e5de61 100644 --- a/packages/eql/src/v3/json/types.sql +++ b/packages/eql/src/v3/json/types.sql @@ -249,6 +249,13 @@ $$; --! GIN index expression: --! `GIN (eql_v3.to_ste_vec_query(col)::jsonb jsonb_path_ops)`. --! +--! @note There is deliberately no `CREATE CAST` from `public.eql_v3_json_search` +--! to `eql_v3.query_json`. PostgreSQL ignores a cast whose source type is a +--! domain (it warns "cast will be ignored because the source data type is a +--! domain"), so `doc::eql_v3.query_json` would never call +--! `to_ste_vec_query`; it is a plain binary coercion that then fails the +--! `query_json` CHECK. Call `eql_v3.to_ste_vec_query(col)` explicitly. +--! --! @param e public.eql_v3_json_search Source encrypted payload --! @return eql_v3.query_json Query-shaped needle, sv elements normalised. --! @see eql_v3.query_json @@ -291,10 +298,3 @@ AS $$ ) )::eql_v3.query_json $$; - ---! @note There is deliberately no `CREATE CAST` from `public.eql_v3_json_search` ---! to `eql_v3.query_json`. PostgreSQL ignores a cast whose source type is a ---! domain (it warns "cast will be ignored because the source data type is a ---! domain"), so `doc::eql_v3.query_json` would never call ---! `to_ste_vec_query`; it is a plain binary coercion that then fails the ---! `query_json` CHECK. Call `eql_v3.to_ste_vec_query(col)` explicitly. diff --git a/packages/eql/tasks/test/splinter.sh b/packages/eql/tasks/test/splinter.sh index 058689892f..bb1ecb3873 100755 --- a/packages/eql/tasks/test/splinter.sh +++ b/packages/eql/tasks/test/splinter.sh @@ -101,7 +101,7 @@ function_search_path_mutable eql_v3 @> function Containment (@>) operator wrappe function_search_path_mutable eql_v3 <@ function Contained-by (<@) operator wrapper on the eql_v3 encrypted-JSONB surface: same rationale as eql_v3.@>. Three overloads. function_search_path_mutable eql_v3 selector function STE-vec entry selector extractor: typed (public.eql_v3_jsonb_entry) overload, inlinable so `eql_v3.selector(col -> 'sel')` folds into the calling query. Structural domain-arg skip. The (jsonb) overload is plpgsql with a pinned search_path and does not surface. function_search_path_mutable eql_v3 ope_term function STE-vec ordered-term extractor for public.eql_v3_jsonb_entry: inlinable so `eql_v3.ope_term(col -> 'sel')` folds into the calling range query and matches the functional btree index built on the same expression. SET search_path would disable SQL function inlining. The deprecated eq_term(json_entry) compatibility alias is pinned and does not surface. -function_search_path_mutable eql_v3 to_ste_vec_query function Encrypted-JSONB query-document constructor (CAST WITH FUNCTION for eql_v3.query_jsonb): inlinable SQL over a public.eql_v3_json domain arg, structural domain-arg skip. Builds the ste_vec query value the @>/<@ wrappers compare against; must inline to fold into the calling query. +function_search_path_mutable eql_v3 to_ste_vec_query function Encrypted-JSONB containment-needle constructor, called explicitly (there is no CAST to eql_v3.query_json; PostgreSQL ignores casts from a domain): inlinable SQL over a public.eql_v3_json_search / eql_v3.query_json arg, structural domain-arg skip. Builds the ste_vec query value the @>/<@ wrappers compare against; must inline to fold into the calling query and to match the functional GIN index on eql_v3.to_ste_vec_query(col)::jsonb. function_search_path_mutable eql_v3 jsonb_document_contains function Typed encrypted-JSONB document containment engine backing the public.eql_v3_json_search @>/<@ operators: the inlinable SQL (public.eql_v3_json_search) overload folds to native jsonb containment over eql_v3.to_ste_vec_query(a/b)::jsonb, so the planner can match the functional GIN index on eql_v3.to_ste_vec_query(col)::jsonb. Structural domain-arg skip leaves this overload unpinned; the raw jsonb[] plpgsql overload has a fixed search_path and does not surface. function_search_path_mutable eql_v3 ste_vec_contains function Deprecated compatibility alias for the typed eql_v3.jsonb_document_contains overload. It must remain inlinable so existing function-form queries retain the same functional-GIN-index matching as before the rename. The public.eql_v3_json_search domain arguments leave this overload intentionally unpinned; the raw jsonb[] alias is pinned by tasks/pin_search_path_v3.sql and does not surface. function_search_path_mutable eql_v3 jsonb_array function ste_vec deterministic-field array extractor on the eql_v3 encrypted-JSONB surface: public inlinable SQL (raw jsonb arg) behind the documented functional GIN index expression eql_v3.jsonb_array(col). Takes bare jsonb, so it carries the documented `eql-inline-critical` COMMENT marker that pin_search_path_v3.sql honours rather than the structural skip.