diff --git a/.changeset/eql-drop-ignored-query-json-cast.md b/.changeset/eql-drop-ignored-query-json-cast.md new file mode 100644 index 0000000000..c9cb12b546 --- /dev/null +++ b/.changeset/eql-drop-ignored-query-json-cast.md @@ -0,0 +1,5 @@ +--- +'@cipherstash/eql': patch +--- + +Drop the `CREATE CAST (public.eql_v3_json_search AS eql_v3.query_json)` that PostgreSQL always ignored (it warned "cast will be ignored because the source data type is a domain" on every install). `doc::eql_v3.query_json` was never converting; call `eql_v3.to_ste_vec_query(col)` instead. Also corrects the `eql-bindings` wire-format docs: `op` backs `_ord` / `_ord_ope` (and `text_ord` / `text_search` carry `hm` + `op`), `ob` only `_ord_ore`. diff --git a/packages/eql/crates/eql-bindings/src/v3/mod.rs b/packages/eql/crates/eql-bindings/src/v3/mod.rs index a230940561..133741fab0 100644 --- a/packages/eql/crates/eql-bindings/src/v3/mod.rs +++ b/packages/eql/crates/eql-bindings/src/v3/mod.rs @@ -23,22 +23,24 @@ //! For the generated flat-scalar families, every payload has the shared envelope //! keys (required by every scalar domain CHECK, mirroring `ENVELOPE_KEYS` in //! `eql-domains`): `v`, `i`, `c`. Then the domain's required term keys — `hm` -//! for `_eq`, `ob` for `_ord`/`_ord_ore`, `op` for `_ord_ope`, `bf` for -//! `_match`, none for -//! storage-only. `Option` does not appear in the generated scalar structs: the +//! for `_eq`, `op` for `_ord`/`_ord_ope`, `ob` for `_ord_ore`, `bf` for +//! `_match` (and, alongside the ordering term, for the `_search` domains), +//! none for storage-only. `Option` does not appear in the generated scalar structs: the //! capability **is** the type identity. Hold a [`integer::IntegerEq`] and `hm` is //! present, guaranteed by the Rust type and (SQL-side) by the domain CHECK. A //! missing term key is a deserialization error — the Rust analogue of the CHECK //! constraint. //! -//! One exception to "`ob` for `_ord`": `text`'s ordered domains carry **both** -//! `hm` and the ordering term (`text_ord`/`text_ord_ore`/`text_search` carry -//! `hm` + `ob`; `text_ord_ope` carries `hm` + `op`), where the non-text +//! One exception to "the ordering term alone": `text`'s ordered domains carry **both** +//! `hm` and the ordering term (`text_ord`/`text_ord_ope` carry `hm` + `op`; +//! `text_ord_ore` carries `hm` + `ob`; `text_search` carries `hm` + `op` + +//! `bf`; `text_search_ore` carries `hm` + `ob` + `bf`), where the non-text //! ordered domains carry the ordering term alone. Text routes `=`/`<>` //! through `hm` rather than the ordering term because lexicographic ORE/OPE //! over text is not equality-lossless, so equality needs the HMAC. The //! generated struct doc surfaces this structurally — its required-keys line -//! lists `hm` `ob` (or `hm` `op`) rather than just the ordering key. +//! lists `hm` `op` (or `hm` `ob`, plus `bf` for the `_search` domains) rather than +//! just the ordering key. //! //! The generated flat-scalar types are also **strict**: every scalar struct is //! `#[serde(deny_unknown_fields)]`, so a payload carrying keys outside the diff --git a/packages/eql/docs/reference/json-support.md b/packages/eql/docs/reference/json-support.md index aabbce99ac..b7dacd0200 100644 --- a/packages/eql/docs/reference/json-support.md +++ b/packages/eql/docs/reference/json-support.md @@ -214,7 +214,7 @@ separate equality-indexed scalar column when server-side grouping is required. - **`eql_v3.jsonb_document_contains(a public.eql_v3_json_search, b public.eql_v3_json_search) RETURNS boolean`** — true if every selector in `b` is present in `a` (selector-subset containment); backs the `@>` operator. - **`eql_v3.jsonb_contains(a jsonb, b jsonb)` / `eql_v3.jsonb_contained_by(a jsonb, b jsonb)`** — function-form containment entrypoints for platforms that cannot type an operator call (Supabase PostgREST RPC, for example); same result as `@>` / `<@` (a parity test pins this). - **`eql_v3.jsonb_array(val jsonb) RETURNS jsonb[]`** — function-form array accessor for the same platforms. -- **`eql_v3.to_ste_vec_query(val public.eql_v3_json_search) RETURNS eql_v3.query_json`** — the GIN-indexable query shape `@>` inlines to. +- **`eql_v3.to_ste_vec_query(val public.eql_v3_json_search) RETURNS eql_v3.query_json`** — the GIN-indexable query shape `@>` inlines to. This is the only document-to-needle conversion: `doc::eql_v3.query_json` is **not** equivalent (PostgreSQL ignores casts from a domain, so it is a plain coercion that fails the `query_json` CHECK for a stored document, whose `sv` entries carry ciphertext). - **`eql_v3.meta_data(val jsonb)`** — envelope accessor: returns `{i, v, h}` (the key header `h` included so an extracted entry can be decrypted). **`eql_v3.ciphertext(val jsonb)`** — returns the `c` field; on the ste_vec surface `c` is raw AEAD output, decryptable only together with the entry's `s` and the document `h`. **`eql_v3.selector(val jsonb)` / `(entry public.eql_v3_json_entry)`** — selector accessor. ### Path query functions diff --git a/packages/eql/src/v3/json/types.sql b/packages/eql/src/v3/json/types.sql index 0bc34904b9..4536e5de61 100644 --- a/packages/eql/src/v3/json/types.sql +++ b/packages/eql/src/v3/json/types.sql @@ -249,6 +249,13 @@ $$; --! GIN index expression: --! `GIN (eql_v3.to_ste_vec_query(col)::jsonb jsonb_path_ops)`. --! +--! @note There is deliberately no `CREATE CAST` from `public.eql_v3_json_search` +--! to `eql_v3.query_json`. PostgreSQL ignores a cast whose source type is a +--! domain (it warns "cast will be ignored because the source data type is a +--! domain"), so `doc::eql_v3.query_json` would never call +--! `to_ste_vec_query`; it is a plain binary coercion that then fails the +--! `query_json` CHECK. Call `eql_v3.to_ste_vec_query(col)` explicitly. +--! --! @param e public.eql_v3_json_search Source encrypted payload --! @return eql_v3.query_json Query-shaped needle, sv elements normalised. --! @see eql_v3.query_json @@ -291,7 +298,3 @@ AS $$ ) )::eql_v3.query_json $$; - -CREATE CAST (public.eql_v3_json_search AS eql_v3.query_json) - WITH FUNCTION eql_v3.to_ste_vec_query(public.eql_v3_json_search) - AS ASSIGNMENT; diff --git a/packages/eql/tasks/test/splinter.sh b/packages/eql/tasks/test/splinter.sh index 058689892f..bb1ecb3873 100755 --- a/packages/eql/tasks/test/splinter.sh +++ b/packages/eql/tasks/test/splinter.sh @@ -101,7 +101,7 @@ function_search_path_mutable eql_v3 @> function Containment (@>) operator wrappe function_search_path_mutable eql_v3 <@ function Contained-by (<@) operator wrapper on the eql_v3 encrypted-JSONB surface: same rationale as eql_v3.@>. Three overloads. function_search_path_mutable eql_v3 selector function STE-vec entry selector extractor: typed (public.eql_v3_jsonb_entry) overload, inlinable so `eql_v3.selector(col -> 'sel')` folds into the calling query. Structural domain-arg skip. The (jsonb) overload is plpgsql with a pinned search_path and does not surface. function_search_path_mutable eql_v3 ope_term function STE-vec ordered-term extractor for public.eql_v3_jsonb_entry: inlinable so `eql_v3.ope_term(col -> 'sel')` folds into the calling range query and matches the functional btree index built on the same expression. SET search_path would disable SQL function inlining. The deprecated eq_term(json_entry) compatibility alias is pinned and does not surface. -function_search_path_mutable eql_v3 to_ste_vec_query function Encrypted-JSONB query-document constructor (CAST WITH FUNCTION for eql_v3.query_jsonb): inlinable SQL over a public.eql_v3_json domain arg, structural domain-arg skip. Builds the ste_vec query value the @>/<@ wrappers compare against; must inline to fold into the calling query. +function_search_path_mutable eql_v3 to_ste_vec_query function Encrypted-JSONB containment-needle constructor, called explicitly (there is no CAST to eql_v3.query_json; PostgreSQL ignores casts from a domain): inlinable SQL over a public.eql_v3_json_search / eql_v3.query_json arg, structural domain-arg skip. Builds the ste_vec query value the @>/<@ wrappers compare against; must inline to fold into the calling query and to match the functional GIN index on eql_v3.to_ste_vec_query(col)::jsonb. function_search_path_mutable eql_v3 jsonb_document_contains function Typed encrypted-JSONB document containment engine backing the public.eql_v3_json_search @>/<@ operators: the inlinable SQL (public.eql_v3_json_search) overload folds to native jsonb containment over eql_v3.to_ste_vec_query(a/b)::jsonb, so the planner can match the functional GIN index on eql_v3.to_ste_vec_query(col)::jsonb. Structural domain-arg skip leaves this overload unpinned; the raw jsonb[] plpgsql overload has a fixed search_path and does not surface. function_search_path_mutable eql_v3 ste_vec_contains function Deprecated compatibility alias for the typed eql_v3.jsonb_document_contains overload. It must remain inlinable so existing function-form queries retain the same functional-GIN-index matching as before the rename. The public.eql_v3_json_search domain arguments leave this overload intentionally unpinned; the raw jsonb[] alias is pinned by tasks/pin_search_path_v3.sql and does not surface. function_search_path_mutable eql_v3 jsonb_array function ste_vec deterministic-field array extractor on the eql_v3 encrypted-JSONB surface: public inlinable SQL (raw jsonb arg) behind the documented functional GIN index expression eql_v3.jsonb_array(col). Takes bare jsonb, so it carries the documented `eql-inline-critical` COMMENT marker that pin_search_path_v3.sql honours rather than the structural skip. diff --git a/packages/eql/tests/sqlx/snapshots/eql_v3_public_surface.txt b/packages/eql/tests/sqlx/snapshots/eql_v3_public_surface.txt index 85c9bfaf6d..f7e7ac8ec1 100644 --- a/packages/eql/tests/sqlx/snapshots/eql_v3_public_surface.txt +++ b/packages/eql/tests/sqlx/snapshots/eql_v3_public_surface.txt @@ -59,7 +59,6 @@ aggregate eql_v3.min(public.eql_v3_text_search_ore) aggregate eql_v3.min(public.eql_v3_timestamp_ord) aggregate eql_v3.min(public.eql_v3_timestamp_ord_ope) aggregate eql_v3.min(public.eql_v3_timestamp_ord_ore) -cast public.eql_v3_json_search -> eql_v3.query_json function eql_v3.->(e public.eql_v3_json_search, selector integer) function eql_v3.->(e public.eql_v3_json_search, selector text) function eql_v3.->>(e public.eql_v3_json_search, selector integer)