fix(plugin-security): retire seven dead { records } find-result limbs, and stop the permission-set loader inventing an empty page

.changeset/quiet-pans-repair.md

@@ -0,0 +1,9 @@
+---
+'@objectstack/plugin-security': patch
+---
+
+Remove seven dead `{ records }` union-normalizer limbs on engine `find()` results, and repair the one that was silently dropping instead of gapping.
+
+Six seams in this plugin normalized an engine read as `Array.isArray(x) ? x : x.records`. The envelope limb was unreachable: `ObjectQL.find` resolves a bare array of row objects, measured by booting a real engine over a real `SqlDriver` and driving each seam through the shipped function that owns it, rather than inferred from `IDataEngine.find`'s declared `Promise<any[]>` (a declared type is not proof — this repo also has a `find()` that resolves an envelope). Each seam keeps its existing disposition for a non-array; only the dead limb is gone.
+
+The seventh is repaired in the opposite direction. `SecurityPlugin`'s `sys_permission_set` loader mapped three different facts onto one value: a read that succeeded on an empty catalog, a read that threw, and a read that resolved something it could not read all left as `[]`. On the enforcement plane that silently withdraws grants that exist while every request still looks normal, and it made `PermissionEvaluator`'s existing "db lookup failed" warning unreachable — so a transient database error and an empty catalog produced identical, undiagnosable 403s. The loader now lets the read fault propagate and refuses an unreadable result with `DATABASE_ERROR`. Enforcement is unchanged for every result the shipped engine produces; an envelope or a non-row element now refuses (fail-closed) where the old code read through it. An unanswered read still grants nothing; what changes is that it is now reported instead of silent.

content/docs/permissions/system-context.mdx

@@ -87,14 +87,14 @@ that silently does not happen.
 
 | # | Behaviour when `isSystem` | Package | What you get / what you lose | Anchor |
 |:--|:---|:---|:---|:---|
-| 1 | **The whole security middleware short-circuits** before any gate runs | plugin-security | Get: every CRUD/FLS/tenant/owner gate below skipped in one branch. Lose: all of rows 2–6 at once — this is the single largest behaviour on the page | `security-plugin.ts:1620` |
-| 2 | **`owner_id` is not auto-stamped on INSERT** (the step 3.5 anchor guard is inside the block row 1 skips) | plugin-security | Lose: the row lands `owner_id = NULL`, so the default `owner_only_writes` policy hides it **from its own creator**. Get: nothing — this is a gap, not a capability | guard at `security-plugin.ts:2546` (the step 3.5 block), skipped by `:1620` |
-| 3 | Row-level read filter resolves to "no filter" | plugin-security | Get: unscoped reads. Lose: row-level scoping entirely | `security-plugin.ts:4374` |
-| 4 | Field-level security returns **all** fields | plugin-security | Get: every column readable. Lose: field masking | `security-plugin.ts:4525` |
-| 5 | Export permission granted unconditionally | plugin-security | Get: `canExport` is `true` | `security-plugin.ts:4603` |
-| 6 | Write bypass = `true`, effective write scope = `org` | plugin-security | Get: widest write scope without holding any capability | `security-plugin.ts:1447`, `:1469` |
+| 1 | **The whole security middleware short-circuits** before any gate runs | plugin-security | Get: every CRUD/FLS/tenant/owner gate below skipped in one branch. Lose: all of rows 2–6 at once — this is the single largest behaviour on the page | `security-plugin.ts:1686` |
+| 2 | **`owner_id` is not auto-stamped on INSERT** (the step 3.5 anchor guard is inside the block row 1 skips) | plugin-security | Lose: the row lands `owner_id = NULL`, so the default `owner_only_writes` policy hides it **from its own creator**. Get: nothing — this is a gap, not a capability | guard at `security-plugin.ts:2612` (the step 3.5 block), skipped by `:1686` |
+| 3 | Row-level read filter resolves to "no filter" | plugin-security | Get: unscoped reads. Lose: row-level scoping entirely | `security-plugin.ts:4440` |
+| 4 | Field-level security returns **all** fields | plugin-security | Get: every column readable. Lose: field masking | `security-plugin.ts:4591` |
+| 5 | Export permission granted unconditionally | plugin-security | Get: `canExport` is `true` | `security-plugin.ts:4669` |
+| 6 | Write bypass = `true`, effective write scope = `org` | plugin-security | Get: widest write scope without holding any capability | `security-plugin.ts:1513`, `:1535` |
 | 7 | Metadata-plane schema masking exempt (ADR-0106 D4) | metadata-core | Get: unmasked object schema. Note: the exemption is a **caller** property — it short-circuits before the security service is consulted | `object-schema-fls.ts:228` |
-| 8 | `explain()` may target a principal other than the caller | plugin-security | Get: no `manage_users` / delegated-admin check | `security-plugin.ts:3887` |
+| 8 | `explain()` may target a principal other than the caller | plugin-security | Get: no `manage_users` / delegated-admin check | `security-plugin.ts:3953` |
 | 9 | Anonymous-deny treats the caller as authenticated | core | Get: passes the 401 seam with no `userId` | `anonymous-deny.ts:154` |
 | 10 | Permission-set projection middleware skipped | plugin-security | Lose: projection of permission-set-derived columns | `permission-set-projection.ts:1015` |
 | 11 | Session-resolution middleware skipped | plugin-auth | Get: no session lookup attempted | `auth-plugin.ts:1427` |
@@ -120,7 +120,7 @@ that silently does not happen.
 | 26 | Search-companion column **kept** in a read's rows when it was explicitly requested | objectql | Get: the internal companion column is readable. Lose: nothing for app code — this is the engine reading its own index | `objectql/src/engine.ts:6881` |
 | 27 | Dependent-count disclosure on a blocked delete | objectql | Get: the count of blocking children. Nothing was elevated past the caller, so nothing is withheld | `objectql/src/engine.ts:12477` |
 | 28 | Reference-cleanup log attributes the write to `'system'` | objectql | Get: an honest actor label instead of `anonymous` when the context carries neither `userId` nor `actor` | `objectql/src/engine.ts:12406` |
-| 29 | **Bulk data event `organizationId` OMITTED** — the batch is published "not asserted" | plugin-security | Get: nothing — the `data.records.*` event still publishes. Lose: the per-organization attribution: this exit is taken before the security middleware composes any tenant wall, so it records no Layer 0 verdict on the operation (`OperationContext.tenantLayer0Verdict`, #15813), and the engine's bulk producer — which reads that recorded verdict and nothing else — omits the key rather than filling it from the caller's `tenantId`; a tenant-scoped consumer then does not deliver the event inside an organization wall (#15225) | `security-plugin.ts:1620` |
+| 29 | **Bulk data event `organizationId` OMITTED** — the batch is published "not asserted" | plugin-security | Get: nothing — the `data.records.*` event still publishes. Lose: the per-organization attribution: this exit is taken before the security middleware composes any tenant wall, so it records no Layer 0 verdict on the operation (`OperationContext.tenantLayer0Verdict`, #15813), and the engine's bulk producer — which reads that recorded verdict and nothing else — omits the key rather than filling it from the caller's `tenantId`; a tenant-scoped consumer then does not deliver the event inside an organization wall (#15225) | `security-plugin.ts:1686` |
 
 ### 3. Sharing (`plugin-sharing`)
 

packages/plugins/plugin-security/src/auto-org-admin-grant.ts

@@ -149,7 +149,14 @@ async function tryFind(
 ): Promise<any[]> {
   try {
     const rows = await ql.find(object, { where, limit }, { context });
-    return Array.isArray(rows) ? rows : Array.isArray(rows?.records) ? rows.records : [];
+    // Bare array, driven — see `engine-find-bare-array.pin.test.ts`, which boots
+    // a real engine over a real `SqlDriver` and pins this seam. The `{ records }`
+    // limb removed from here was dead code that read as a contract.
+    //
+    // The `[]` arm is left exactly as it was: this function's whole contract is
+    // `Promise<any[]>` best-effort, and turning it into a gap is a different
+    // change with a different blast radius than removing an unreachable limb.
+    return Array.isArray(rows) ? rows : [];
   } catch (e) {
     // Reads legitimately fail before the tables exist (boot ordering), so this
     // is debug rather than warn — but it is no longer nothing (#4640).

packages/plugins/plugin-security/src/bootstrap-declared-permissions.ts

@@ -91,7 +91,12 @@ async function defaultLookup(ql: any, name: string, organizationId?: string): Pr
   } catch {
     return { status: 'unknown' };
   }
-  const list = Array.isArray(rows) ? rows : Array.isArray(rows?.records) ? rows.records : null;
+  // Bare array, driven against a real engine over a real `SqlDriver` — not
+  // inferred from `IDataEngine.find`'s declared `Promise<any[]>`, which is not
+  // proof (this repo has a `find()` that resolves an envelope instead). The
+  // `{ records }` limb that stood here was dead; `engine-find-bare-array.pin.test.ts`
+  // pins this seam. A non-array is still `unknown` — never "no such row".
+  const list = Array.isArray(rows) ? rows : null;
   if (list === null) return { status: 'unknown' };
   // [#10103] This organization's own row answers; an organization-less leftover
   // is reported beside `absent` and never returned as `present`. One spelling of

packages/plugins/plugin-security/src/claim-seed-ownership.ts

@@ -173,13 +173,18 @@ function affectedRowCount(value: unknown): number | undefined {
   return value;
 }
 
-/** Ids from a `find` result, tolerating both the array and `{ records }` shapes. */
+/**
+ * Ids from a `find` result.
+ *
+ * `ObjectQL.find` resolves a BARE array — driven against a real engine over a
+ * real `SqlDriver` through this module's own paging fallback, the only path that
+ * reaches here (`engine-find-bare-array.pin.test.ts`). It is driven rather than
+ * read off `IDataEngine.find`'s declared `Promise<any[]>` because a declared
+ * type is not proof: this repo also has a `find()` that resolves an envelope.
+ * The `{ records }` limb this carried was dead.
+ */
 function idsFrom(rows: any): string[] {
-  const list: any[] = Array.isArray(rows)
-    ? rows
-    : Array.isArray(rows?.records)
-      ? rows.records
-      : [];
+  const list: any[] = Array.isArray(rows) ? rows : [];
   const out: string[] = [];
   for (const r of list) if (r?.id) out.push(String(r.id));
   return out;