chore(protect-ffi): add integration suite to pnpm workspace

.github/workflows/integration-protect-ffi.yml

@@ -14,17 +14,17 @@ name: Integration — protect-ffi (native + WASM)
 # `packages/protect-ffi/src/integrationSuiteCi.test.ts` asserts this file (or a
 # successor) still invokes it.
 #
-# WHY THERE IS npm IN A pnpm REPO. `integration-tests/` is deliberately NOT a
-# pnpm workspace member: it pins `@cipherstash/auth ^0.39.0`, `vitest ^3.1.3`
-# and `@cipherstash/eql 3.0.2` independently of the repo catalog, which is part
-# of what it tests — the suite proves the published FFI surface works for a
-# consumer resolving its own dependency tree, not the one the monorepo happens
-# to hoist. So it keeps its own `package-lock.json` and `mise run setup` installs
-# it with `npm ci`. That is not a supply-chain regression: `npm ci` installs the
-# lockfile exactly (it fails rather than resolving anything new) and verifies
-# every tarball against the `integrity` hash recorded there, and the registry is
-# npmjs. Absorbing the suite into the pnpm workspace is a follow-up that changes
-# those pins, and changed pins can only be validated by a credentialed run.
+# WHERE THE SUITE'S DEPENDENCIES COME FROM. `integration-tests/` is a pnpm
+# workspace member (CIP-3744) and installs with the repo's own
+# `pnpm install --frozen-lockfile`, like everything else here. It used to sit
+# outside the workspace with its own `package-lock.json` and an `npm ci`, pinning
+# `@cipherstash/auth ^0.39.0`, `vitest ^3.1.3` and `@cipherstash/eql 3.0.2`
+# against the repo catalog. That last pin was the point: `tests/postgres-v3.test.ts`
+# queried SQL installed from the PUBLISHED EQL bundle while the payloads under
+# test were emitted by the in-tree `eql-bindings`, so the two halves of EQL could
+# disagree — and would have disagreed in a database, not in CI. `@cipherstash/eql`
+# now resolves `workspace:^`, and `@cipherstash/auth` and `vitest` take
+# `catalog:repo`.
 #
 # Separate from `tests.yml` on purpose, and separate from `tests-rust.yml`: this
 # needs CipherStash credentials AND Docker, and the suites THROW rather than
@@ -36,8 +36,24 @@ on:
     branches: [main]
     paths:
       # The suite itself: tests, fixtures, its compose file, its `tasks.toml`
-      # and its lockfile.
+      # and its manifest.
       - 'packages/protect-ffi/integration-tests/**'
+      # The suite's dependency versions, now that it is a pnpm workspace member:
+      # `@cipherstash/auth`, `vitest` and `typescript` reach it through
+      # `catalog:repo`, so a catalog bump changes what this job runs while
+      # editing no file under the suite. Same entry, same reason, as the other
+      # integration workflows — and like them, `pnpm-lock.yaml` is deliberately
+      # NOT listed: it moves on roughly every dependency bump in the monorepo,
+      # and a catalog version cannot reach it without this file changing first.
+      - 'pnpm-workspace.yaml'
+      # The EQL v3 SQL this suite installs into its database, which
+      # `tests/postgres-v3.test.ts` then queries. It came from a published
+      # tarball until the suite joined the workspace; it now resolves
+      # `workspace:^`, which is what makes it an input to this job rather than a
+      # pinned external. Not narrowed to `sql/` — `tasks.toml` reads the bundle
+      # through `@cipherstash/eql/sql`, i.e. through the package's tsup build,
+      # so the build config and its asset copier are inputs too.
+      - 'packages/eql/packages/eql/**'
       # The Rust the suite round-trips every payload through, and the manifests
       # that change what cargo builds without touching a .rs file. Matching the
       # filter in tests-rust.yml.
@@ -81,6 +97,8 @@ on:
     # on `main` — the exact inversion of what you want.
     paths:
       - 'packages/protect-ffi/integration-tests/**'
+      - 'pnpm-workspace.yaml'
+      - 'packages/eql/packages/eql/**'
       - 'packages/protect-ffi/crates/**'
       - 'packages/protect-ffi/Cargo.toml'
       - 'packages/protect-ffi/Cargo.lock'
@@ -185,11 +203,9 @@ jobs:
       # Not `./.github/actions/integration-setup`, and the difference is
       # deliberate: that action builds the `stash` CLI because the stack
       # integration harness installs EQL v3 by shelling out to it. This suite
-      # installs EQL from its own pinned `@cipherstash/eql` instead (see
-      # `eql:v3:install` in integration-tests/tasks.toml), so the CLI build
-      # would be minutes of runner time for nothing — and it pins Node 24,
-      # where this suite's `npm ci` is verified against the npm that ships with
-      # 22 (see the Node step below).
+      # reads the SQL straight out of the workspace `@cipherstash/eql` instead
+      # (see `eql:v3:install` in integration-tests/tasks.toml), so the CLI build
+      # would be minutes of runner time for nothing.
       - uses: actions/checkout@v6
         with:
           persist-credentials: false
@@ -198,14 +214,14 @@ jobs:
         with:
           run_install: false
 
-      # Node 22, not the 24 the other integration jobs use. `integration-tests`
-      # is installed with `npm ci` against a lockfile whose `".."` entry
-      # records an OLDER manifest of this very package (protect-ffi 0.29.0, and
-      # different devDependencies) — a link entry npm re-reads from disk. That
-      # resolves cleanly under npm 10, which ships with Node 22 and is what the
-      # lockfile was last verified against; npm 11's stricter
-      # manifest-vs-lockfile agreement check is an unforced risk for a job whose
-      # point is the encryption coverage, not the installer.
+      # Node 22, not the 24 `.github/actions/integration-setup` defaults to.
+      # This used to be an npm-version argument — the suite installed with
+      # `npm ci` against a lockfile npm 11 reads more strictly than npm 10 — and
+      # that argument died with the lockfile. What is left is the plainer one:
+      # 22 is `engines`' floor and what every other workflow in this repo pins,
+      # so it is the version this job's Rust binding and WASM artifacts are
+      # built and cached against everywhere else. 24 is covered for the JS
+      # surface by `tests.yml`'s matrix.
       - uses: actions/setup-node@v6.5.0
         with:
           node-version: 22
@@ -226,12 +242,18 @@ jobs:
       - name: Install node-gyp
         run: npm install -g node-gyp
 
-      # The workspace install, needed before the binding build: `build-ffi-binding`
-      # runs `pnpm --filter @cipherstash/protect-ffi run build`, i.e. the repo's
-      # tsc and neon. It is also what puts `@neon-rs/load` in
-      # packages/protect-ffi/node_modules — the suite reaches the package by
-      # symlink, so the parent's own dependencies come from pnpm, not from
-      # `npm ci`.
+      # The one install for this job, and it now covers the suite too: since
+      # CIP-3744 `integration-tests` is a pnpm workspace member, so this is what
+      # puts its `vitest`, `pg` and `dotenv` on disk and symlinks
+      # `@cipherstash/protect-ffi` and `@cipherstash/eql` to the packages in
+      # this tree. It is also required before the binding build —
+      # `build-ffi-binding` runs `pnpm --filter @cipherstash/protect-ffi run
+      # build` — and is what puts `@neon-rs/load` in
+      # packages/protect-ffi/node_modules.
+      #
+      # `mise run setup` below runs the same command again, deliberately: the
+      # task has to stand on its own for a contributor running it locally. On
+      # the runner it is a no-op.
       - name: Install dependencies
         run: pnpm install --frozen-lockfile
 
@@ -295,9 +317,10 @@ jobs:
           working_directory: packages/protect-ffi
 
       # One task, four things the suite cannot run without (see
-      # integration-tests/tasks.toml): `npm ci` for the suite's own dependency
-      # tree, `docker compose up` for Postgres, the EQL **v2** bundle from a
-      # GitHub release, and EQL **v3** from the pinned `@cipherstash/eql`.
+      # integration-tests/tasks.toml): `pnpm install --frozen-lockfile` (a
+      # no-op here — the step above already ran it), `docker compose up` for
+      # Postgres, the EQL **v2** bundle from a GitHub release, and EQL **v3**
+      # built from the in-tree `@cipherstash/eql`.
       #
       # BOTH EQL versions are required and nothing else in this repo installs
       # v2: `tests/postgres.test.ts` creates `eql_v2_encrypted` columns and
@@ -313,23 +336,24 @@ jobs:
       # `integration-db`'s per-job project names deliberately do not produce.
       # The fixed host port (5436) that comes with it cannot collide with the
       # shared stacks, which publish on ephemeral ports.
-      - name: Provision the suite (npm ci, Postgres, EQL v2 + v3)
+      - name: Provision the suite (Postgres, EQL v2 + v3)
         working-directory: packages/protect-ffi
         run: mise run setup
 
       # vitest directly rather than `mise run test:integration:all`, because
       # that task's first two steps are the debug cargo build and a wasm-pack
       # rebuild that `build-ffi-binding` has already done (above). What it runs
-      # after those is exactly this: `npx vitest`, the whole directory, INCLUDING
+      # after those is exactly this: vitest over the whole directory, INCLUDING
       # `tests/lock-context.test.ts`. Its sibling `test:integration` excludes
       # that file; upstream CI ran `:all` for a reason, and an invocation that
       # inherits the exclusion silently drops the identity-aware coverage.
       #
-      # `npx` resolves the vitest 3 pinned in integration-tests/package-lock.json,
-      # not the workspace catalog's — that pin is part of what the suite fixes.
+      # `pnpm exec`, which resolves the catalog `vitest` the workspace install
+      # put in this package's `node_modules/.bin`. It was `npx` against the
+      # suite's own `package-lock.json` until CIP-3744 retired that lockfile.
       - name: Integration suite (live ZeroKMS + Postgres, native and WASM)
         working-directory: packages/protect-ffi/integration-tests
-        run: npx vitest run
+        run: pnpm exec vitest run
 
       # `-v` drops the volume too. The runner is ephemeral, so this is not about
       # reclaiming it: it is so a re-run on a warm runner cannot inherit a

.github/workflows/tests.yml

@@ -217,6 +217,26 @@ jobs:
       - name: Typecheck (e2e)
         run: pnpm exec turbo run typecheck --filter @cipherstash/e2e
 
+      # The protect-ffi live integration suite, compiled but NOT run here — it
+      # needs Docker and CipherStash credentials, so it runs in
+      # integration-protect-ffi.yml behind a `paths:` filter.
+      #
+      # Which is exactly why this step exists. The suite's `tsc` used to ride
+      # along on its own `npm test`; CI invoked vitest directly, so it compiled
+      # nowhere, and the credentialed job is path-filtered — a repo-wide change
+      # that broke the suite's types (an `@cipherstash/auth` catalog bump moving
+      # `AccessKeyStrategy.create` to a `Result`, say) would not have started
+      # anything that noticed. Cheap: `tsc --noEmit` over the suite, no
+      # credentials, no database.
+      #
+      # That includes the two `.cjs` fixtures — the tsconfig sets `checkJs` and
+      # globs them deliberately. One of them is a real `AccessKeyStrategy` call
+      # site and was among the four the auth bump broke, so a `tests/**/*.ts`
+      # scope would have left this step green over it.
+      - name: Typecheck (protect-ffi integration suite)
+        run: >
+          pnpm exec turbo run typecheck --filter @cipherstash/ffi-integration-tests
+
       # Everything else typechecks against SOURCE. This one reads the emitted
       # `.d.ts`, which is what customers consume — and where typed `encryptQuery`
       # sat broken for every column through the whole rc series, because `tsc`

AGENTS.md

@@ -82,7 +82,7 @@ If these variables are missing, tests that require live encryption will fail or
 - `packages/nextjs`: Next.js helpers and Clerk integration (`./clerk` export)
 - `packages/utils`: Shared config (`utils/config`) and logger (`utils/logger`)
 - `packages/bench`: Performance / index-engagement benchmarks (private, not published)
-- `packages/protect-ffi`: Native FFI bindings to the CipherStash Client SDK (`@cipherstash/protect-ffi`) — the Rust core that `packages/stack` encrypts and decrypts through, absorbed from `cipherstash/protectjs-ffi`. Contains a **nested Cargo workspace** (`crates/`) and six per-platform binary packages under `platforms/*`, each published as `@cipherstash/protect-ffi-<platform>` and linked here via `workspace:*`. See the "Working on protect-ffi" notes below before touching it — its default `test` and `build` are deliberately Rust-free.
+- `packages/protect-ffi`: Native FFI bindings to the CipherStash Client SDK (`@cipherstash/protect-ffi`) — the Rust core that `packages/stack` encrypts and decrypts through, absorbed from `cipherstash/protectjs-ffi`. Contains a **nested Cargo workspace** (`crates/`) and six per-platform binary packages under `platforms/*`, each published as `@cipherstash/protect-ffi-<platform>` and linked here via `workspace:*`. Also holds the repo's live FFI integration suite at `integration-tests/` — a private workspace member (`@cipherstash/ffi-integration-tests`) enrolled by its own literal entry in `pnpm-workspace.yaml`, needing Docker and credentials, and deliberately carrying **no `test` script** so `pnpm test` cannot reach it. See the "Working on protect-ffi" notes below before touching it — its default `test` and `build` are deliberately Rust-free.
 - `packages/eql`: The Encrypt Query Language subtree — the SQL bundle that stores and queries encrypted payloads — absorbed from `cipherstash/encrypt-query-language`. **The directory is the subtree root, not the package.** It was imported at a *verbatim prefix* so its repo-root-relative paths (mise tasks, `Doxyfile`, `sync-generated.mjs`) keep resolving, which puts the npm package `@cipherstash/eql` two levels down at `packages/eql/packages/eql` — the same shape as `packages/protect-ffi/platforms/*`, and enrolled the same way, by an explicit `packages/eql/packages/*` glob in `pnpm-workspace.yaml`. The subtree root deliberately carries no `package.json`. Also contains a **nested Cargo workspace** at `packages/eql/crates/` (`eql-bindings`, published in lockstep with the npm package, plus `eql-domains` / `eql-codegen` / `eql-tests-macros`, which are not), a SQLx test crate at `packages/eql/tests/sqlx`, an ~900-line `mise.toml` task surface, its own `AGENTS.md`, and `docs/`. See the "Working on EQL" notes below before touching it.
 - `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README)
 - `examples/*`: Working apps (basic, prisma, supabase-worker)
@@ -213,17 +213,36 @@ several of those paths are exercised at all. It needs three things a normal
 `pnpm test` does not have: **Docker**, **CipherStash credentials**, and **both
 EQL versions installed** in the database.
 
-- **It is not a pnpm workspace member.** `pnpm-workspace.yaml` globs
-  `packages/*` (one level) plus `packages/protect-ffi/platforms/*`, so this
-  directory is invisible to pnpm and has its own `package-lock.json` with pins
-  that deliberately differ from the repo catalog (`@cipherstash/auth ^0.39.0`,
-  `vitest ^3.1.3`, `@cipherstash/eql 3.0.2`). `npm ci` installs it. Absorbing it
-  into the workspace is a follow-up, not a tidy-up: it changes those pins, and
-  only a credentialed run can prove the change is neutral.
+- **It is a pnpm workspace member, and `pnpm test` must never reach it.** Named
+  literally in `pnpm-workspace.yaml` (the `packages/*` glob is one level deep and
+  stops short of it), so its dependencies come from the repo lockfile:
+  `@cipherstash/eql` at `workspace:^`, `@cipherstash/protect-ffi` at
+  `workspace:*`, `@cipherstash/auth` / `vitest` / `typescript` from
+  `catalog:repo`. It had its own `package-lock.json` and an `npm ci` until
+  CIP-3744; the pin that mattered was `@cipherstash/eql 3.0.2`, the last place in
+  the tree where the SQL that STORES a payload could disagree with the Rust that
+  EMITS it — and it would have disagreed in a database, not in CI.
+
+  The cost of membership: root `pnpm test` is `turbo test --filter
+  './packages/**'`, which now reaches this package. It is kept out by **naming no
+  live script after a turbo task** — the suite's runners are `vitest:live` and
+  `vitest:live:coverage`, which `turbo.json` knows nothing about. `test` is the
+  obvious trap and `test:integration` is the less obvious one (a real turbo task,
+  invoked by four integration workflows — all `--filter`ed today, so an
+  unfiltered `turbo run test:integration` is what would bite).
+  `src/integrationSuiteCi.test.ts` derives the forbidden set from `turbo.json`
+  rather than listing it, so a task added there tomorrow is covered.
+
+  `typecheck` is the deliberate exception: it *should* run under `turbo run
+  typecheck`, and does, from `tests.yml`, on every PR. It needs no credentials
+  and no database, and its tsconfig sets `checkJs` so the suite's two `.cjs`
+  fixtures are compiled too — one of them is a real `AccessKeyStrategy` call
+  site, and a `tests/**/*.ts` scope would leave it checked by nothing but the
+  path-filtered credentialed job.
 - **Run it locally** from `packages/protect-ffi`:
 
   ```bash
-  mise run setup                 # npm ci, docker compose up, EQL v2 + v3
+  mise run setup                 # pnpm install, docker compose up, EQL v2 + v3
   mise run test:integration:all  # includes tests/lock-context.test.ts
   ```
 
@@ -252,6 +271,12 @@ EQL versions installed** in the database.
   `eql_v2.add_encrypted_constraint`) while `tests/postgres-v3.test.ts` needs the
   `eql_v3_*` domains. Skip either and half the suite fails on missing SQL
   functions.
+- **`eql:v3:install` builds `@cipherstash/eql` first, and has to.** The task
+  reads the bundle through `@cipherstash/eql/sql`, which the package's `exports`
+  map resolves to `dist/sql.js` — a tsup output. That was free while `npm ci`
+  unpacked a published tarball with `dist/` already in it; from the workspace it
+  is a build. Without it the task dies on `ERR_MODULE_NOT_FOUND`, which reads as
+  a broken dependency rather than an unbuilt one.
 - **`src/integrationSuiteCi.test.ts` asserts a root workflow still runs it.**
   The suite ran on every upstream PR and then ran *nowhere* for the whole
   absorption, because the workflow that drove it was deposited under
@@ -457,10 +482,13 @@ monorepo, which is where the silent failures are.
   ignored, so it is the one place a workspace-wide pin can be written and take
   effect). It exits **2**, not 0, when its own configuration has gone stale —
   a source it could not read, a declaration it expected and no longer sees, or
-  an exemption excusing nothing. There is one exemption today
-  (`packages/protect-ffi/integration-tests`, which installs with `npm ci` and
-  cannot take a `workspace:` specifier); adding another means writing the reason
-  down.
+  an exemption excusing nothing. There are **no exemptions today**: the only one
+  there had ever been (`packages/protect-ffi/integration-tests`, which installed
+  with `npm ci` and could not take a `workspace:` specifier) was retired when
+  that directory joined the pnpm workspace, and the guard's own staleness rule —
+  keyed on "excuses nothing", not "names nothing" — is what forced it out in the
+  same commit rather than leaving a standing permission behind. Adding one means
+  writing the reason down.
   It also reads the Cargo redirect tables — `[patch.*]` (including
   `[patch."https://…"]` and the dotted `[patch.crates-io.eql-bindings]` form)
   and `[replace]` — plus **`.cargo/config.toml`**, because cargo honours a