Repository navigation
test(oracles): move TanStack DB onto @tanstack/oracles, declare ownership and routes in every oracle - #2378
Merged
Conversation
…te-order and peer properties - packages/db/tests/oracle-property.ts: one constructor for an oracle's law, histories, model, production driver, and check. The driver only records observations; the check alone returns a violation, which fails the history with the law, checkpoint, difference, authority, limits, and a rerun command. - oracle/no-unit-test: an oracleProperty call from oracle-property with a literal check is its own declaration; any other call reports incompleteProperty. part-reach-oracle generates these calls and three lookalike modules (RED on the old rule; three mutants killed). - local-storage.write-order: every history of up to three updates and deletes, every handler completion order, and every decision vector (1,448). It replaces the two-write loops in local-storage-order-oracle and detects every mutant they detected, plus one. - local-storage.peers: same-tab peers, another tab, storage events, held writes, manual acceptance, handler and custom-parser writes, clearStorage from any Collection, number keys, and reload (19,289 histories). A write reads storage for its whole tab (maintainer decision D43). - scripts/oracle-gaps: a mutation-based gap report. Mutant schemata, warm Vitest contexts, per-site hits, typed kill classification, and differential observation that separates comparison gaps from generator gaps. Against src/local-storage.ts mutants the LocalStorage oracles now detect 201 of 320 (153 before), and 70 of 131 in the peer and event functions (36 before). Co-authored-by: Isaac <no-reply@databricks.com>
fast-check 4 removes RunDetails.error and the errorWithCause option. A failure now always carries the property's error as its cause, so failure text reads errorInstance and the replay witness always attaches the cause. fast-check 4 also infers literal and readonly types where 3 did not, so a few casts go away and the optimistic history arbitrary names its element type. fast-check 4 shrinks to different paths, so the two includes cross-formulation calibrations pin the new replay coordinates. Both documented direct replays were rerun and still fail once at checkpoint 0. Co-authored-by: Isaac <no-reply@databricks.com>
…sample sites The gap-density survey ran every db oracle through the gap report. A hung run stalled a context, so each run now has a wall-clock limit and a replaced context, and results are written as they arrive. The default context count follows the available parallelism, --sample draws a seeded subset of sites, sites are numbered across source files, labels use paths, and the report adds a reach summary. Co-authored-by: Isaac <no-reply@databricks.com>
…erated histories
oracleProperty gains what the first generated migration needed:
- Reach from the model: `reached(history, trace, expected)`, for states
only the model can name; the driver's `observe('reach')` stays the default.
- Declared states: every state in `states` needs a segment (ORC-015 with
D21); `measured` states are counted only; a reached undeclared state fails;
each lane prints a count for every declared state, through stdout because
Vitest hides console output from passing tests.
- Segments stay in the generated language, are rebuilt from the lane's seed
so a seed-and-path replay reproduces them, and build half of each lane's
histories, so the histories after each required state are explored.
- Failure fidelity, from a loss audit of the migrations: a driver error after
a violated checkpoint no longer hides the violation (it becomes the cause);
a violation can carry actual and expected values for Vitest's diff; rerun
commands escape the test name (an unescaped `|` selected 543 of 1,448).
live-query-observer.hydration-terminal: the driver records each step's
Collection status and rows as input and the observer's view and
announcements as output; the model reads only inputs; the check judges laws
`view` and `announcement`. Five prefixes plus three new ones are segments;
the three replays in both modes are pins.
Evidence (migration, no production change):
- Mutants of src/live-query-observer.ts (seed hides failure; failure never
ends the seed; late granular listener unseeded): the original and the
migrated oracle fail the same 5/9, 9/9, 5/9 tests.
- Gap report on src/live-query-observer.ts, seed 42: identical outcome for
every site (82 killed, 53 generator-gap, 89 no-observed-difference, 45
unreached). Before segments joined the generated histories, 5 sites fell
to unreached.
- A reported seed and path replays the same counterexample.
- Driver-error masking: a mutant that violates then throws in dispose()
reported the raw error before (RED) and the violation with that error as
its cause after (GREEN).
- Limits: one source file and one gap-report seed; the old fixed seeds now
draw different histories; histories cap at 10 steps (the old prefixed ones
reached 13).
The write-order and peers checks judge completeness last and give diff
values; write-order restores the prose the loss audit found missing. The
coverage map records the prompt-rejection witness gap for write order.
Co-authored-by: Isaac <no-reply@databricks.com>
A gap report compares an unmutated run with each mutant run, so a test whose captured values differ between two unmutated runs is left out of the comparison. On db's oracles that was 5,397 of 13,740 tests; it is now 0. Almost none were oracles checking unstable values; the capture recorded measurement artifacts: - wall-clock stamps, now rendered as `time`; - performance.now() deadlines inside production objects. A class instance now renders as its identity (`ClassName#n`), as `toBe` compares it; a toJSON class and a Temporal value render through their text. Limit: a mutant that changes only a class instance's fields under `toEqual` reads as no observed difference; - Vitest's `withTest` and chai's `assert`, which are not matchers; - UUIDs inside error messages. In a gap-report run, Math.random, crypto.getRandomValues, and crypto.randomUUID draw from a generator seeded by the test's name (and one for module load), and fast-check gets a global seed. On today's code, 4 of the first survey's 7 comparison gaps do not reproduce even with the old harness; with the new one 2 remain and lifecycle.ts M152 (reportRequestError `&&`->`||`) moves to generator-gap, cause undetermined. oracleProperty gains a determinism test: every pin and segment history (or eight evenly spaced enumerated histories) runs twice and must record the same trace. A driver that recorded performance.now() fails it at the first pin with both values; the three properties pass. Lint (no inline exception): - oracle/replayable-seed: a property seed must be a literal or constant, a recorded seed, a number-only `.each` table, the shared config's seed with a fallback, or an `undefined` lane guarded to `oraclePropertyOptions`. It found six lanes TANSTACK_DB_ORACLE_SEED could not replay (notification-laws 3, topKWithFractionalIndex 2, proxy-detachment 1); each now uses the shared config under a registered id. - oracle/pure-model: an oracleProperty model, check, or reached part may not read a clock, ambient randomness, or a timer, directly or through a same-file function (0 findings). 18 rule tests; the plugin suite passes (4,113); 0 findings across 344 oracle files. Co-authored-by: Isaac <no-reply@databricks.com>
Brings in #2302, #2331, #2303, #2333, #2332, and #2318, adapted to fast-check 4 and this branch's determinism rules: - transaction-serializer-oracle: keeps main's `replaySeed` and this branch's `errorInstance`. - Five `${result.error}` failure texts in the sync-metadata campaigns read `errorInstance` (RunDetails.error is gone in fast-check 4). - sync-metadata-campaign-result: the calibration replays `0:999`, a shrink step that does not exist. fast-check 4 still answers it with no history, but `0:0:0`, stale under fast-check 3, is a real history under 4. - The replay manifest names owners for this branch's nine registered ids (hydration-terminal, notification-laws x3, proxy.json-data-keys, and the four topK lanes), and the topK lanes name their ids literally. The eight incoming oracle files are deterministic under the gap-report capture (0 differing tests each) and pass oracle/replayable-seed and oracle/pure-model. Review records under docs/contributing/oracle-reviews keep their fast-check 3 replay coordinates as history. Co-authored-by: Isaac <no-reply@databricks.com>
…l time and a named cut check Two framework helpers from the optimistic-history migration (stage 1): - withinTurns(promise, turns), form F-F: resolves true when the promise settles, at the microtask an await would, or false after `turns` event-loop turns. The optimistic driver raced each settlement against a 20 ms timer as a deadlock guard. Across all four owners, 1,037 settlements arrived before the first event-loop turn and the timer never won, so a 10-turn bound keeps every checkpoint and removes the wall-clock dependence that a loaded machine could turn into a failure. A deterministic-simulation scheduler is not needed for this case. - forwardCut(cuts, observed, from, same), form F-E: the first allowed cut at or after `from` equal to an observation, or -1. The driver's hand-built publication matching now uses it; its backwards-cuts mutant still fails the publication owner. Owners: outcomes, publication, and settlement-boundaries (353 tests) and the persisted lane (10 of 12, as before) pass. Co-authored-by: Isaac <no-reply@databricks.com>
…operty
Three parts that the optimistic-history split needs:
- resolve(history) -> plan. A history may name actions indirectly ("settle
the second active transaction"); the plan holds the concrete actions that
the driver runs and the model and check read, never expected
observations. The Plan type defaults to the history, so existing
properties are unchanged.
- calibrations (design G8, rubric ORC-006): a named wrong answer and the
law that must reject it, either a `corrupt` trace transform (an
observation mutant) or a hostile `driver` (production set up wrongly).
The test finds a candidate history where the corruption applies,
requires its clean trace to pass, and requires the named law for the
corrupted one. runOne is now record + judge, so a corrupted trace is
judged on the same path.
- A pin may name the states it `enters`.
The hydration property gains two calibrations: a failure status shown as
`ready` must violate `view`, and a stale last read must violate
`announcement`. Both pass; a copy with the wrong law named, and a copy whose
corruption changes nothing, fail with messages that say so.
Co-authored-by: Isaac <no-reply@databricks.com>
…y parts
The optimistic-history runner fused grammar resolution, model, driver, and
assertions in one 830-line function. Its laws and model are unchanged; the
jobs are now separate parts:
- resolveOptimisticHistory: a history names actions indirectly ("settle
active transaction `slot`", an edit that may be a no-op), so it resolves,
by replaying the model's state, to a plan of concrete actions.
- optimisticHistoryModel: the same replay owes, per checkpoint, the
allowed publication cuts, visible rows, request outcomes and states,
waiting receipts, and unsettled transactions.
- optimisticHistoryDriver(lane): runs the plan through a real Collection
and records publications, raw batches, rows, replicas, downstream rows,
request outcomes, receipts, and tracked transactions. It decides nothing.
Waits for settlement are bounded in event-loop turns, including at
cleanup, so a request that never settles fails the cleanup law instead of
hanging the run.
- checkOptimisticHistory: 18 named laws (same-tick, forward-cuts, outcomes,
rows, tracked-transactions, and others), judged in the old order except
that callback shape now precedes event semantics: the old wrong-key
mutant was always caught by event semantics, so the callback-shape law
had never been calibrated.
- optimisticHistoryCalibrations: the seven observation mutants as trace
corruptions, each required to fail its named law.
The new owner optimistic-history.core runs the core grammar with the
settlement-boundary regressions as pins; that owner file is removed. The
pins were not members of the grammar until `deletes: fc.constant([])`
became a mapped constant (fast-check compares a constant by identity); the
generated sequences are unchanged. eventSemanticsDifference is now a pure
function under expectHistoryEventSemantics.
Evidence: a gap report on a seeded sample of 240 mutants of state.ts,
transactions.ts, mutations.ts, and sync.ts (seed 42), against a control of
the old campaign plus the same pins: killed 63 -> 65 (three
rollbackConflictingTransactions mutants the old campaign never reached),
unreached 94 -> 81, comparison gaps 2 -> 5 (three capturePreSyncVisibleState
mutants change a random-lane publication the check accepts: open leads).
One kill became a timeout and, with bounded cleanup waits, is killed again.
Limits: the core grammar does not generate pending manual edits, external
completion, rejected handlers, reentrant drains, partial updates, ordered
operations, or schema defaults; those stay pinned in the owners that still
drive runOptimisticHistory (coverage map).
Co-authored-by: Isaac <no-reply@databricks.com>
The framework's own promises now have an oracle of the same shape: synthetic properties (histories of one to three integers; 3 and 4 violate laws, 6 makes the driver throw, 0-2 mark reach states) are written to temporary test files and run under real Vitest in a child process with a reporter that keeps each failure's cause. An independent model predicts each test from the spec and the histories the synthetic driver logged: registration, verdict (law, setup error, or a violation that outranks a later driver error), shrinking, membership, reach, calibration, and determinism. The driver also probes replay (a failing fixed lane's seed and path through the random lane) and rerun (a failure's printed -t command). Child runs fix the random lane's seed, so the self-oracle's own determinism test holds. RED on the unchanged framework: a lane whose first failure was a driver error reported a shrunk violation instead (shrinking kept only laws), and the determinism test timed out at Vitest's 5 s default (only lanes had a limit). Fixes: shrinking keeps the first failure's kind (setup error, missed segment state, or one law), and a property-level `timeout` (120 s by default) applies to every test it registers. GREEN: 11 of 11. Framework mutants: the driver error masking a violation, any law satisfying a calibration, a determinism test that never compares, and shrinking that keeps only laws are rejected. Unescaped rerun commands, random-lane segments seeded from the clock, and unchecked pin reach survive for now (open: the pins do not yet distinguish them). Co-authored-by: Isaac <no-reply@databricks.com>
…und unstated The self-oracle's model had copied three framework behaviors that the framework's contract never stated (review: model rules without authority). Maintainer decisions, 2026-11-03: - A calibration skips a candidate whose clean trace fails or whose driver throws, for both the trace-corruption and hostile-driver forms; that history fails in its own test. Before, one failing first pin made every calibration fail. - Within one history a driver error decides first, then a law violation, then a missed segment state. Before, a missed segment state hid a law violation, and kind-preserving shrinking then kept it hidden. - A generated property's determinism test also runs three generated histories from the first fixed seed, so a varying driver is caught without pins or segments. The self-oracle states each rule beside its model, owes the determinism test's history count independently of what the framework chose to run, and has a pin for each rule. RED on the parent framework: 6 of 14 tests fail (the calibration pin, the precedence pin, and the determinism scope in four pins). GREEN: 14 of 14; the hydration, LocalStorage, and optimistic properties and the replay manifest pass (20,860 tests). Co-authored-by: Isaac <no-reply@databricks.com>
Three reviews of the self-oracle (adversarial, mutant gap hunt, law
coverage) found no framework bug among their survivors; each survivor was a
gap in the self-oracle. Closed:
- Replay: every failing lane is replayed across all files; other tests must
stand aside, the replay must start from the counterexample (fast-check
walks the path through the shrink tree without running it), and reproduce
it. The model owes the probes, so a missing replay fails.
- Rerun: each property's first violating pin or history is rerun, and a
sibling label (`items 3 | 2`) makes an unescaped `-t` run two tests.
- Witnesses for every reach and membership rule (pins and segments, inside
and outside the language), the default `observe("reach")` path, the
hostile-driver calibration form, a violation's driver-error cause, and a
property's timeout. Segment prefixes can now lie outside the language.
- Reach counts: each passing lane's printed counts are compared with counts
from the logged histories (read from the child's stdout, which the
framework writes past Vitest's console capture).
- Determinism: the nondeterministic counter is recorded last, so a
comparison of only the first observations fails.
- The self-oracle declares its own framework states as measured, and each of
its 18 pins names the state it witnesses.
- Prose beside every section; corrected limits; a coverage-map row; the
framework header now states its rules plainly instead of citing design
notes outside the repository.
Evidence: 21 of 21 pass (about two minutes). Seventeen framework mutants,
covering every survivor class from the reviews and the three behaviors the
2026-11-03 decisions replaced, are each rejected by at least one pin.
Co-authored-by: Isaac <no-reply@databricks.com>
…t tests The self-oracle spawns real Vitest runs and takes about two minutes, which belongs in the oracle framework's own CI, not db's. The package config excludes it; `pnpm test:oracle-framework` runs it with a config that replaces that exclusion (mergeConfig concatenates arrays). Its child runs use the default config on temporary spec files, so they are unaffected. Co-authored-by: Isaac <no-reply@databricks.com>
…perty The result-shape law runs through seven framework drivers in six packages. It is now one oracleProperty definition registered by each framework's conformance suite with its own driver, as framework.<driver>.result-shape. - The driver mounts the controllable binding and records raw public reads: the projection's shape, selected rows, and keyed state at each settled cut, and the shape, compiled mode, and observation count at each native cut. It no longer asserts. - The model owes each checkpoint's mode and rows, and names the twelve required states; two segments enter all of them on every lane, and the 26 former fixed examples are pins. - The check judges the six existing laws by name. The previous-mode projection test is a calibration, and stale settled rows is a second one. - The replay manifest names the random lane (`result-shape > random seed`). Evidence: all seven lanes pass (React, React with a dependency list, Preact, Vue, Solid, Angular, Svelte; 33 tests each). Reintroducing the historical Angular defect (the previous query's projection at the signal read) fails 32 tests at law `current query projection`; the old oracle failed 2. A production mutant that lets a list and findOne share a moved window (both guards removed) fails 36 React tests; removing either guard alone survives both oracles, because the other still refuses the move. Co-authored-by: Isaac <no-reply@databricks.com>
A type-level oracle in the core shape. Each of the 36 type shapes (adapter, key and scalar types, schema, Query row or envelope selection) resolves to a generated consumer program; the plan holds its source and each candidate declaration's position, never expected admission. The TypeScript compiler is the driver and records every diagnostic as plain data. The model owes the rejected candidates from each handler result's description; the check judges four named laws (candidate-diagnostics, admission, diagnostic-cause, distinct-diagnostics). The six compiler-observation controls are now calibrations, each required to fail its named law. Evidence: 43 of 43 pass. Typing the Query `onUpdate` handler as Promise<unknown> fails the 24 Query shapes at `diagnostic-cause`; the old oracle failed the same shapes and, with them, its six controls, which required the first shape to pass. The new calibrations skip failing candidates, so they stay meaningful under a production defect. Limit: the old per-state reach minimum over the enumeration is not asserted; oracleProperty declares no reach for enumerated sources yet. Co-authored-by: Isaac <no-reply@databricks.com>
Drivers hand-roll async interleavings with deferred gates. fast-check's fc.scheduler() would generate and shrink them, but a spike inside oracleProperty showed it cannot be a history value: the determinism test caught two runs of one scheduler history recording different orders (a scheduler is consumed as it runs), three clones of one unrun scheduler did not all reproduce, a generated scheduler value is not a member of its own language (so no pin or segment can hold one), and it prints as internal state. scheduleFrom(choices) keeps fc.scheduler's method names (schedule, waitOne, waitIdle) with decisions as plain data in the history. A task belongs to a queue, and a queue releases in scheduling order, as a storage engine completes one store's operations in order; each choice picks which queue's oldest task is released next. A history holding the choices replays exactly, shrinks like an array, can be a pin, and prints as JSON. self-oracle.scheduler checks the contract against an independent model of the release rule (100 generated histories plus a pin). RED: the scheduler without queues fails 3 of 4 tests at law release-order. GREEN: 4 of 4. Co-authored-by: Isaac <no-reply@databricks.com>
The run-count script read budgets only from `oraclePropertyOptions`,
`oracleRandomParameters`, and `{ property, defaultRuns }`. A property
moved onto `oracleProperty` declares its budget as
`histories: { runs }`, so the script reported the moved budget as
removed.
- Law: every run budget an oracle property uses is tracked, wherever and
however it is written (scripts/run-budget-oracle.property.test.js; the
removal rule is the maintainer decision of 2026-10-27).
- Reach: the grammar now draws `oracleProperty` definitions and property
budgets that move to one at head; both new states are declared and
required (minimum 1), and the GREEN run reached them.
- RED: `vitest run --config scripts/oracle-scripts.vitest.config.mjs
run-budget` on the unchanged script: 8 of 8 campaigns fail, with the
migration as counterexample.
- GREEN: the same command, 8 of 8 pass.
- Mutant: a script that reads `runs` beside `id` instead of inside
`histories` fails both "every source" campaigns.
- Limits: inline `histories` only; a `histories` constant, and an id
built by a template with expressions, are not read or generated.
Co-authored-by: Isaac <no-reply@databricks.com>
…arison helper
The random lane called `oraclePropertyOptions` twice: the first call
selected and logged the seed it ran, and the second logged a fresh seed
that was then overridden. A tool that read the last `ORACLE_SEED` line
replayed the wrong seed. The lane now selects its seed once, and
`withOracleExamples` holds the "examples come on top of the budget" rule
that both lanes and `oraclePropertyOptions` share.
The loss audits of four ports found failures without diffs and checks
that compared JSON text, which erases symbol keys, undefined fields, and
-0. `differs(checkpoint, law, actual, expected)` is one exact comparison
(Node's isDeepStrictEqual) with both values attached for the diff. A pin
that misses a state now also prints the states it entered.
- Law: a random lane logs exactly one ORACLE_SEED line for its property,
the seed it runs, and no other test logs one (self-oracle law
`seed-log`; authority: the replay contract in oracle-config.ts,
rubric ORC-007).
- Reach: new measured state `random-lane`, entered by every generated
pin.
- RED: `pnpm --filter @tanstack/db test:oracle-framework` on the parent:
17 of 21 fail with law seed-log ("logged 2 ORACLE_SEED lines, owed 1").
- GREEN: the same command, 21 of 21 pass.
- Mutant: a lane that logs one seed and runs seed + 1 fails ("logged seed
20251103 but ran seed 20251104").
- Limits: `differs`, like `forwardCut` and `withinTurns`, has no
self-oracle law; it is a thin wrapper over isDeepStrictEqual.
Co-authored-by: Isaac <no-reply@databricks.com>
…ty, and repair port losses Interruption settlement now runs on oracleProperty, and its driver records a trace that a separate check judges. A new scheduled timing holds storage operations and provider answers for a data schedule (`scheduleFrom`): storage completes in order, and the provider may answer before or after any pending storage operation. So an interruption can land while a write or an answer is still outstanding. The framework's rerun checks found two defects in the old oracle. Its three "issued" examples used turn 50, which the generator cannot make (maximum 40). Its trace held random transaction IDs, which the trace now names by commit position. Loss audits of the four ports (old version against new, one fresh reader per port) found losses that this commit repairs: - result-shape: the previous-projection calibration covered one direction; it is now one calibration per direction. Rows and keyed state compare exactly with `differs`, keeping enumerable symbol keys. Each native observation records its framework's cut name. The printed law says only the shape is owed at the native cut. - handler-types: a diagnostic records its file name, not a boolean; a repeated diagnostic is printed; the owner names its coverage entry. - optimistic core: the model again checks that each request's owed outcome agrees with its transaction state; four more violations carry diff values; the limits name the acknowledged handler write. - interruption: 120 runs keep 60 per timing; the printed law includes the unhandled-rejection law; order, outbox, and unhandled use `differs`. Evidence for the interruption port: - Law: interruption at any point keeps admission, fencing, exactly-once provider calls in commit order, settlement, durability, and no unhandled rejection (offline-transactions README; the settlement and leadership oracles). - Reach: measured per lane; fixed seed 20251103 entered all eight landings (dispose refused 4, leadership refused 3, others 16 to 30); eight pins enter every landing on every run. - RED/Mutant: against the old oracle and the new one, on mutated production. Dropping the execution fence, dropping the admission check, or admitting before initialization fails both (new: laws fencing, admission). An un-awaited outbox deletion passes all 9 old tests and fails 2 new ones with law durability. - GREEN: 13 of 13 on unchanged production. - Limits: disjoint keys; one store; no overlapping executors. Design questions the audits raised (enumerated reach, behavior test titles, segment budget, uncompared evidence, several violations per history, owner validation) are recorded for the design rerun. Co-authored-by: Isaac <no-reply@databricks.com>
The persisted lane ran the shared optimistic model through `runOptimisticHistory` with a wrapping-sync lane. It is now the `oracleProperty` `sqlite-persistence.optimistic-history`: the split model, check, and seven calibrations, with the persisted adapter as a per-history driver over the wrapping-sync grammar. The framework's membership check found that no pin or segment could be a member of that grammar: `abortStep`, `acknowledgedEdit`, the step list's settle-and-observe tail, and the core `handlerSourceBatch` were mapped without unmappers, and `storeStep` was a constant of objects, which fast-check compares by identity. Only 38 of 50 generated histories were members of their own language. Each now has an unmapper or is a record; all generated histories are members. The core grammar generates the same values. - Law: a persisted Collection exposes what the optimistic-history model owes at every step, while a durable store step is held and after a deferred source transaction aborts (optimistic-history-oracle.ts header). - Reach: `deferred-hold` and `abort-release` are required through two segments. A model-only search over 2,000 generated contexts found segments that enter them every time: a hold after the generated body, and an abort that comes first and releases the store itself. Fixed seed 22012 entered deferred-hold 24 times and abort-release 12. - RED/Mutant (old lane against new, on mutated production): a wrapper that reports no deferred keys to core fails both (new: law settled-rows); a discarded deferred transaction that keeps its held rows fails both (new: law rows). - GREEN: 11 of 11; the core owner and three legacy owners that share the grammar still pass (28 and 337). - Limits: an in-memory adapter; no awaited acceptance during a hold; no schema, partial-update, or reentrant lane. The old pinned history set only `a` on acknowledged edits, outside the grammar; it now sets the whole server row. Co-authored-by: Isaac <no-reply@databricks.com>
…lanned faults The FIFO retry oracle judged one delayed head and ready tails with successful providers; failures were out of scope. It is now the `oracleProperty` `offline.fifo-retry`, and each transaction's attempt plan is history data: retriable provider failures, a final success or permanent failure, and a storage fault on the deletion-phase write or the deletion. A model of the README's contract judges provider calls, caller settlements, and the outbox before the deadline, just before it, at it, and after every retry. A calibration found a defect in the first check: it classified a tail that ran before the head as a missed wakeup. Calls outside the full attempt order now break `fifo`; a correct prefix of the wrong length breaks `progress`. - Law: one transaction at a time in creation order; a retried transaction keeps its FIFO position; a permanent failure removes its row and the queue continues; a failed phase write or deletion rejects the caller with the storage error and stops the queue (README: FIFO Sequential Processing, Retry decisions, offline-execution settlement). - Reach: a segment requires a phase-write fault before a queued peer on every run; fixed seed 20260916 also entered deletion faults 4, permanent failures before a peer 4, retried heads 5, retried tails 9. - RED/Mutant (old oracle against new, on mutated production): a ready tail that passes the delayed head fails both (new: fifo). A retry that moves behind its peers, a queue that continues after a storage fault, and a permanent failure that stops the queue pass all old tests and fail the new one (fifo, fifo, progress). - GREEN: 10 of 10. - Limits: one mutation function, no retry hooks, no leadership changes, no storage fault on a permanent failure's removal; backoff lengths are not judged. An attempt outside the plan never settles, so a runaway executor stops at its first extra call instead of hanging the driver. Co-authored-by: Isaac <no-reply@databricks.com>
A fixed-seed campaign `{ seed: s, numRuns: n }` that moves onto
`oracleProperty` becomes `histories: { runs: n, fixedSeeds: [s] }`. The
run-count script did not read `fixedSeeds`, so it reported the seed's
budget as removed (the KeyScheduler port hit this).
- Law: every run budget an oracle uses is tracked wherever it is
written (scripts/run-budget-oracle.property.test.js; maintainer
decision of 2026-10-27 on removals).
- Reach: the grammar now moves fixed-seed campaigns into an
`oracleProperty` definition's `fixedSeeds`; the new state
`seed-migrated-to-framework` is required and was reached.
- RED: on the previous script, 8 of 8 campaigns fail with a migrated
fixed seed as counterexample.
- GREEN: 8 of 8.
- Mutant: a script that reads only literal seeds in `fixedSeeds`, not
imported constants, fails both "every source" campaigns.
- Limits: inline `histories` and inline `fixedSeeds` arrays only.
Co-authored-by: Isaac <no-reply@databricks.com>
…generate its whole grammar The KeyScheduler oracle ran its ledger model and the real scheduler in lockstep with inline assertions. It is now the `oracleProperty` `offline.key-scheduler-fifo-ledger`: the model owes a record per command, the driver records the scheduler's snapshot, what `start` was given, and what a removal returned, and the check maps each field to a law (fifo, update, ledger, serial, retirement). The model throws a named error for a command the lifecycle does not allow. Measured reach exposed a starved generator. Ported unchanged, 4 of 150 histories per lane started any transaction and 1 retried: fast-check's default array size kept histories near 10 commands (at most 12) although the grammar declared 1 to 50, and the biased `fc.nat()` selector picked the first few choices. With `size: 'medium'` and `fc.noBias`, 37 to 50 of 150 start work and 13 to 19 retry. The generated language is tokens mapped through `buildLegalHistory`, so `tokensOf` inverts it by search and accepts only an exact rebuild; pinned command lists are members and shrink. The "every modeled transition" pin removed `[active, sibling, missing]`, which the grammar could not generate; the active-removal choice can now name `missing`. - Law: one globally serial FIFO queue; equal creation times keep scheduling order; a delayed head blocks younger work; a failure updates retry data before release without moving the transaction; reconciliation retires only unissued IDs; clear retires all work (the established FIFO tests and TransactionExecutor's calling order). - Reach: every command type is measured; the pins enter all of them. - RED/Mutant (old oracle against new, on mutated scheduler code): a ready transaction passing the head, a retry moved to the back, a removal that retires the active ID, clear keeping the active ID, and reordered equal creation times fail both versions (new: fifo, fifo+ledger, retirement, serial, fifo+ledger). A grammar whose `nextFreeSlot` reuses a pending slot fails with the model's "already pending" error. - GREEN: 16 of 16, with eight calibrations, including the old five observation mutants. - Limits: persistence, caller settlement, leadership, and real timers have separate owners; at most five IDs. Co-authored-by: Isaac <no-reply@databricks.com>
Loss audits of the persisted optimistic lane, FIFO retry, and KeyScheduler ports (old version against new, one fresh reader each): - FIFO retry: the law statement and prose state the progress law (eligible work runs when its deadline passes) and its liveness-not- latency limit, cite the KeyScheduler oracle for the eligibility boundary, and restore the grammar's rationale and exclusions. The never-wakes calibration leaves the whole queue untouched, not only the calls. The driver installs fake timers inside its `try`, runs each release even when another fails, and keeps its own error as the cause of a cleanup failure. - Persisted lane: a pin restores the partial optimistic insert and update behind a held store, as core edits in the language. - Optimistic check: six more violations carry diff values (handler batch, reentrant suffix, aborted open batch, callback shape, receipts, cleanup receipts). - Framework header: `oracleProperty` does not do cleanup; the header said it did. Cleanup stays with the driver. - Coverage map: the scheduler entry records the starved-generator finding. Design questions the audits raised (generated share of a measured state, reach denominators, per-history calibrations, behavior titles, segment budget share) are recorded for the design rerun. Co-authored-by: Isaac <no-reply@databricks.com>
…erty
The consolidation oracle is now the `oracleProperty`
`ivm.multiset-consolidate`. Its laws are about reference identity (the
retained record is the first input object), and its values include
symbols, functions, bigints, NaN, and -0, none of which is plain data.
The driver names an output object, array, function, or symbol by the
input record that is the same reference, and a primitive, which has no
identity, by its type and text; the check decodes names against the
plan's own records and applies the model's identity rule.
Grammar changes: the keyed witnesses that were separate `it` cases (a
delimiter key beside an object tuple, reference values that still
merge, signed zero and non-finite keys, NaN values) are now witnesses of
the `keyedCollision` dimension, named by index so a pinned witness is a
member; structural steps are records, not mapped values; arrays use
`size: 'medium'` so the declared 24 steps are drawn. A mutant that
treated every 2-element array as keyed passed the old and the first new
version: no history held only pairs whose first element is no key. A new
structural step `[true, { v }]` and a pin close that.
- Law: one record per identity, the first in input order (-0 written as
0 for unkeyed numbers), with the summed multiplicity; zero sums absent;
identity by the shape of the whole multiset; the input unchanged
(contract comment on MultiSet.consolidate(), b5d92ce).
- Reach: every mode is measured, with "records merged", "join tuples
merged", and "only pairs without a key"; pins enter each mode. The
last two are rare in generated lanes (0 or 1 of 300), so pins carry
them.
- RED/Mutant (old oracle against new, on mutated src/multiset.ts):
unmerged fresh join tuples, the last record retained, zero-sum keyed
entries kept, and -0 not normalized fail both (new: merge, retained,
merge, retained). Every pair keyed passes the old oracle and fails the
new one (merge).
- GREEN: 29 of 29, with four calibrations (repeated identity, unsummed
multiplicity, a later record retained, a changed input).
- Limits: output order is not judged; MultiSet directly, not the query
shapes that produce these records; structural identity uses a 32-bit
hash.
Co-authored-by: Isaac <no-reply@databricks.com>
…n exact page-count law `cursor-pagination.history` is now an `oracleProperty`. Its grammar folds in what were fixed cases: invalid windows, the largest safe window end, peer reads issued at once, a window object changed right after its call, a backend whose first page is empty and nonterminal, and a page larger than the relation. The old property only checked that a reverse reread fetched nothing; the README's transport rules now give an exact law: after each step the pager has fetched the shortest prefix of backend pages that covers every window read so far (every page when a window drains the source; none for a zero limit). The model computes it from the backend's page sizes alone. - Law: a read returns exactly the requested slice of the whole filtered ordered relation; an invalid window rejects with RangeError before transport; a window is captured when called; zero limit does no transport; fetched pages are reused (cursor-pagination/README.md, "Contract before implementation"). - Reach: invalid windows, peer reads, empty first pages, zero limits, windows past the end, and changed windows are measured; each is entered by 13 to 100 of 100 histories per lane. - RED/Mutant (old file against new, on mutated src/cursor-pagination.ts): an empty page taken as exhaustion, over-fetching one page, a lazily read window, and an accepted unsafe window end fail both (new: transport, transport, invalid+slice, invalid). A zero limit that fetches passed both; a peer's fetch hid it in every history that had one, so a pin reads a lone zero limit first, and it now fails (transport). Unserialized reads still fail the file's held-response tests, which stay as they were. - GREEN: 39 of 39 in the file. - Limits: one scope per pager; reset, faults, and cancellation stay with the file's other properties. Co-authored-by: Isaac <no-reply@databricks.com>
Only the random lane checked that a property id was registered, through
`oraclePropertyOptions`. An enumerated source has no random lane, so
all four enumerated ports registered unknown ids, with no replay
manifest entry: `local-storage.peers`, `local-storage.write-order`,
`adapter.handler-types`, and the new `sqlite-persistence.setup-reopen`.
`oracleProperty` now checks its id when it registers; the four ids are
registered and have manifest entries.
- Law: a property registers its tests, or fails to load when its id is
not a registered oracle property, a required state has no segment, or
a segment names an undeclared state (self-oracle `registration`;
authority: the registry in oracle-config.ts and the replay manifest).
- Reach: drafts now include, rarely, an unregistered id; a pin registers
an unregistered enumerated property (state `registration-error`). The
pin was first rejected by the framework's own membership check, so the
draft grammar gained the dimension.
- RED: on the parent, the pin fails with law registration ("loaded, but
its id is unregistered ...").
- GREEN: `pnpm --filter @tanstack/db test:oracle-framework`, 22 of 22;
the manifest test, 81 of 81; every oracleProperty owner loads.
- Mutant: the parent is the mutant (no check for enumerated sources).
Co-authored-by: Isaac <no-reply@databricks.com>
… oracleProperty The setup-reopen oracle looped over 140 starting states inside one test and closed its databases only on success. It is now the enumerated `oracleProperty` `sqlite-persistence.setup-reopen`: each starting state is its own test, the driver closes both in-memory databases however the history ends, and the model owes the seven schema objects, the rows, and whether the work law applies. The old model took SQL text from a production fresh install; that is now an observation: the driver records a fresh install on a separate database, and the check compares the reopened schema with it (a differential law) and the fresh install's objects with the model's. - Law: after a fresh adapter's first read, the Collection's schema equals a fresh install's, both bookkeeping rows exist, and stored rows are unchanged, from every legal starting state; complete setup issues no DDL or writes and costs at most one call over a cached read (the adapter's per-Collection setup contract; issue #2204). - Reach: enumerated; every starting state runs (no declared reach for enumerated sources yet, recorded for the design rerun). - RED/Mutant: see the next run's results in the misfit record; four calibrations reject a lost trigger, a missing epoch row, dropped rows, and repeated DDL. - GREEN: 145 of 145. - Limits: one node:sqlite database per side; no cache-generation claims, concurrent adapters, or dropped row tables with rows. Co-authored-by: Isaac <no-reply@databricks.com>
Loss audits of the multiset, cursor, and setup-reopen ports (old version against new, one fresh reader each): - Cursor histories: an invalid window must reject with a RangeError instance (the name alone could be spoofed); any other rejection keeps its name and message. Invalid-window pins are named by their values. Histories allow ten steps, past the old 20 sequential windows. The largest-safe boundary and capture-at-call rationale are back. The authority now says where minimality comes from: the README's transport rules with the exact counts the old fixed tests pinned. - Multiset: collision witnesses are named by case name, not index; an `opposed` dimension restores the old rule that later records negate the first; each history clones witness objects (shared references stay shared), so no object is reused across runs; the printed limits keep the #1948 and non-finite key scope; pin rationale and a stale comment fixed. - Setup-reopen: the printed law keeps "exactly (type, name, table, SQL)", "for the registered schema version", and "a partial state may cost more"; the fresh-install-versus-model comparison has its own law (`fresh-install`); the prose no longer says one database. Design questions the audits raised again (behavior test titles, generated-share minimums for measured states, one printed law string) are recorded for the design rerun. Co-authored-by: Isaac <no-reply@databricks.com>
Adds the terms settled after the v5 design grammar: oracle property, generated, fixed, and constructed histories, campaign (fixed-seed, random, reseeded), campaign seed and replay path, run budget, verdict, declared state and reach, calibration, killed or survived, differential law, receiving path, oracle owner, decision record, limit, gap, and self-oracle. Separates reach (histories entering a declared state) from reach witness (the driver reaching a boundary), widens mutant to cover corrupted traces and hostile drivers, and says a seed-only run is a reseeded campaign, not a replay. Retires lane, pin, segment, and hostile mutant, and lists the oracle words to qualify at first use. Co-authored-by: Isaac <no-reply@databricks.com>
…clares owns The six shared e2e suite files said that the consuming package owns the production entries they judge, but no consumer (electric.e2e.test.ts, query.e2e.test.ts, trailbase.e2e.test.ts, or the persisted-collection conformance contract) declares owns, so the gap report counts none of their kills. The comments now say so, and the coverage map's Backend Collection E2E row records the follow-up: owns in each consumer, which needs a consumer under the package's Vitest config. The notification-laws checks are also used by the db oracle tests/notification-laws-oracle.test.ts, which owns the change-message path it judges with them. Co-authored-by: Isaac <no-reply@databricks.com>
The coverage map's oracleProperty framework row linked packages/db/tests/oracle-property-self-oracle.test.ts and its fixture, which this branch deleted when the framework moved to @tanstack/oracles. It now links the self-oracle and its synthetic properties in TanStack/oracles at the pinned commit b937148, and says that packages/db/tests/oracle-property.ts only re-exports the framework after loading TanStack DB's oracle-config. Co-authored-by: Isaac <no-reply@databricks.com>
…chored helper eagerLiveSourceProperty's calibration mode named a law when violated.match(law)?.[0] === violated, the leftmost match, while the test file's expectNamedLaw anchored the pattern at both ends. A pattern whose first alternative is a prefix of the law, such as /EVID-2|EVID-2 \(awaitTxId \d+\)/, named nothing in the property, and the g and y flags made the answer depend on lastIndex. namesLaw is now exported, anchors the pattern, drops the g, m, and y flags, and is used by both. The leaks-waits-after-fail calibration now uses that prefix-alternative pattern. RED/mutant: with the old leftmost-match namesLaw, the calibration fails (the property never fails, `expected false to be true`); GREEN: it passes with the anchored helper. Co-authored-by: Isaac <no-reply@databricks.com>
consolidateKeyed's tuple branch calls the records-merged and join-tuples-merged probes with the same condition. Both are declared reach states of ivm.multiset-consolidate: records merged counts a merge on every path, and join tuples merged only a merge of join tuples, so the branch keeps both. A comment now says so. Co-authored-by: Isaac <no-reply@databricks.com>
…le entries change Every pnpm-lock.yaml change counted as a lint-configuration change, so an unrelated dependency bump re-linted every oracle file. The lint scope now selects every oracle file for a lockfile edit only when the lockfile's entries for @tanstack/eslint-plugin-oracles (its importer entry, resolution, and snapshot with its resolved dependencies) differ from the merge-base's. A settings edit still selects every oracle file. - Law: L4 (lint scope) in scripts/lint-scope-oracle.property.test.js; the oracle results are a function of the installed rule and its settings, so a lockfile edit that leaves the plugin's resolved entries alone cannot change one (round-1 review finding B1-10). - Grammar: a configuration edit is one part of the plugin's entries (its version, integrity, or a resolved dependency), another package's entries, or the settings. The driver writes a pnpm v9 lockfile with the plugin and an unrelated package. Constructed histories (a generated skeleton with the edit fixed) enter each configuration state and an edit hook that names an unedited lockfile in every campaign. - Reach (fixed + random, seed -1677849190): plugin-version-edit 4, plugin-integrity-edit 6, plugin-dependency-edit 8, lockfile-edit 8, settings-edit 7, hook-lockfile-same-plugin 3. - RED: the extended oracle on the parent script fails the fixed campaign: an unrelated or unedited lockfile selects every oracle file where the model owes none. - GREEN: both campaigns pass. - Mutant: comparing only the lines that name the plugin (not the lines under them) misses an integrity or dependency edit and fails the fixed campaign. - Limits: a change deeper in the plugin's dependency tree that leaves its own entries alone is not drawn and is not selected; the lockfile is hand-written, not one pnpm wrote. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
… error but a missing file pluginChanged() read the merge-base's pnpm-lock.yaml through execFileSync's default 1 MiB output buffer. The repository's lockfile is 1,030,032 bytes, so a little growth would raise ENOBUFS; the bare catch then read the base as having no plugin entries, and every lockfile edit re-linted every oracle file again. The same catch swallowed mergeBase's "Cannot find origin/main or main" error. Now mergeBase runs outside any catch, `git ls-tree` decides whether the merge-base has a lockfile (an absent one has no plugin entries), and `git show` reads it with no buffer limit; any other git failure throws. - Law: L4 (lint scope), unchanged; round-2 findings B2-1 and B2-6. - Grammar: the lockfile is small or padded past 1 MiB (about 1.2 MiB), and the base commit may lack it (the branch adds it). New declared states large-lockfile-same-plugin and missing-base-lockfile, each with a constructed history in every campaign. - RED: on a6fd192's script the fixed and random campaigns fail; the fixed counterexample is a large lockfile that the edit hook names with the plugin's entries unchanged, where every oracle file was selected and the model owes none. - GREEN: both campaigns pass (random seed 980076029; large-lockfile-same-plugin 3, missing-base-lockfile 20). - Mutant: reading the base with `git show` and no missing-file check crashes the CLI on a missing base lockfile ("exists on disk, but not in <base>"). - Limits: a checkout without origin/main or main is not drawn; the fail-fast there is by inspection. Co-authored-by: Isaac <no-reply@databricks.com>
…ted histories as the framework does
B2-2: the fixed-seed campaign's budget was `numRuns: oracleRuns(BASE_RUNS) +
examples.length`, which check-oracle-run-counts.mjs cannot read, so the
`fixed seed 2234` budget went unrecorded. It now declares
`oracleFixedSeed(FIXED_SEED, { numRuns: oracleRuns(BASE_RUNS) })` and adds the
constructed histories with withOracleFixedHistories. RED: with BASE_RUNS
lowered to 12, lint:changed reported only `property scripts.lint-scope`;
GREEN: it now also reports `fixed seed 2234: run budget lowered from 24 to
12`, and nothing at 24.
B2-3: the random campaign added the constructions to numRuns even on a path
replay, so a replay ran several histories from the path. Both campaigns now
use withOracleFixedHistories, which keeps numRuns at 1 on a path. A replay
with seed 980076029, path 3 runs one history. Both campaigns pass.
Co-authored-by: Isaac <no-reply@databricks.com>
This branch moved the write-order and peer loops out of local-storage-order-oracle.test.ts into local-storage-write-order-oracle.test.ts and local-storage-peers-oracle.test.ts, but test:oracles listed only the first, so the soak no longer ran the other two. Both are listed now. Every oracle test file that the branch adds under packages/db/tests is in the script, and all 90 paths exist. Co-authored-by: Isaac <no-reply@databricks.com>
The ownership comment named handleCollectionCleanup "and the functions it calls" next to a sentence about whole-file entries, which read as a contradiction. It now separates the two rules: a whole-file entry owns the functions defined in its file only, and a #function entry owns its text and every package function the text refers to (owners.mjs in @tanstack/oracles, maintainer decision 2026-11-08). The gap report's ownership analysis over this file alone lists the oracle as the owner of discardTruncateReplay (5 mutants), setReadyIfIdle (6), and cancelAcquisition (2), as well as handleCollectionCleanup (27). Co-authored-by: Isaac <no-reply@databricks.com>
2 of 3 tasks
A server that loads db-ivm's published files without a bundler read process.env once per probing call: two guards per consolidate() cost about 280 ns, which made a 4-record consolidate 2.5 to 5 times slower. The maintainer decided (2026-11-09) that the published build carries no probes. scripts/strip-oracle-probes.mjs is a build-only Vite plugin. It blanks each probe guard declaration and probe statement, which the probe-form lint rule fixes to one shape, and keeps every line break so the source maps stay correct. A probe or guard reference that survives fails the build. db-ivm and offline-transactions wire it. Vitest and the gap report load the sources with their probes. scripts/check-dist-probes.mjs runs after the full build in the Preview and Release jobs, before anything publishes. It fails if any package's built JavaScript still mentions __oracleProbe, which catches a package that gains probes without the plugin. - RED: on 7980a03 the built JavaScript carries 16 probe mentions and 6 NODE_ENV reads in db-ivm, 10 and 6 in offline-transactions. - GREEN: 0 and 0 in both. db-ivm 693 tests, offline-transactions 396. - Mutants: a plugin that removes only the guards fails the build at multiset.ts:103. A package without the plugin fails the dist check and names its four files. The plugin applied under Vitest too fails 40 consolidate oracle tests on reach, so build-only is required. Co-authored-by: Isaac <no-reply@databricks.com>
offline-transactions type-checks its vite.config.ts, which imports scripts/strip-oracle-probes.mjs, so tsc checked the untyped plugin and reported 14 implicit-any errors in the ivm-offline job's Typecheck Test Sources step. A declaration file beside it, as for commonjs-declarations.mjs, gives the import its type. RED: pnpm run typecheck:tests reports 14 errors without the file. GREEN: db-ivm and offline-transactions type-check. Co-authored-by: Isaac <no-reply@databricks.com>
A #function entry owns its text, the functions defined inside it, and its static reference tree within the package, followed transitively (owners.mjs in @tanstack/oracles, maintainer decisions 2026-11-05 and 2026-11-08). The comment named one level of callees; the walk reaches further. Law: the gap report's ownership rule (owners.mjs ownership(): walk() recurses into nested functions and referencesOf(), stopping at the package boundary). Reach: not applicable, documentation only; no behavior changes. RED: the comment's one-level claim contradicts the walk. A scratch ownership() run (outside the repo) lists 11 functions owned by CollectionSubscription.handleCollectionCleanup and 156 by DbClient.hydrate, including discardTruncateReplay, setReadyIfIdle, and cancelAcquisition. GREEN: the comment now states the walk and names those three through "including". Verified by the same scratch run; no owner set changed. Mutant: not applicable, documentation only. Limits: counts are not written into the comment, since they drift with the code. Co-authored-by: Isaac <no-reply@databricks.com>
The stripper is the build-time removal of the oracle state probes
(maintainer decision, 2026-11-09). Until now no test or oracle checked it,
and the dist check only greps for the probe name (round-3 finding B3-7).
Law: for every program the probe-form rule accepts, the stripped program
keeps no runtime reference that came from a guard, and behaves like the
source with process.env.NODE_ENV defined as "production". Where it cannot,
it fails the build with a message. Authority: the 2026-11-09 decision and
the probe-form rule header in @tanstack/eslint-plugin-oracles.
Reach: 16 constructed histories (fixed) plus drawn programs. Each
accepted history enters the declared placement, guard, form, and mention
states (scripts.strip-oracle-probes, createOracleReach).
RED (parent of the fix, the unchanged stripper): constructed histories
and the drawn scan reproduce:
- B3-1: an unbraced if, else, for, while, or do body with a probe takes
the next statement as its body (trace loses marker-1), and a do body
yields unparseable output ("Expected while").
- B3-2: a guard with sibling declarators drops the siblings (the side
effect of `sibling = trace.push(...)` disappears).
- B3-3: a parenthesized probe statement or guard operand is refused by
the stripper ("outside a probe statement") although the rule admits it.
- B3-4: a legal property name, label, or type-position `probing` is
refused as a surviving reference.
- Gate: a file that declares the guard without any probe is skipped
(the gate tests only __oracleProbe), so `process.env` ships.
- A non-guard `const probing = <side effect>` declaration is blanked
and the side effect is dropped (must be refused).
Mutant and GREEN come with the fix commit.
Co-authored-by: Isaac <no-reply@databricks.com>
Both lint-scope campaigns called fc.assert without registering their route, so the gap report saw no route for either. Each now builds its options once, registers oracleRoute(oracleRouteOf(options)) on those exact options, and passes them to fc.assert. Law: the framework's route rule (@tanstack/oracles oracle-route.js): a test registers its route before its campaign runs, and the route names the campaign's declared seed, path, or random seed. Reach: not applicable; the campaigns' reach is unchanged. RED: parent commit, verbose probe that installs globalThis.__oracleRoute and logs each call (scratch config in /tmp, outside the repo). Both tests start and pass (95s and 95s), with no route event. GREEN: same probe after the change. fixed-seed campaign registers "fixed seed"; random-seed campaign registers "random seed". Both pass. Mutant: the parent commit is the hostile case: removing the registration produces no route event, which the probe rejects. Limits: a path replay registers "fixed history" for the random campaign, which is covered by the replay commit's run, not here. Co-authored-by: Isaac <no-reply@databricks.com>
The oracle declared a reach state for every drawn unit kind, including the rule-rejected units that no admitted history can enter, so the reach check could never pass. The declaration now lists the admitted kinds. Rule-rejected draws are counted, not declared. No production change; the RED result on the stripper is the commit before this one. Law: scripts.strip-oracle-probes (unchanged). Reach: declared states are the guard tops, admitted unit kinds, and probe forms; the fixed and random campaigns reach each at least twice (ADMISSION 195 accepted, 159 rejected by the rule). RED: the parent commit's run (see the previous commit), unchanged. GREEN: not applicable to this commit; see the next commit. Mutant: see the next commit. Limits: as in the oracle's header. Co-authored-by: Isaac <no-reply@databricks.com>
The stripper matched the probe form loosely, so it failed or broke published code in the cases that the probe-form rule admits. It now mirrors the rule's form, and it fails the build instead of emitting a changed program. - B3-1: a probe statement that is the sole body of an unbraced if, else, loop, or label is replaced by `;`, not whitespace, so the next statement does not become its body. A do body no longer yields unparseable output. - B3-2: a guard declaration with sibling declarators keeps them; only the guard's own text and its comma are blanked. - B3-3: parentheses are unwrapped in the statement, the guard operand, and the guard initializer, as the rule does. - B3-4: a property name, a label, a JSX attribute name, a binding's property key, and a type position named `probing` are not references and are kept. Every `__oracleProbe` left still fails the build, because the rule rejects every mention outside a probe. - A guard declaration is blanked only when its initializer is the guard form. A non-guard `const probing = <side effect>` is refused, not silently dropped. - The build check runs for a file that declares the guard, not only one that names the probe. A guard with no probe no longer ships `process.env.NODE_ENV`. - Refused: a value read, a binding, a `for`-head guard, and a non-guard `probing` declaration, each with a build error naming the identifier. check-dist-probes also rejects a surviving `process.env.NODE_ENV` in a built package that wires the plugin (db-ivm, offline-transactions). Law: for every program the probe-form rule accepts, the stripped program keeps no runtime reference from a guard and behaves like the source with process.env.NODE_ENV defined as "production". Authority: maintainer decision 2026-11-09 and the probe-form rule header. Reach: scripts.strip-oracle-probes, fixed and random campaigns together: 195 histories accepted by the rule, 159 rejected by it (not checked), 49 accepted histories that the stripper must refuse. Every declared state is entered at least twice; the lowest counts are guard-top-non-guard (6), unit-do-body (11), and unit-labeled-body (9). RED (parent commit bb3a7de, unchanged stripper): the constructed histories failed on the unbraced if body (trace lost marker-1), the else body, the for/while/do bodies (unparseable output for do), the sibling guard (sibling side effect dropped), the parenthesized probe statement and guard operand (refused), a legal property name, label, or type position (refused), a guard in a file without a probe (process ships), and a non-guard side-effect declaration (side effect dropped). A drawn scan of 400 programs reproduced each class. GREEN: pnpm exec vitest run --config scripts/oracle-scripts.vitest.config.mjs scripts/strip-oracle-probes-oracle.property.test.js: 2 passed. node scripts/check-dist-probes.mjs: clean after building db-ivm and offline-transactions; a planted process.env.NODE_ENV in a built wired package is rejected. Mutant: six hostile edits to the fixed stripper, each rejected by the oracle's constructed histories on a behavioral comparison: M1 blank without `;` (trace expected [marker-1], got []), M2 drop siblings (sibling side effect lost), M3 no parenthesis unwrap in the statement (refused), M4 gate on the probe name only (refused value read not reached, reach fails), M5 any-initializer guard (non-guard side effect dropped), M6 type positions refused (refused a legal program). Limits: the grammar draws one exported function with up to six units; no imports, JSX, or bundler; the host process is a stub. Type-only `__oracleProbe` mentions are refused; the rule rejects them too, so the stripper must change when the rule admits them (G12). The dist check covers only the two wired packages. Shipped bytes are unchanged: the stripper runs in the build and its output for these packages is the same as before. Co-authored-by: Isaac <no-reply@databricks.com>
A path replay runs one property's recorded history, and every other test stands aside through the replay-skip helper. The fixed-seed campaign ignored that rule, so a replay of scripts.lint-scope also ran 24 fixed-seed histories that build temp git repos. The random campaign did not stand aside for other properties either. Both now use oracleReplayTest: the fixed-seed campaign with no property (it stands aside on any path replay), the random campaign with PROPERTY (it stands aside for other properties). Law: the framework's replay rule (@tanstack/oracles oracle-config.js: oracleReplayTest, oracleReplaySkips): a path replay runs only its recorded property, and the other tests skip. Reach: the non-replay run still reaches the reach states (reach.assertReached runs in afterAll when no replay is requested). RED: parent commit, replay TANSTACK_DB_ORACLE_SEED=980076029, PATH=3, PROPERTY=scripts.lint-scope. The fixed-seed campaign runs and passes (97s); the witness shows the random campaign's one history (numRuns 1). GREEN: same replay. fixed-seed campaign is skipped; random-seed campaign runs once, route "fixed history", witness numRuns 1. Replay of ivm.distinct-collisions: both campaigns skipped. Full run without replay: both pass (94s, 93s). Mutant: the parent commit is the hostile case; it fails the RED check above. Limits: the replay checks cover one recorded path. The skip rule is the framework's; replays of fixed histories and calibration are not exercised here. Co-authored-by: Isaac <no-reply@databricks.com>
…ade #2374) Conflicts resolved as the standing rule says: main's tooling, with this branch's oracle changes reapplied. - package.json, packages/db/package.json: main's dependency versions, plus the @tanstack/oracles and @tanstack/eslint-plugin-oracles previews. fast-check stays at ^4.10.2 here and in db-adapter-conformance. - packages/db-ivm/vite.config.ts: main's publishableModuleOutput build output, plus the stripOracleProbes() plugin. - electric-sdk-delivery-oracle: main's resume-checkpoint check and model, plus this branch's owns entry. - solid-db row-materialization-oracle: main's keyed visibility histories, plus this branch's case route. - pnpm-lock.yaml: resolved from the public registry by a temporary CI job (run 42653737079, which also checked a frozen install), because the local registry proxy does not yet carry #2374's versions. Co-authored-by: Isaac <no-reply@databricks.com>
The db-replay job failed: `names an owner for every registered property` reported scripts.strip-oracle-probes, which the stripper oracle registered without a manifest owner. The manifest now names scripts/strip-oracle-probes-oracle.property.test.js, beside scripts.lint-scope. RED: CI db-replay on ecb1679 (expected ['scripts.strip-oracle-probes'] to deeply equal []). GREEN: the test passes locally. Co-authored-by: Isaac <no-reply@databricks.com>
TanStack/oracles#1 and #2 merged; main is 98a9610. Both packages move from the #2 preview (b937148) to the pkg.pr.new preview of that main commit, which has the same source tree as the preview this branch tested (both are tree 95dbce3). The coverage map's two framework links now point at the main commit, because b937148 was on #2's deleted branch. pnpm-lock.yaml was resolved from the public registry by a temporary CI job (run 42664121157, which also checked a frozen install), because the local registry proxy cannot fetch the new previews. Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
🎉 This PR has been released! Thank you for your contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
This PR moves TanStack DB's oracle tooling into the new
@tanstack/oraclespackages. It also makes every oracle file state what production code it judges and how each test runs. The framework (oracleProperty, reach, replay, the controlled clock), the gap report, and the lint rules now live in TanStack/oracles, and this repository consumes them. It also fixes one production defect that the stronger oracles found (closes #2334).@tanstack/oraclesand@tanstack/eslint-plugin-oraclescome from TanStack/oracles#1 and TanStack/oracles#2, which have merged. They are pinned to the pkg.pr.new preview of TanStack/oracles main at98a9610, which has the same source tree as the #2 preview that this branch tested. The pins move by hand when TanStack/oracles main changes.The branch merges main a third time, for the production-bundle benchmark (#2379) and the dependency upgrade (#2374). Its lockfile came from a temporary CI job, because the local registry proxy did not yet carry #2374's versions.
The branch had 152 commits before review. It merges main twice: once for the work that landed while it was open, and once for the Vitest 5 and Vite 8 upgrade (#2371). The sections below follow its parts.
The oracle tooling moves to
@tanstack/oraclespackages/db/tests/oracle-*.tsbecome one-line re-exports of@tanstack/oracles.oracle-config.tskeeps TanStack DB's property registry and callsconfigureOracles({ envPrefix: 'TANSTACK_DB_ORACLE', … }), so every replay command and CI setting keeps its name.scripts/oracle-gaps/is deleted.pnpm gap-report --package packages/<pkg> --source <file>runs the package'soracle-gapsbin with this repository's prefix.scripts/eslint-plugin-oracle/is deleted.eslint.config.jsand the agent lint hooks use@tanstack/eslint-plugin-oracles, with the settings inscripts/oracle-lint-settings.mjs.agent-lint-feedback.mjsandcheck-oracle-run-counts.mjsstay here, because they are specific to this repository.The framework changed its rules during its review, and this branch follows them:
OracleDataErrorthat names the value's path. TanStack DB's drivers already record only data, so no driver needed a change.oracleRoute(oracleRouteOf(parameters)), and only a seed declared withoracleFixedSeedroutes asfixed seed.configureOraclesthrows if it would change a replay selection that something already read.Ownership and routes in every oracle file
Every oracle file declares the code it judges with
declareOracle({ owns: [...] }), and every test registers the route it runs:fixed seed,fixed history,case,random seed, orcalibration. The gap report judges a mutant only by the tests of the oracles that own it, and counts a kill only on a reproducible route.createDeferred,errors.ts, query builder functions), and this PR deletes them. A mutant in a helper would count kills for laws that do not judge it.oracleFixedSeed. A route probe ran every package's oracle files with a fixed seed and compared each test's route with the parent commit. No test moved fromfixed seedtorandom seed, and 524 tests that had routed as random or as nothing now route as fixed.oracleRoutefrom the browser-safe@tanstack/oracles/declarationentry. Before this change, the browser build of both apps pulled in Node built-ins.oracleFixedSeed(seed, { numRuns }).Production changes
Fixed: a restarted sync run's row was hidden from a live-query observer (#2334). It affects observers, and so
useLiveQueryin every adapter. An affected observer stays subscribed through a Collection's cleanup and restart. Its Collection subscription kept the rows it had published before cleanup. The new sync run could write a row equal to one of those, or an on-demand restart could load it again. The subscription then dropped it as already shown, and the observer showed[]. The observer now subscribes with an internal option that discards those rows and sent keys on cleanup. Other direct subscribers keep the retention that ARCHITECTURE.md documents.restart-rewrites-row-from-before-cleanup(10 to 20 histories per 100-run campaign) andrestart-reacquires-row-from-before-cleanup(4 to 7). Constructed histories enter both in every campaign. The oracle gained an eager or on-demand source dimension and a seed of 0 to 3 rows or none.view(rows[]where the model owes the row). The original counterexample was seed 1779366423, path30:6:8:8:8:8, after attach, ready, write, cleanup, restart, write.cleaned-up, and the stale copy taken before the discard. The others are the discard only in granular mode, the sent keys kept, and the published rows kept.Changed: the published build carries no state probes. The branch adds state probes to
MultiSet.consolidate,consolidateKeyed, andconsolidateByin db-ivm, and to the offline executor. Each probing function starts withconst probing = typeof process === 'undefined' ? false : process.env.NODE_ENV !== 'production', and each probe isprobing && globalThis.__oracleProbe?.(name, condition). An unbundled module with noprocessglobal runs without aReferenceError.A bundler that sets
NODE_ENVdrops the probes, but a server that loads the published files without a bundler did not. Measured in plain Node, the two guards in eachconsolidate()call cost about 280 ns, so a 4-record consolidate was 2.5 to 5 times slower. The maintainer decided (2026-11-09) that the published build carries no probes:scripts/strip-oracle-probes.mjsis a Vite plugin that runs only invite build. It blanks each guard declaration and probe statement and keeps every line break, so the source maps stay correct. A probe or guard reference that survives fails the build. db-ivm and offline-transactions use it. Vitest and the gap report load the sources, so the oracles keep their probes.scripts/check-dist-probes.mjsruns after the full build in the Preview and Release jobs, before anything publishes. It fails if any package's built JavaScript still mentions__oracleProbe.NODE_ENVreads in db-ivm, and 10 and 6 in offline-transactions. GREEN: 0 and 0. db-ivm passes 693 tests and offline-transactions 396.multiset.ts:103. A package without the plugin fails the dist check, which names its four files. The plugin applied under Vitest too fails 40 consolidate oracle tests on reach, so the plugin must apply only to the build.A review found forms that the probe-form rule accepts but the first stripper broke: a probe that is the only body of an unbraced
if,else, or loop; a guard declared with other names; and parentheses around a probe or its guard. A new oracle,scripts/strip-oracle-probes-oracle.property.test.js, now checks the stripper on generated programs:NODE_ENVdefined asproductionis the reference. The production path is the plugin, then a type strip. The oracle compares the trace, probe calls, the thrown error, and the guard identifiers in the emitted code.;, keeps the other names of a guard declaration, ignores parentheses, and refuses any other use of the guard name.;and a drop of the other names.__oracleProbeis refused, as the rule refuses it. If the rule admits it later, the stripper must change in the same pull request.scripts/check-dist-probes.mjsalso fails now ifprocess.env.NODE_ENVstays in the built JavaScript of a package that uses the plugin.The remaining source changes are comments and documentation.
Other parts of the branch
oraclePropertywith constructed histories, the oracle vocabulary follows the glossary, and db moves to fast-check 4. The lint rules that this work added are now in@tanstack/eslint-plugin-oracles.@fast-check/vitestaccepts fast-check 3 or 4. An override inpnpm-workspace.yamlgives it fast-check 4, sotest.propcampaigns and their types use the same copy as the oracles. Two cross-formulation calibrations from main recorded fast-check 3 shrink paths, and they now record the fast-check 4 paths. Each documented direct replay still fails once, at checkpoint 0.db-adapter-conformancenow names the exact law that must reject its mutant. Before, it took the first law that a history broke. The move to fast-check 4 changed which law failed first for one mutant.tscrun.Run budgets that moved
The run-budget check reports
run budget removedfor these budgets. Each one moved with its oracle to TanStack/oracles, where it runs with the same budget in that repository's CI:oracle.declared-reach(200 runs) and its fixed seed 1packages/db/tests/declared-reach-oracle.property.test.tspackages/oracles/tests/declared-reach-oracle.property.test.tspackages/db/tests/oracle-campaign.test.tspackages/oracles/tests/oracle-campaign.test.tslint.part-reach(3,000 runs, fixed and random)scripts/eslint-plugin-oracle/packages/eslint-plugin-oracles/tests/Maintainer decisions
These decisions, all from 2026-11-08, shape the code in this PR:
probingguard above.Object.definePropertyfor speed, and this is a stated limit.configureOraclesrefuses to change a selection that something already read, and a model must accept a trace cut short.Confirmed by the maintainer (2026-11-09): ARCHITECTURE.md now says that a live-query observer's subscription keeps no rows across cleanup, while other direct subscribers keep theirs. The two rules already existed: the hydration oracle's law 2 and the documented retention. The new sentence records the exception.
🔬 Oracle Coverage
The #2334 fix is the one behavior change. Its evidence is above. The other changes add declarations, routes, and fixed-seed declarations, move tooling, or re-record witnesses. They change no law, and the oracles that own the affected behavior pass unchanged.
The gap report's ownership check (
--only none) accepts every owns entry and route in 17 packages. It refuses three packages, because one oracle in each fails with no mutant active under the gap report while it passes in plain Vitest:bound-schema-options-oracle: "Invalid source for live query … not a Collection". The gap report probably loads a second copy of@tanstack/db.electron-ipc-readiness-oracle.These are defects in the gap report's module resolution, recorded for TanStack/oracles. CI does not run the gap report.
✅ Checklist
pnpm test. I ran each package's tests at two workers, one package at a time.Results on this branch:
pnpm lint:checkpasses.pnpm lint:changedreports only the moved budgets above.Follow-ups
The three gap-report refusals above.
About 115 tests in db and db-sqlite-persistence-core register no route, as on main. Most are unit tests in
oracle-replay.test.ts.A few campaigns take no seed at all, so they route as
random seedand cannot be replayed by seed.Oracles that combine a raw
fc.assertwith a reach minimum above 1 have seed-dependent reach. They should move ontooraclePropertyconstructed histories.The reference adapter's
stale-outage-valuemutant is described as a HEALTH-2 case, but only HEALTH-1 rejects it.live-query-observer-hydration-oraclehas twodeclareOracleblocks that both ownlive-query-observer.tsandDbClient.hydrate, so the gap report lists those owns twice.collection-gc-transaction-oracle's random campaign missed the reach statesettled-while-subscribedat seed -977831430. No law failed. It needs a constructed history for that state.🚀 Release Impact
The changeset covers the #2334 fix in
@tanstack/db. The published files of db-ivm and offline-transactions carry no state probes.This pull request and its description were written by Isaac.