Skip to content

feat: bind calibrations to declared histories - #31

Open
KyleAMathews wants to merge 15 commits into
mainfrom
codex/calibration-history-bindings
Open

KyleAMathews wants to merge 15 commits into
mainfrom
codex/calibration-history-bindings

Conversation

@KyleAMathews

@KyleAMathews KyleAMathews commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Calibrations currently search fixed histories, constructions and generated samples until a mutant applies. Each candidate can open a real database. In the motivating DB release, PowerSync startup calibrations consumed 1,455.762 of the file’s 1,926.534 test seconds.

This change lets a calibration select one declared history. Its clean run must pass and prove its declared reach before the mutant must fail under the named law and optional difference requirement. A failed selected premise fails immediately.

calibrations: [{
  name: 'drops the committed row',
  law: 'durable publication',
  history: { constructed: 'a committed write' },
  corrupt: dropCommittedRow,
}]

Generated sources accept fixed or constructed names. Enumerated sources accept zero-based indices or canonical labels. Invalid selectors fail registration before any driver runs. Fixed and enumerated selections preserve the original identity. Each constructed driver execution builds a fresh history from a stable seed derived from property id and calibration name, then resolves its plan once.

probeHistories binds the two automatic controls independently. A clean entered history detects a probe forced not entered; a clean absent history detects a probe forced entered. Entered selection can infer a unique construction. Absent selection is explicit. Both paths use the ordinary verdict ranking, so an earlier recorded violation survives a later driver error or controlled-clock spin. Callback failures retain calibration and history context.

Unbound calibrations retain search during migration. Registration warns once per property/process and names every unbound calibration; collection remains silent. Bound calibrations keep the calibration route for gap classification. Main’s named campaign/determinism units and UNIT replay are integrated by normal merge. Calibrations still use their stable seed; fresh calibration seeds and replay support remain follow-up work. DB migration and eventual search removal remain separate.

Oracle evidence

The append-only review and repair record records laws, pre-edit gates, receiving paths, mutants and finite limits.

  • Law: A binding selects its authored original independently, proves the clean premise and declared reach, then rejects the designated mutant. Real Vitest receivers preserve complete registration, warnings and failures. Successful campaign units retain their own authored budgets.
  • Reach: Required constructions cover selection/origin, both forced premises, metadata, independent mixed bindings, named seeds, UNIT/replay, successful per-unit budgets and every receiving context. Host forwarding reach requires actual invocation during seed selection. Four additional states require actual bound callback failures in ordinary, reordered, seedless and environment children.
  • RED: The combined oracle ran against unchanged main before source integration: seven binding/control checks and six metadata receivers failed their intended comparisons; seventeen UNIT neighbors passed. Owner repairs additionally demonstrated false failures for legal warning ordering/host forwarding and false acceptance of wrong warning identities or omitted forwarding.
  • GREEN: Focused merged binding/metadata/UNIT checks pass 30/30. The final HOST gate passes 15/15 and the required-forwarding receiver passes 3/3. Types, package build, lint and formatting pass. Plugin assertions pass 6,438 checks plus two package-citation checks. The actual 96-test e2e inventory partitions across five shards without errors. The combined live callback/origin follow-up passes 19/19 focused checks; 36 bad diagnostics and 15 opaque-label association faults are rejected, while all nine callback-kind/selection neighbors pass.
  • Mutants: Wrong identity, selector sharing, forced polarity, required differences, UNIT seed offsets, constructed/generated budgets, warning identity/multiplicity and host draw/use/forwarding are rejected at their named comparisons. Setup failures and narrow insensitive witnesses are retained separately.
  • Limits: Finite synthetic histories and name/fault families under actual Node/Vitest. No downstream DB integration, browser/provider coverage, entropy-quality or measured release-time improvement is claimed. The record preserves the historical missed pre-edit gate (CHRON-1); current tests cannot repair its chronology.

Current validation

Pushed head c18a30f: core/package, plugin, gap oracles and the full Vitest 4 framework suite pass. The Vitest 5 framework run passes 756/757; its remaining timeout-history determinism failure also reproduces on Vitest 4. The recorder retains timing-dependent post-timeout attribution outside the stated contract; a repair is in progress with late-error and per-invocation guarantees retained. The local full sweep was interrupted after both 48-unit campaigns; it is not a full-suite GREEN.

Fresh oracle review also found diagnostic presentation assumptions. Scratch repairs accept alternate source layouts and retain exact selected names/cause across every active receiver; further review found an opaque-reason delimiter assumption, now under repair. No framework production change is indicated by these findings. Final combined-owner validation and independent review remain pending. The append-only record will carry the completed evidence before a readiness verdict. No dependency override or lockfile workaround is committed.

Law: An explicit history binding runs only that history. The selected clean
run and actual reach must pass before exactly the designated law rejects the
mutant. Authority: approved maintainer proposal and review, 2026-11-11.

Reach: Constructed framework histories exercise direct selection, invalid
selectors, isolated constructed drivers, and both forced-probe controls.
Fixed seed 1 reached bound-calibration 6 times, registration-error once, and
bound-constructed-driver once. The random campaign reached them 3, 4, and 1
times respectively. Existing run budgets remain unchanged.

RED: On unchanged source at 49f3bd1, the extended framework self-oracle's
'fixed history: bound calibrations use only' test rejected two driver calls
where the binding permits one.

GREEN: 22 focused binding regressions and the selected clean-driver-error
regression pass. Build, lint, types, format, publint, sherif, 56 core tests,
and 4320 ESLint plugin tests pass. Full framework and gap checks are running.

Mutant: The same comparison rejects restoring search (driver count 2 > 1),
skipping reach, accepting an inapplicable corruption, accepting the wrong
law, and cloning a built history that carries identity metadata.

Limits: Unbound calibrations retain search with named registration warnings
until consumers migrate. Fresh seeds await UNIT replay support. DB migration,
a matched DB timing comparison in CI, and the later F27 rebase remain separate.
@changeset-bot

changeset-bot Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: c18a30f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@tanstack/oracles Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Pre-merge checks: Passed 1 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Passed Passed checks: 1 passed

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pkg-pr-new

pkg-pr-new Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/TanStack/oracles/@tanstack/eslint-plugin-oracles@31
npm i https://pkg.pr.new/TanStack/oracles/@tanstack/oracles@31

commit: fc4bad8

@KyleAMathews
KyleAMathews marked this pull request as ready for review November 11, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant