[KyleAMathews]

Summary

The gap CLI no longer skips a mutant, or marks it dead code, on the basis of types (maintainer decision, 2026-11-11). A mutant that no input of its annotations distinguishes now carries a label. Its survivor is reported in a new class, outside-annotation.

This pull request replaces PR #8's type-based skip on main. It also replaces the held F11 and F17 branches, which were never pushed. It keeps F17's two operators and the analyses that still earn their place, which now feed labels only.

An unsound type analysis can now only mislabel a survivor. It can no longer hide one.

What changes

• PR feat(oracles): an oracle for mutant validity; dead code and the -0 boundary hint #8's ordering-swap skip becomes the label. For a number tie at a nonzero literal, its input class is "a string of the literal's digits". For a bigint tie at a nonzero bigint literal, it is "a number in place of a bigint" (round 8).

• The forced-tie skip for two annotated bigints becomes the label, with the input class "-0 and 0 in place of the bigints".

• A site under identical branches that only an object's coercion distinguishes gets the label "an object whose valueOf has an effect".

• An optional chain on a parameter whose annotation admits no nullish value is always emitted, without a label. The CLI does not read the package's strict null checks, and without them the annotation admits null (round 6).

• The survivor class outside-annotation has a directive that names a step, so it follows the exit law (law 7). The directive is this:

  Add an owner history that reaches line N with <input>, which the annotation excludes, so that it changes an asserted result, and assert on the result.

• An argument typed any is an input that the types allow. So the class claims only that the annotations exclude the input, not that no legal input exists.

• F17's operators come in. return-undefined evaluates the returned value and then drops it. optional-chain reads a?.b as a.b.

What still skips or marks dead code, and why it needs no types

• return void x is no site, because the mutant evaluates x and returns the same undefined.
• return undefined is no site only where nothing in the file binds the name, and the file has no direct eval and no with. The mutant is then the same program.
• A logical operator on one local name (a && a) is dead code, because both sides read the same value. A parameter read before its own declaration ends, a loop's lexical variable read in its own head, and a let or const are no local names.
• A site in a pure condition under identical branches is dead code. No ordering comparison may run or not because of the site, so no coercion changes.
• A forced if whose other side implies x === L is dead code. L must be a literal that === matches only in its own value: a bigint, or a nonzero number. x must be a local name.
• An optional chain on a string or number literal is dead code. So is one on a module-level const that holds such a literal. The const must not be exported by a modifier, ambient, or in a namespace. Its file must have no direct eval and no with. An export list keeps the module's own reads local in TypeScript's CommonJS emit, so it does not count.

Each of these depends only on JavaScript's semantics. The validity oracle checks each one against inputs that JavaScript allows.

The fifth review's causes

1. A function-scoped var in a nested block, and an import binding, now resolve. Round 7 moved all binding resolution to TypeScript's binder.
2. A literal's value is read as JavaScript reads it, so 0x0, 0.0, 0b0, and 0x0n are zeros. This also fixes main.
3. A namespace's const, or one exported by a modifier, is never a constant receiver.
4. A pure condition's dead-code mark needs no ordering comparison to run or not because of the site. A let or const is no longer a local binding, because a closure can read it before its assignment.
5. Overload naming, 6. the package's other program, and 7. the other strict options now matter only to a label. A mistake there names the wrong class for a survivor. It never hides the survivor.

The F11 and F17 trust machinery is gone: the checker plumbing, trusted, the exhaustive-switch skip, and strict null checks from the tsconfig. The label analysis reads parameter annotations by syntax, as PR #8 did.

Size

Minified with esbuild, the CLI's shipped files are 47,074 bytes, and 18,021 bytes with gzip. They were 48,400 bytes (18,284 with gzip) on F17 at b6a4c2c, which this replaces. So the change saves 1,326 bytes, or 263 bytes with gzip.

Against main at d38af38, they grow by 3,870 bytes (1,171 with gzip). The growth pays for F17's two operators and for the soundness fixes. Round 7 saved 1,315 bytes (178 with gzip), because TypeScript's binder replaced the hand-written scope analysis. Round 8 adds 75 bytes (26 with gzip).

Round 6

The round-6 soundness review of fc6a2e2 found 9 cases from 4 causes, and one accuracy problem in the labels. Each cause is now a generated dimension of the validity oracle, judged by running the program's text. Fixed programs had missed whole classes twice.

1. Dead zones. A logical swap under identical branches was dead code where its operand could be in its dead zone, as in function f(c, b = (c && a ? 1 : 1), a). A parameter read before its own declaration ends, and a loop's lexical variable read in its own head, are no longer local names. The generated programs cross the context (body, parameter default, for (let …) head, for (const … of …) head) with an earlier or a later binding, at &&, ||, and ??.
2. Bindings of undefined. A function or class expression's own name bound it unseen. Every declaration's name now binds, and a method name, a label, or an export specifier does not. The generated programs hold each binding form and each form that only names undefined.
3. Dynamic scopes. A direct eval or a with anywhere in the file now keeps every return of undefined and every constant receiver. The with is found in the syntax tree, not by a regular expression. The generated programs are sloppy scripts with a direct eval, a with, or neither. Each holds a return of undefined, a constant receiver, or a && a. Under eval, a && a stays dead code, because an eval cannot make a read of a local name run code. The oracle checks it.
4. Returned calls. A returned call and import() are now return-undefined sites. A test that checks only a call's effect kills value-undefined, never this mutant. The narrowed family's returned calls and a hostile return import('./five') judge it.
5. Label accuracy. The label law is now exact: a label only where an annotation excludes every input that distinguishes the mutant. The coercion label needs annotations that admit primitives only. The generated programs hold if (c || p < q) at number, string | number, object, unknown, and in JavaScript.

The reviewer found these sound, and they stay. They are a && a on a local, forced ties with an exact literal, and an ordering swap under identical branches. Also the optional chains that are not dead code, and return void x.

Round 7

The round-7 review of 10853a2 found 7 cases from 4 causes. It found the three departures of round 6 sound. The causes were dead zones in for…of, for…in, and catch patterns, and a var in a class static block. Also (eval)(…) as a direct eval, and a swap label beside a coercing comparison in the same && chain.

These were rounds 5 to 7 of the same classes, so the fix replaces the construct-by-construct checks with general rules. The maintainer also asked that scope questions come from real tooling, not from a hand-written walk of the syntax tree.

• Scope resolution: TypeScript's binder now answers every binding question. The instrumenter builds a one-file program with noLib and noResolve, and asks the checker which declaration a name resolves to. It reads only symbols and declarations, never types. It works for JavaScript with allowJs. This deletes 148 lines of code: declarationOf, varOf, importsName, bindsName, declaresName, and enclosedByFunction. It also deletes the binding scan for undefined and the dead-zone checks per construct.
• Dead zones, one rule: a parameter, a caught error, a let, or a const is no local name before its declaration ends. A for…of or for…in head's iterated value counts as before the end. A read from another function, or across a switch case, also counts, because that code can run before the declaration. A var has no dead zone.
• Direct eval: a call of the global eval through parentheses, as, !, satisfies, or a type assertion is direct. The binder decides that the name is the global one.
• Labels: a swap or bigint label needs every comparison in its condition to compare primitives.
• with: the binder resolves no name inside a with, so a name there is never local. The file-wide with check stays only for a returned undefined. The check that localBinding had for it guarded nothing, and its mutant passed, so f16fce3 deletes it.

I did not need oxc or @typescript-eslint/scope-manager. The binder answered each question, and both would add a dependency.

The generated dimensions are the binding forms crossed with the read positions:

• Binding forms: a parameter, a destructured parameter, a let, a const, and a class. Also a caught error's pattern, a for…of or for…in pattern, a nested pattern, and a var (nested, or in a static block).
• Read positions: before the declaration ends, inside it, after it, and in a hoisted function.
• The scope family gains (eval)(code), (eval as any)(code), and the indirect (0, eval)(code). A chain family puts a second comparison beside the swap, at a primitive and at object.

Round 8

The round-8 review found that the binder version fixes everything from rounds 5 to 7. Three findings remained.

1. A direct eval through a local binding. A sloppy call of the identifier eval is direct whenever its binding holds the real eval. That binding can be a parameter, a destructured var, or a caught error. Now any call whose unwrapped callee is the identifier eval makes the file's scopes dynamic, whatever it resolves to. The scope family generates the three bindings.

2. Bigint labels without strict null checks. A bigint admits null, which ties with 0n. The CLI does not read the package's tsconfig, so the label must hold without strict null checks too. A bigint swap label now needs a nonzero bigint literal at the tie. The forced-tie label stays, because === never ties null with a bigint. The label law's declared inputs now admit null and undefined, and the first family judges the zero literal, the nonzero literal, and the two-parameter cases.

3. Async functions and generators. The oracle saw every promise and iterator as "object". The model and the driver now await a returned promise, and drive a returned iterator for eight steps at most. They compare how each run settles: fulfilled with which value, rejected with which error, or which yields, return, and throw. The model's runs are computed when the file loads, because a settled run is asynchronous.

   The dead-zone, scope, binding, and returned-call programs are generated again as async functions, generators, and async generators. The reviewer's generator witness is a generator that yields before a let. Production needed no change for this class. The mutant that makes a parameter local at any position now fails 27 async or generator programs, async function f(c, b = (c && x ? 1 : 1), x) among them. The mutant with no dead zone for a lexical name fails 60, the yield witness among them.

Round 9

The round-9 review found no remaining unsound skip, dead-code mark, or wrong label in production. It found the bigint forced-tie label sound, because only -0 and 0 tie at two values under ===. It also found the model independent of production: its runs come from the grammar's text, mutateSite, and ts.transpileModule.

The last changes are to the oracle:

• The law's limits now state the blind spots of the settled comparison, each of which the reviewer found. None reaches a defect in today's grammar.
  • An iterator is compared for at most eight steps.
  • Only next() is driven, never return(), throw(), or a finally.
  • A rejection or a throw compares by its error's name, not its message.
  • A thrown value that is no error compares by its kind, so throw 1 and throw 2 are alike.
  • A fulfilled object compares by its kind.
  • A thenable is awaited as a promise is.
  • A rejection that floats unhandled is not compared.
• A run that has not settled after two seconds is recorded as such, and its program's test fails. So a program that never settles cannot hang the file's collection.
• A thrown value's name is guarded, so a throw null no longer throws in the oracle.

This pull request changes no end-to-end test name and adds no end-to-end test. The shard partition of #35 holds: partitionErrors on vitest list --json finds 96 tests and no error.

Changes

• schemata.mjs: no type skip. outsideOf and swapOutside compute the label, and add records it. The dead-code rules get the fixes above. F17's operators are included.
• classify.mjs, report.mjs, index.mjs: the outside-annotation class, its directive, and its legend placeholder.
• capture-setup.mjs: the switch's chain, for optional-chain.
• gap-report-mutant-validity-oracle.property.test.mjs: the new law, the foreign inputs, the label law, and the hostile programs.
• gap-report-classify-oracle.property.test.mjs: the label in its grammar, the class in its model and directives, and a constructed history.
• gap-report-model-oracle.mjs, gap-report-oracle.property.test.mjs: F17's operators, and the class in the end-to-end class list.

Evidence

• Law: no mutant is skipped or marked dead code on the basis of types. Every emitted mutant that is not dead code differs from the program on some input that JavaScript allows. No site of the operator contract is skipped. A dead-code mark sits exactly where no input distinguishes the mutant. A mutant that no declared input distinguishes, and some other input does, carries the label. Erasing a hostile program's types changes none of its sites and no dead-code mark. Authority: the maintainer's decision of 2026-11-11, and freeze.md, "Undetectable mutants are invalid".

• Reach: the validity oracle is enumerated, and its reach test passes. The label state and the impure-operand state are entered. So are an annotated optional receiver, a constant receiver in each language, and each excluded and shadowed return. Each of the 83 hostile programs has an input that detects a mutant, and each of the 49 round-6 programs decides its law.

• RED: on main's production with this oracle (3506293), the validity oracle fails 32,247 of 32,972. Most failures are F17's missing operators. Among the hostile programs, main fails the type-erasure law for PR feat(oracles): an oracle for mutant validity; dead code and the -0 boundary hint #8's swap and forced-tie skips. These include the short-circuit cases a < 1 && check() ? a : 1 and a < b || check(). Main also marks if (a !== 0x0), 0.0, and 0b0 as dead code, and an input distinguishes each. Its coercion marks under identical branches fail the same way. The classify oracle fails 2 of 11, because main has no outside-annotation class.

• GREEN: on c0fc574, the validity oracle passes 32,972 of 32,972, and on 6b03ea0 it passes 32,973 of 32,973. The static, classification, and probe-form oracles pass 31 of 31. The unit tests pass 56 of 56, and the framework suite passes 120 of 120. The end-to-end oracle's fixed seed 20261105 passes under the lock, in 295 s. After the merge of origin/main c34b693 (fc6a2e2), the validity oracle passes 32,973 of 32,973. The static, classification, and probe-form oracles pass 31 of 31, and the unit tests pass 56 of 56. The framework suite passes 124 of 124, and the end-to-end fixed seed passes under the lock, in 494 s.

• Mutant: 10 hostile mutants each restore a type-based skip or weaken a syntactic rule, and the validity oracle rejects every one:

  • PR feat(oracles): an oracle for mutant validity; dead code and the -0 boundary hint #8's swap skip restored: 4,256 failures.
  • The forced-tie skip restored: 144.
  • An annotated optional chain skipped: 11.
  • A labelled site marked dead in place of the label: 4,412.
  • Zeros read by their text: 3, the 0x0, 0.0, and 0b0 programs.
  • Coercions ignored under identical branches: 1,946.
  • Whatever binds undefined ignored: 12.
  • An exported constant receiver trusted: 1.
  • A let or const as a local binding: 1, on the closure program that 6b03ea0 adds. It survived until that program was added.
  • The label dropped: 1,944.

• Round 6: RED: on fc6a2e2, the extended validity oracle fails 42 of 33,022. The failures cover all 5 causes: 9 dead-zone programs, 4 binding programs (2 missed, 2 over-bound), and 3 scope programs. Also 19 returned calls (16 narrowed programs, the generated call, and 2 hostile programs), 3 coerced programs, and 4 chain labels. GREEN: on 10853a2, the validity oracle passes 33,022 of 33,022. The static, classification, and probe-form oracles pass 31 of 31, and the unit tests pass 56 of 56. The framework suite passes 124 of 124, and the end-to-end fixed seed 20261105 passes under the lock, in 202 s. Mutants, one per fix, each rejected:

  • a parameter read in its own dead zone as local: 3 failures.
  • a loop head read as local: 6.
  • a function or class expression's name binding nothing: 2.
  • every name of undefined binding it: 8.
  • a direct eval leaving scopes static: 2.
  • a with leaving scopes static: 1.
  • the returned-call skip restored: 20.
  • a coercion label whatever the annotation: 3.
  • The export-list check's mutant passed (0 failures). TypeScript's emit shows that the check guards nothing, so 10853a2 deletes it.

• Round 7: RED: on 10853a2, the extended validity oracle fails 27 of 33,092:

  • 21 zone programs, across the caught, destructured, let, for…of, for…in, nested-pattern, and static-block forms.
  • 4 programs for the (eval) spellings, and 2 chain labels beside object.

  GREEN: after the merge of origin/main f8a58a3 (ed779a5), the validity oracle passes 33,092 of 33,092. The static, classification, and probe-form oracles pass 31 of 31, and the unit tests pass 56 of 56. The framework suite passes 133 of 133, and the end-to-end fixed seed 20261105 passes under the lock, in 1,593 s on a loaded machine. On f16fce3 the validity and probe-form oracles pass 33,096 of 33,096, and the fast gap oracles pass 31 of 31.

  Mutants, one per rule, each rejected:

  • no dead zone for a lexical name: 24 failures.
  • a loop head outside the dead zone: 3.
  • a read from another function counted as local: 3.
  • a parameter local at any position: 12.
  • a global name counted as local: 12.
  • eval only as a bare name: 4.
  • a swap label beside any comparison: 2.
  • The with check's mutant passed (0 failures), because the binder already resolves nothing there. f16fce3 deletes the check.

• Round 8: RED: on f16fce3, the extended validity oracle fails 1,177 of 33,395. These are 1,152 first-family bigint labels, 24 programs that call eval through a local binding, and 1 bigint chain label. GREEN: on c846069, the validity oracle passes 33,395 of 33,395. After the merge of origin/main a19c01f (7cd842f), the validity oracle passes 33,395 of 33,395. The static, classification, and probe-form oracles pass 31 of 31, and the unit tests pass 56 of 56. The end-to-end fixed seed 20261105 passes under the lock, in 1,032 s. The framework suite passes 132 of 133. Its determinism test timed out at 600 s under a load average of 26 to 29 on 18 cores. The same test on main a19c01f timed out the same way under the same load, and the branch changes no framework file. It passed 133 of 133 on ed779a5. Mutants, each rejected:

  • an eval through a local binding not direct: 24 failures.
  • a bigint label at any tie: 1,153.
  • a bigint label at a zero literal: 576.
  • a parameter local at any position: 39, 27 of them async or generator programs.
  • no dead zone for a lexical name: 84, 60 of them async or generator programs.

• Round 9 (final): after the merge of origin/main d38af38 (815a999):

  • The validity oracle passes 33,395 of 33,395.
  • The static, classification, and probe-form oracles pass 412 of 412 units, and the unit tests pass 114 of 114.
  • The framework suite passes 299 of 299, its determinism units included.
  • The end-to-end fixed-seed units pass under the lock: 28 of 28, in 487 s.
  • The load average was 25 to 33 on 18 cores.

• Limits: the validity domain is finite: each type's values and a fixed set of foreign ties. The label law covers the first family and the coerced programs, and forbids a label elsewhere. Narrowing is not labelled, so a survivor in code that a narrowing excludes takes another class. The hostile programs are fixed texts. Their types are compared by erasure, as operators and marks, and their dead-code marks are judged on their own inputs. The outside-annotation directive names no test, so the report order of fix(oracle-gaps): name tests in report order, whatever the context count or owner order #30 does not apply to it.

This pull request and its description were written by Isaac.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 815a999

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[coderabbitai]

Important

• Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: d69dec23-7253-457e-8784-d04218b26c03

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/TanStack/oracles/@tanstack/eslint-plugin-oracles@36

npm i https://pkg.pr.new/TanStack/oracles/@tanstack/oracles@36

commit: 815a999