Skip to content

test(cli): judge the closed-read-end case by exit status, not a wall clock - #14863

Merged
os-trump merged 1 commit into
mainfrom
claude/issue-14716-closed-read-end-oracle
Sep 3, 2026
Merged

test(cli): judge the closed-read-end case by exit status, not a wall clock#14863
os-trump merged 1 commit into
mainfrom
claude/issue-14716-closed-read-end-oracle

Conversation

@os-trump

@os-trump os-trump commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Fixes #14716

File face

One file, and it is the one the card fences to me:

  • packages/cli/test/run-dev-unbuilt-workspace.e2e.test.ts (+58 / -12)

packages/cli/bin/run-dev.js is not in this diff. It belongs to #14832, and that card stays open and untouched here. The shim was mutated only inside this throwaway worktree for the ablation below, and restored by blob-hash equality every time (transcript in the report comment on #14716).

What the bound was standing for, and what it actually measured

The assertion was expect(closedEnd.elapsedMs).toBeLessThan(STALL_MS) — 10 s borrowed from case 4's parent stall, which has no relationship to this case. Its stated job was to pin "a CLOSED read end is released at once, not held for the bound (EPIPE reaches the callback)."

Traced, that is not what happens. With the read end destroyed, oclif's displayWarnings() makes the first stderr write, the pipe is already gone, node raises write EPIPE as an error event on process.stderr, nothing is listening, and the child dies of an uncaught exception at ~1.4 s with exit 1. writeStderr() is never called, so the bound the case was named after is never armed, let alone paid.

Two observers, both --import hooks writing only to a file (stderr is destroyed, so it cannot carry a trace):

  • a minimal one with no listener on process.stderr and no write wrapper — 3/3 uncaughtException code=EPIPE, msg=write EPIPE, at afterWriteDispatched (node:internal/stream_base_commons:159:15), then exit code=1, at 1192 / 1269 / 1149 ms;
  • a fuller one that does swallow the error event, which is the only way the run gets far enough to see the drain: the shim's own 415-byte write is write 🔗 Broken links detected in documentation #175, at 9250 ms, behind 174 oclif writes that all EPIPE, and its callback fires 1 ms later. That is 7.8 s after the real, unobserved child is already dead.

So the measured term is child cold start, ~100% of it, and the property the case names contributes ~1 ms — in a run that does not happen.

The bound was a phantom, by ablation

Mutations on bin/run-dev.js, in this worktree only, each proved on disk by blob hash and marker count before any reading, each restored by blob-hash equality under a trap … EXIT INT TERM on absolute paths. elapsed verdict under the old bound in brackets:

bin/run-dev.js state exit elapsed old bound
pristine (12 iterations) 1 1387-1711 ms green
write callback removed, so a closed path could only finish on the bound — the regression this case named 1 1517-1633 ms GREEN
process.stderr.on('error', noop) added, callback kept 2 8787-8979 ms green, 1.2 s to spare
both, so the path really pays the bound 2 23601-23712 ms red

Row 2 is the finding: the assertion could not fail for the reason it named. It moved only on rows 3 and 4, which change the exit code too — and on row 3 its margin was already down to 1.2 s on an ordinary box.

Positive controls, same probe and flags: a drained reader gives exit 2 in ~8.3 s with 145696 bytes delivered; a paused reader gives exit 2 in 8.4 s and, on another iteration, 23.5 s — the second having paid the shim's 15 s bound. So the harness can see a healthy run and can see the bound being paid; the destroyed read end is what is different.

What is asserted instead

The exit status, which is the observation the wall clock was standing in for and carries no load term at all:

  • 1 means the child died on its first stderr write and never reached the drain;
  • 2 means it reached handle(), which is only reachable through writeStderr() — bound paid or not.

Every ablation above that reaches the drain flips it, including the one the old assertion could not see. signal stays asserted, and the elapsed reading is kept as evidence in the failure message rather than as an input to a threshold — the same split #14715 made for case 5.

⚠️ Exit 1 is what the CLI does, not what anyone contracted: a caller whose stderr is closed gets 1 where every other reader gets 2, and cannot tell a failed command from a crashed CLI. That is filed separately as #14858 and is out of scope here — no product change rides along. Pinning today's value is what stops it changing silently; if #14858 is repaired to exit 2, this case reds and that PR flips the number and says why.

⛔ What I did not do, and why: STALL_MS is not raised, no bound is re-derived from a calibration run (that is what #14715 removed), and nothing is skipped, quarantined or retried. Row 2 above is why simply keeping a looser wall clock would have been the worst option — it would have re-hidden a check that never worked.

Readings — all on b90986c7cf, the head of this branch

The target suite, run 8 times (intermittency is the whole subject). 11 passed / 11, every time; durations 44.58, 61.24, 45.38, 44.72, 59.98, 60.67, 58.48, 58.67 s. The last four were joined with && so one exit code certifies all of them: VERDICT command-exit 0.

pnpm exec vitest run --project integration --maxWorkers=2 test/run-dev-unbuilt-workspace.e2e.test.ts
 Test Files  1 passed (1)
      Tests  11 passed (11)

Gate families derived with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (27 for this card, from the script's own changeset derivation, not a hand-written diff). Run locally, each exit code captured before any pipe, quoting each gate's own verdict line:

gate reading
check:nul-bytes OK, 8070 text files, no raw control bytes
check:cli-test-child-env OK, 46 spawners of 116 under packages/cli/test/**, 53 spawns all declare their child env
check:engine-double-contract OK, 759 pinned
check:where-matcher OK, 332 matchers, none new
check:query-options-erasure ratchet holds, none new
check:objectql-double-limit OK, none new
check:cross-package-test-inputs OK, 25 packages read outside themselves, all declared
check:published-files OK, 69 publishable packages
check:type-check-coverage OK
check:test-completeness NOT MEASURED — exit 3, the gate's documented PREREQUISITE NOT MET: it grades a saved turbo run test log and there is none locally. Its own text says this is not a red. CI runs it with a real log.
check:type-check-debt --re-measure NOT MEASURED — needs the whole ./packages/* + ./packages/*/* closure built; I built only the CLI's dependency closure. CI's.

⚠️ One reading worth flagging, because it is the shape that reads as a clearance and is not: packages/cli's own typecheck is include: ["src"], so tsc --noEmit --listFiles for the package returns 0 hits for this file. "Package typecheck clean" therefore says nothing about the edit. Checked directly instead, with node types supplied — tsc --noEmit over the file: exit 0, 0 errors. Positive control that the check is not vacuous: the same invocation without @types/node resolvable reported 23 errors on the same file.

Changeset fork

skip-changeset label, no changeset file. packages/cli's files is ["dist","README.md","CHANGELOG.md"], so neither test/ nor bin/ is published and a test-only diff publishes nothing from any released package. The label is applied additively and read back on this PR (the size-labeler does a whole-set PUT and has been measured stripping a correct additive label about a second later).

🤖 Generated with Claude Code

https://claude.ai/code/session_016yfqQh2dBgPAymYd7xipza


Generated by Claude Code

…clock

`run-dev-unbuilt-workspace.e2e.test.ts` case 6 asserted
`closedEnd.elapsedMs < STALL_MS` — a fixed wall-clock bound borrowed from
case 4's parent stall, against a term that is entirely elastic. Tracing what
produces that timing shows the bound was not merely fragile, it was a phantom.

With the read end destroyed, oclif's `displayWarnings()` makes the first
stderr write, the pipe is already gone, node raises `write EPIPE` as an
`error` event on `process.stderr`, nothing is listening, and the child dies
of an uncaught exception at ~1.4 s with exit 1. `writeStderr()` is never
reached, so the bound the case was named after is never armed. Traced with a
`--import` observer: the shim's own 415-byte write is #175 at 9250 ms, behind
174 oclif writes that all EPIPE — 7.8 s after the real child is already dead.

Ablated on `bin/run-dev.js`, same box, same probe (old bound's verdict in
brackets): pristine exit 1 at 1387-1711 ms [green]; write callback removed so
a closed path could only finish on the bound, which is the regression this
case named, exit 1 at 1517-1633 ms [GREEN]; EPIPE made non-fatal, exit 2 at
8787-8979 ms [green, 1.2 s spare]; both, so the path really pays the bound,
exit 2 at 23601-23712 ms [red]. The measurement moved only where the exit
code moved too, and it stayed green on its own regression.

So the exit status is the observation the wall clock was standing in for, and
it carries no load term: 1 means the child died on its first write, 2 means it
reached `handle()`, which is only reachable through `writeStderr()`. This
asserts that directly and keeps the elapsed reading as evidence in the failure
message, the same split #14715 made for case 5.

Exit 1 is what the CLI does rather than what anyone contracted — filed as
#14858, and pinning today's value is what stops that changing silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yfqQh2dBgPAymYd7xipza
@github-actions github-actions Bot added the size/s label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 2263ca4d679026335f559184c0bed4e76d35a242packageMentionDocs.

@github-actions github-actions Bot added the tests label Sep 3, 2026
@os-trump os-trump added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 3, 2026 — with Claude
@os-trump
os-trump marked this pull request as ready for review September 3, 2026 06:17
@os-trump
os-trump enabled auto-merge September 3, 2026 06:17
@os-trump
os-trump added this pull request to the merge queue Sep 3, 2026
Merged via the queue into main with commit a6d278b Sep 3, 2026
38 of 39 checks passed
@os-trump
os-trump deleted the claude/issue-14716-closed-read-end-oracle branch September 3, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The closed-read-end case in run-dev-unbuilt-workspace.e2e.test.ts still uses a fixed wall-clock bound as its oracle

2 participants