Skip to content

ci: keep compiler differential failure evidence - #1069

Merged
kvlonge merged 1 commit into
mainfrom
ci/keep-differential-failure-evidence
Oct 9, 2026
Merged

kvlonge merged 1 commit into
mainfrom
ci/keep-differential-failure-evidence

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Why

The compiler differential intermittently reports a warm-compile stderr difference that we cannot reproduce locally or in diagnostic CI runs. The next natural failure should carry its own evidence instead of one differing line.

Changes

  • --evidence-dir PATH: when a project differs, copy each engine's target/ (including the SQL-test artifact cache), every command's stderr, and a compiled-tests-stat.tsv listing size, mtime_ns and ctime_ns of every compiled test file.
  • The differential job passes it to both differential steps and uploads the directory as an artifact on failure only, following the existing upload step.
  • No comparison logic changes: evidence is written after differences are computed and never affects the result.

Verification

  • New parametrised harness test: a differing project keeps both engines' evidence (target files, stderr, stat listing); identical runs keep nothing.
  • Differential script unit tests (415 passed); a real --corpus examples --evidence-dir run passes and writes nothing.
  • make check-ci; hygiene scan.
  • No test expectations or thresholds changed.

@kvlonge
kvlonge enabled auto-merge October 9, 2026 14:27
@kvlonge
kvlonge added this pull request to the merge queue Oct 9, 2026
Merged via the queue into main with commit 12fb5f7 Oct 9, 2026
69 of 72 checks passed
@kvlonge
kvlonge deleted the ci/keep-differential-failure-evidence branch October 9, 2026 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant