Skip to content

test(compiler): cover every render input in the compiler differential - #1001

Merged
kvlonge merged 4 commits into
mainfrom
test/differential-render-coverage
Oct 7, 2026
Merged

kvlonge merged 4 commits into
mainfrom
test/differential-render-coverage

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Why

Before M3 ports scopes, rendering and the macro bridge to Rust, the compiler differential must cover every render input and render-time error code and compare CompileProjectInputs, as #994 did for discovery.

Changes

  • Generator: nine render blocks for macros reading context, constants, enums and the environment; nested and typed-reference macro calls; macro-generated references with enforce_explicit = false; table functions; @@var, @@ENV: and @@CTX: interpolation; runtime placeholders and cursor intrinsics; macros in tests, scenarios, audits, functions, hooks and source expressions; enum-typed and model-private declarations; named hook arguments; and model, source and seed audits.
  • Render coverage classifier over the CompileProjectInputs capture, with --require-render-coverage, now required in the per-PR job. A kind counts only with rendered evidence. Encoded empties count for nothing. Macro context reads are named as indirect. No kind is config-only.
  • Failure corpus: one case per reachable render, scope and attachment code, with P001 cases checked by message. Unreachable codes are listed with reasons; the code set comes from compiler constants and an AST scan of inline codes.
  • E2E tests: the capture holds every field canonically and proves all 63 kinds, and a perturbed native render is localised to compile_project_inputs.

Verification

  • Unit and E2E harness tests, make compiler-differential with the new flag (139 projects identical, 63/63 render kinds, also native vs native), make check-ci, make test, hygiene.
  • Per-PR differential, same machine, 3 alternating runs: CPU median 294 s → 341 s (+16%). Wall times unreliable (load above core count).
  • CHI-539: CompileProjectInputs captures match python-vs-python on 46 projects, repeated seeds and dense-1000. The CompiledProject mismatch comes from a real scheduling-order nondeterminism in the shared-analysis memo, so it is reported in CHI-539 and not canonicalised.

Add rendering feature blocks to the seeded generator: macros reading
context, constants, enums and the environment, nested and typed-reference
macro calls, macro-generated references with relaxed explicit-reference
rules, table functions, variable, environment and hook-context
interpolation, runtime placeholders, cursor intrinsics, macros in tests,
scenarios, audits, functions and source expressions, enum-typed columns,
model-private declarations, named hook arguments, and model, source and
seed audits. The harness provides the fixed environment variable the
generated projects read.
Classify CompileProjectInputs stage captures into render input kinds,
crediting a kind only from rendered evidence: a template absent from the
rendered SQL, a recorded usage, reference, grant, hook entry or attached
audit. Encoded empty sets and mappings prove nothing. Kinds credited from
a used macro's source text are reported as indirect.

Add --require-render-coverage and require it in the per-PR compiler
differential job. Share the capture readers with discovery coverage.
Add one failing project per reachable render, scope and attachment code
(P005, P007, P008, P009, S006, S007, S008, S009, K012, C214 and the C216
warning) and P001 cases for macro, interpolation, reference and hook
argument errors, each checked against its first error message.
render_error_codes derives the code set from the compiler; the end-to-end
test requires every code to be reached unless it is listed as
unreachable with a reason. Share the base-project case builder across
the corpus modules.
Plan a generated project with every feature block and require its
CompileProjectInputs capture to hold every field canonically, name every
callable, and prove every required render kind. Perturb the native
compile inputs to show the differential localises a render mismatch to
the compile_project_inputs capture.
@kvlonge
kvlonge enabled auto-merge (squash) October 7, 2026 09:50
@kvlonge
kvlonge merged commit 1d70758 into main Oct 7, 2026
64 of 66 checks passed
@kvlonge
kvlonge deleted the test/differential-render-coverage branch October 7, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant