Skip to content

feat: run only affected tests and scenarios with --since - #1228

Merged
kvlonge merged 11 commits into
mainfrom
feat/test-since
Oct 23, 2026
Merged

kvlonge merged 11 commits into
mainfrom
feat/test-since

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Why

CI on branch pushes runs every unit test and scenario, even when a change can only affect a few. --since <ref> runs only what the change can affect. When in doubt it runs more, never less.

Changes

  • sqb test --since <ref> and sqb scenario test --since <ref>. The base is compiled from git objects at the merge base, in a separate interpreter. It is cached under target/since/ (newest few kept) and shared by both commands. @base reads [git] base_ref.
  • Each node gets a definition fingerprint, so formatting and description edits select nothing. The fingerprint covers:
    • model SQL, resolved config, columns, audits and meta
    • @@var and @@ENV: values
    • macros with the macros and project modules they import
    • enums, constants, audits, seeds, sources, functions and Python nodes
  • A test or scenario is selected when one of these changed:
    • its own file
    • anything it executes, including every case of a parameterized test
    • the declared columns of a relation it mocks
    • a project-wide input: settings, connection and targets, adapter modules, or Python lock files
  • migrate_from renames compare with the old definition. Deleted nodes are listed and select nothing.
  • --explain and --inspect show the reasons. --json adds a since section.
  • Clear errors (C480–C485) with no tracebacks. A target missing at the base falls back to the default target.
  • Docs, including known limits, and the docs skill are updated.

Verification

  • E2E test_since.py has 39 cases, including:
    • variables, defaults, macro imports, parameterized cases, mocked column shapes, connection settings, and tracked vs untracked lock files
    • scenarios, JSON output, renames, shallow clones, a broken base, and a target missing at the base
  • Rust tests cover change detection, import closures and dependency files.
  • Existing test, scenario and custom-rule tests pass.
  • make check-ci passes.

@kvlonge
kvlonge enabled auto-merge (squash) October 23, 2026 01:48
@kvlonge
kvlonge merged commit 720539c into main Oct 23, 2026
46 checks passed
@kvlonge
kvlonge deleted the feat/test-since branch October 23, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant