Repository navigation
fix(test-impact): let a build drop the suites only some runners register - #2684
Merged
Merged
Conversation
…ONAL on every selection path A runner suite registered only under a preprocessor condition (the four coverage_* suites under `#if defined(CBM_TEST_COVERAGE) && !defined(_WIN32)` in tests/test_main.c) is absent from builds without that condition. The selection marked such a suite CONDITIONAL only on the per-registration path, and only when a registration of its own was conditional and contributed a reason. Every suite-level path (setup unknown or hit, suite-body, changed and rule triggers, inventory, rejected artifact) made the suite whole without it, and the runner registration's own condition was never recorded. PR CI on the x86 Linux gcc leg then failed with "the test selection names suites this build does not register: coverage_setup_success coverage_setup_uncertain" (#2679, #2680): the suite-body trigger and an incomplete setup row selected both suites whole without CONDITIONAL, so selection.py left them out of optional-suites.txt. The model now records whether a runner registration sits under a condition (conditional only when every site of the suite is), and the selection marks any selected conditionally registered suite whole with CONDITIONAL after every whole-suite path. Unconditionally registered suites keep the strict contract. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… map
A coverage map must cover every function it lists, and five of the seven
variants covered only function 1, so the map failed to parse and the test
stopped at its setup ("map is NULL") with the fix applied or not. The
changed function (0) is now covered by the setup row of a suite no runner
registers, so alpha's own rows stay quiet and each variant still reaches
alpha by exactly one path. RED with the selection fix reverted (alpha
lacks CONDITIONAL), green with it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added a commit
that referenced
this pull request
Oct 18, 2026
…lection fix #2684) Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added a commit
that referenced
this pull request
Oct 18, 2026
…art CI selection fix #2684) Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added a commit
that referenced
this pull request
Oct 18, 2026
…Smart CI selection fix #2684) Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added a commit
that referenced
this pull request
Oct 18, 2026
… up the Smart CI selection fix #2684) Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added a commit
that referenced
this pull request
Oct 18, 2026
…Smart CI selection fix #2684) Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added a commit
that referenced
this pull request
Oct 18, 2026
…the Smart CI selection fix #2684) Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
timothybrush
pushed a commit
to timothybrush/codebase-memory-mcp
that referenced
this pull request
Oct 19, 2026
…ed-class (pick up ef90708: origin/main incl. the Smart CI selection fix DeusData#2684) Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Smart CI: a build may drop the suites only some runners register
Since
TEST_IMPACT_MODE=gate(2026-10-17 22:10Z), a PR whose selection gets narrowed fails its x86 Linux test leg before any test runs:Seen on #2679 and #2680. Only that leg consumes a selection; every other leg runs every test, so their results stand.
Cause
The selection comes from the coverage runner's map.
tests/test_main.cregisters four suites only under#if defined(CBM_TEST_COVERAGE) && !defined(_WIN32).scripts/run-tests-parallel.shtolerates a selected suite the build lacks only whenoptional-suites.txtnames it, andselection.pylists a suite there only when the answer marks it whole withCONDITIONAL. The model recorded a condition around aRUN_TESTregistration, but not around the runner'sRUN_SELECTED_SUITE, so these suites reached the leg as required. A real answer for #2679's head showed it: of the four, onlycoverage_children(whose own registrations are conditional) carriedCONDITIONAL.Fix
cbm_test_runner_suite_t.conditional: every site that registers the suite sits under a preprocessor condition. A suite also registered outside any condition runs in every build and keeps the strict contract.CONDITIONAL.selection.pythen lists it as optional, and a build without it drops it with a notice, exactly as its full run never runs it. The leg's guard is unchanged: a selected suite that no condition explains still fails the run.Verification
.cchanges reverted,test_selection_conditionally_registered_suite_is_conditional_on_every_path(seven selection paths, plus an unconditional control) andtest_model_marks_runner_suites_registered_under_a_conditionfail on their assertions; with them,test_impactpasses 89/89.354ccecbmarks all four coverage suitesCONDITIONAL(before: one), andselection.py'sconditional_suiteslists all four as optional (before:coverage_childrenonly).scripts/test.sh --contracts-only) and the security audit are clean.After the merge
A rerun of an affected PR replays its frozen merge commit, and
select-testsbuilds its engine from that checkout, so it would rerun the old engine. Affected PRs need a new merge commit: merge main into the branch, or push again.The diagnosis and the fix come from the graph-quality session, which found it independently; this PR carries their commit unchanged, plus a fixture correction (the selection test's coverage map did not parse).