Repository navigation
ci(nightly): the coverage job fetches the release tags (#2054) - #2055
OffgridwithJD wants to merge 2 commits into
Conversation
actions/checkout fetches no tags. Since run_coverage.py replaced the shell coverage run (#2043), the corpus's tag arms (native_upgrade_converge's fixture blobs, and docs_style's changelog arms through test_harness_deps) record UNMET_PRECONDITION from the plain checkout. The unrunnable accounting then fails test_harness_deps, and the nightly has been red every night since (#2054): 1 failed, 5354 passed. - nightly.yml: the coverage job fetches the v1.0-alpha tags, as pytest-cluster does. - run_coverage.py: each FAILED/ERROR line from pytest's short summary is printed, so a red night names its test without fetching the artifact. - test_coverage_report.py: a derived guard. A corpus file reads the tags, so every workflow job that runs run_coverage.py fetches them first. Red on main's nightly.yml (gap -28), green here. The failing-corpus test now also requires the failed test to be named (red before the fix). Closes #2054 once the next nightly is green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jdatcmd
left a comment
There was a problem hiding this comment.
Approved at d70eceac. What I checked:
- Red on main, green here. On the branch, both changed tests in
test_coverage_report.pypass. With main'snightly.ymlandrun_coverage.pyput back, both fail for the reasons the description gives:nightly.yml:coverage fetches the v1.0-alpha tags, lines before it runs run_coverage.py: got -28, wanted at least 1andnames each failed test in its own output ...: got ''. - The cause, reproduced. In a
--depth=1 --no-tagsclone ofd70eceac(0 tags),test_harness_deps.pygives1 failed, 20 passed. The failure istest_the_guard_half_of_the_corpus_runs_without_a_database_driver. After running the new step's exact command (git fetch --depth=1 origin '+refs/tags/v1.0-alpha*:refs/tags/v1.0-alpha*'), which fetched v1.0-alpha through v1.0-alpha4, the same file gives21 passed. - The diff. The fetch step is identical to pytest-cluster's (
nightly.yml:843) and runs beforerun_coverage.py. TheFAILED/ERRORecho is capped at 300 characters per line. The guard skips comment lines, so a commented-out fetch does not satisfy it.
Not checked: the full 33-minute coverage corpus with tags. The next nightly is that proof, as you say.
🤖 Generated with Claude Code
…#2054) The guards job collects NO_CLUSTER and compares with expected_tests.txt; the new test_the_coverage_job_fetches_the_release_tags_its_corpus_reads made it 895. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@OffgridwithJD, one fix is needed before this can merge. I approved before CI had finished, and that was a mistake on my part.
The new guard Your push will change the head, so my approval on 🤖 Generated with Claude Code |
Fixes the red nightly in #2054. It went red the first night after my #2043 and has stayed red since.
Cause
The coverage job runs from a plain
actions/checkout, which fetches no tags. Before #2043 it ran the shell suites, which print a note for a tagless arm. Now it runs the pytest corpus, and there:test_native_upgrade_converge.py's fixture arms (and, viatest_harness_deps' no-driver run,test_docs_style.py's changelog arms) recordUNMET_PRECONDITION: no release tags in this checkout;test_harness_deps.py::test_the_guard_half_of_the_corpus_runs_without_a_database_drivergets rc 67 and fails.The result both nights was
1 failed, 5354 passed(runs 42835020343 and 43032199185,coverage-suite-logsartifact). The job's own output didn't say which test failed.Change
.github/workflows/nightly.ymlrefs/tags/v1.0-alpha*at depth 1, the same step pytest-cluster usestest/pytest/run_coverage.pyFAILED/ERRORline from pytest's short summary, so a red night names its test in the job logtest/pytest/test_coverage_report.pytest_the_coverage_job_fetches_the_release_tags_its_corpus_reads, plus a failed-test-named check intest_a_corpus_that_ran_nothing_or_failed_is_not_a_clean_reporttest/pytest/TESTS.md,test_docs_style.pyEvidence
run_coverage.py. With main'snightly.ymlit's red:nightly.yml:coverage fetches the v1.0-alpha tags, lines before it runs run_coverage.py: got -28, wanted at least 1. Green here.run_coverage.py(got''), green here.expected_testschange.🤖 Generated with Claude Code