Skip to content

Speed up the coverage job (about 25 minutes since coverage became real) #898

Open

Description

@petercorke

Since #897 made coverage real, the coverage job in ci.yml takes about 25 minutes (1,503 s on the run for main), where the other jobs take 10 to 15. It runs on every pull request and every push to main, since the job has no if:. It is not a required check, so it never blocks a merge, but it is the last job to finish on every run, and the one that tells the Codecov number.

Why it is slow

It runs the whole suite with coverage tracing, including the tests marked slow (the analytic IK tests that compile generated C and C++ code, about 110 s untraced), and it runs them under the default tracing core.

Ideas, cheapest first

  • COVERAGE_CORE=sysmon in the coverage step. Python 3.12 has a lighter tracing core. Try it on the job, and compare the totals with the current 85.98%, since a different core can change which lines count.
  • Do not trace the slow tests: run them in the matrix job on Linux with no coverage, and take coverage from the fast tests only. Check what the analytic IK tests contribute (they exercise about 4,000 lines).
  • Split the job with pytest-xdist and combine the data (coverage combine), as the tests are independent.
  • Run coverage on pushes to main and weekly, not on every PR. The PR would lose its Codecov patch status, which is informational now anyway.
  • Cache what is rebuilt on every run: the package install, robot_descriptions assets (already cached).

Measure first

The job log has the timing for each step. Note the compile-based tests in particular: on the Windows runner they were slow enough to hit the 50 s limit, so a Linux job that traces them may be paying the same cost.

Related: #894 (code-quality follow-ups), petercorke/rvc-ecosystem#26 (the CI audit that found the coverage problem).

Activity

  1. added
    tech-debtKnown technical debt / deferred cleanup, not a live bug
    on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    tech-debtKnown technical debt / deferred cleanup, not a live bug

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions