Skip to content

ci: migrate 3 maturin workflows to needs.setup.outputs.matrix (maturin batch 2/3) - #4921

Open
luhenry wants to merge 1 commit into
mainfrom
fix/matrix-maturin-2
Open

luhenry wants to merge 1 commit into
mainfrom
fix/matrix-maturin-2

Conversation

@luhenry

@luhenry luhenry commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Part of the repo-wide migration of build-<pkg>.yml workflows off their own hardcoded interpreter lists onto needs.setup.outputs.matrix (_setup.yml's generated include:-ready matrix, derived from each pending version's docs/packages/<pkg>.yaml files: entries). This batch owns 3 maturin/PyO3 (Rust) based files; see skills/python-project-porting/ for the general playbook these builds otherwise follow.

Converted

build-demoparser2.yml

  • build_wheels.strategy.matrix now has include: ${{ fromJSON(needs.setup.outputs.matrix) }} as its sole key, replacing version: + python: ["cp312","cp313","cp314","cp314t"] + the include: entry that set test_skip: "*" for cp314t.
  • Field mapping: matrix.python for CIBW_BUILD's interpreter component and both artifact names; matrix.platform for CIBW_BUILD's libc component and the artifact names (always resolves to manylinux, matching the old hardcoded "manylinux_riscv64" literal — docs/packages/demoparser2.yaml declares no musllinux target for any version).
  • CIBW_TEST_SKIP: ${{ matrix.test_skip }} is removed outright: with cp314t gone from the matrix (see mismatch below) there is no leg left that ever sets it.
  • Added python-exclude: 'cp39 cp310 cp311' to the setup job. docs/packages/demoparser2.yaml declares 3.10/3.11 file targets (riscv64-buildable, not excluded by _setup.yml's own cp39 default), but this workflow's hardcoded list never built them — only cp312+. The added exclusions keep that scope.
  • Mismatch found (not fixable from this workflow alone, flagging rather than guessing further): cp314t was in the old hardcoded list, but docs/packages/demoparser2.yaml declares no 3.14t file target for any version, so needs.setup.outputs.matrix can never produce a cp314t entry — converting drops that leg. Checking ci_scripts/update_doc.py's publish-time validation (merge_publication_files), a cp314t wheel was already guaranteed to fail publish before this change too: its parsed target (3.14t:manylinux) isn't in the version's declared files: list either, so update_doc.py --validate-only raises built wheels have undeclared python targets: ... regardless of the build_wheels matrix shape. So the cp314t leg was pre-existing dead weight on the publish side; this conversion just stops building it. If cp314t support is still wanted, docs/packages/demoparser2.yaml needs a 3.14t:manylinux files entry added first — out of scope here (workflow-only PR).
  • Verified with uv run ci_scripts/pending_versions.py --python-matrix --python-exclude 'cp39 cp310 cp311' demoparser2: produces exactly cp312/cp313/cp314 × manylinux for both pending versions (0.42.0, 0.42.1), matching the old hardcoded scope minus the dead cp314t leg.

build-ormar-utils.yml

  • build_wheels.strategy.matrix now has include: ${{ fromJSON(needs.setup.outputs.matrix) }} as its sole key, replacing version: + python: ["cp312"] + libc: [manylinux, musllinux].
  • Field mapping: matrix.python for CIBW_BUILD's interpreter component and both artifact names; matrix.platform in place of the old matrix.libc everywhere (same manylinux/musllinux values, renamed to match the generated matrix's field).
  • Added python-exclude: 'cp39 cp310 cp311' to the setup job. ormar-utils' Cargo.toml carries pyo3's abi3 floor at py310, so the un-excluded generated matrix would pick cp310 (the oldest riscv64-buildable interpreter at or above that floor) as the build interpreter; this workflow's own comment established cp312 as the real floor instead. The added exclusions make the generated matrix pick cp312 the same way.
  • cp314t requires no exclusion: docs/packages/ormar-utils.yaml declares no 3.14-abi3/3.14t target at all (only 3.10-abi3), so needs.setup.outputs.matrix never produces that leg — same situation as demoparser2, not a new issue introduced here.
  • Verified with uv run ci_scripts/pending_versions.py --python-matrix --python-exclude 'cp39 cp310 cp311' --glob '0.2.1' ormar-utils: produces cp312 × {manylinux, musllinux} with wheel-tag: cp310-abi3, matching the old hardcoded python: ["cp312"] / libc: [manylinux, musllinux] exactly (no currently-pending version to verify against directly — ormar-utils has none right now — so --glob was used to force the one declared version through the matrix generator).

Skipped

build-grpcio.yml — left unconverted, per the task's own flag to be careful here rather than guess.

build_wheels.strategy.matrix today has no platform/libc dimension at all: CIBW_BUILD: ${{ matrix.python }}-manylinux_riscv64 hardcodes manylinux, and there's no CIBW_MUSLLINUX_RISCV64_IMAGE, no musllinux-aware CIBW_CONTAINER_ENGINE/ccache wiring, etc. But docs/packages/grpcio.yaml declares a musllinux target for essentially every python entry of every version (pending, no filename ever recorded). Converting straight to include: ${{ fromJSON(needs.setup.outputs.matrix) }} would silently double the matrix and start attempting real musllinux builds of a large C++ project (480-minute timeout already, vendored boringssl/abseil/re2, apk-vs-yum CIBW_BEFORE_ALL, ccache paths) that this workflow was never resourced or tested for — a scope change well beyond "source the interpreter axis from the generated matrix," and not something I can verify safely without actually running the musllinux leg.

The version-exclude logic (1.65.5 has no cp313/cp314) converts cleanly on its own — docs/packages/grpcio.yaml's 1.65.5 entry simply has no 3.13/3.14 file targets, so the generated matrix naturally omits them for that version already, no explicit exclude: needed. It's only the manylinux-only restriction that doesn't have a clean mapping onto the generated matrix (which carries both platforms) without either filtering the matrix in a bespoke way (no precedent for that elsewhere in the repo) or adding a musllinux build path, neither of which I want to force through here.

Hard rules followed

  • Worked in an isolated worktree (fix/matrix-maturin-2 branch off main), no force-push, no merge.
  • Pure .github/workflows/* changes only — docs/packages/*.yaml, .queue.yml, skills/ untouched.
  • Commit authored as Ludovic Henry <git@ludovic.dev>, no AI/Claude attribution anywhere.

Both build_wheels jobs now source their interpreter/platform axis from
the shared setup job's generated matrix instead of a hardcoded python
(and libc) list.

demoparser2: drops the explicit python list and the cp314t test_skip
include; adds python-exclude: 'cp39 cp310 cp311' to the setup job so
the generated matrix keeps matching this workflow's own cp312+ scope.
docs/packages/demoparser2.yaml declares no 3.14t file target, so the
generated matrix never produces a cp314t leg either (and the
CIBW_TEST_SKIP workaround that leg alone needed goes with it).

ormar-utils: drops the explicit python/libc lists; adds
python-exclude: 'cp39 cp310 cp311' to the setup job so the abi3-floor
selection still lands on cp312 (this workflow's own established
floor) rather than the cp310 the generated matrix would otherwise
pick from pyo3's unconditional abi3-py310 floor. cp314t stays absent
the same way as demoparser2: no 3.14-abi3/3.14t target is declared.

grpcio is left unconverted; see the PR description.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant