Repository navigation
Let CI test and check jobs use another runner pool, set by repository variables - #2598
Conversation
The test suite, Changes, Tested and the Lint, install and Docs checks now read their runner from CI_RUNNER_ARM or CI_RUNNER_X64, optionally limited to the events in CI_RUNNER_EVENTS, with today's GitHub-hosted labels as the defaults. Nothing changes until a variable is set. Release versioning, build and publish, fix evidence and aggregate impact keep GitHub-hosted runners. test_ci_runner_variables.py holds the workflows to this. Fixes #2597 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The organisation's default GITHUB_TOKEN can write, and Changes, the install check, Lint and Docs had no permissions block, so a moved job would take a write token to the other runner. They now read only; none of them uses the token beyond checkout. test_ci_runner_variables.py checks every job that can move holds no write scope apart from id-token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Spending by benefit and service
Caseloads
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileNo household's net income changes in any year. No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles. Checks failing on both base and head (1)
Counted-once checkEach of 84 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped. Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
The release workflow runs the test suite on the installed wheel, which has no .github folder, so the test could not find the workflows and failed, which would block Publish. It now skips at module level there, with the reason the test-outcome allowlist accepts for code_health tests in wheel mode, as test_aggregate_impact_workflow.py does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Queued for merge on Max's go ("Merge policyengine-uk#2598 and policyengine.py#706 on their normal gates", 2026-10-24). Handing it to the UK hub for merge order.
Nothing changes until someone sets |
Fixes #2597
Draft: do not merge before Max decides on CI runner capacity (a cos decision; analysis in
~/reviews/ci-runner-capacity-2026-10/REPORT.mdon the hub host). Companion: PolicyEngine/policyengine.py#706.What and why
The org's 60-job limit for standard GitHub-hosted runners is shared by every PolicyEngine repository. This repository's merge queue groups, main pushes and release test suites queue for runners behind other repositories' pull request runs. This PR lets those jobs move to another runner pool, such as GitHub-hosted larger runners, by setting a repository variable, and move back by deleting it. Nothing changes until a variable is set.
CI_RUNNER_ARMubuntu-24.04-armYAML,Python(5 shards),Result; pr_code_changes.yamlChanges; code_changes.yamlTestedCI_RUNNER_X64ubuntu-latestPython-Compat,Lint,Docs; code_changes.yamlLint; versioning.yamlLint,Compat,SdistCI_RUNNER_EVENTSmerge_group,push. Only those events use the two variables. Empty means every event.The test suite is a reusable workflow, so the release's
Release testfollowsCI_RUNNER_ARMwhenpushis listed.Never moved (the test pins these): versioning.yaml
Versioning(version commit, App token),Build(the files that are published),Publish(PyPI trusted publishing); pr_fix_evidence.yaml (runs onpull_request_targetwith base-repository secrets); aggregate-impact.yaml (household-level detail must stay on the runner); deploy.yml.Read-only tokens. The org's default
GITHUB_TOKENpermission is write (gh api orgs/PolicyEngine/actions/permissions/workflow), andChanges,Python-Compat,LintandDocs(pr_code_changes.yaml) and main'sLint(code_changes.yaml) had nopermissionsblock, so a moved job would take a write token to the other runner. They now havepermissions: contents: read. None of them uses the token beyond checkout. The test suite and versioning.yaml were already read-only at the workflow level.Data. The moved test jobs download the licensed dataset with
HUGGING_FACE_TOKENand authenticate to GCP through workload identity federation. The workflow comment says to point the variables only at runners where that data may be processed, such as the organisation's GitHub-hosted larger runners. A third-party runner provider would need a data-licence decision first. GitHub-hosted larger runners reportrunner_environment: github-hostedin the OIDC token, as standard runners do (GitHub OIDC reference).Each job's runner is
An unset variable reads as
'', so the default label is used. The commas around both sides makepull_requestnot matchpull_request_target.To switch, for example only the merge queue and main to a larger runner named
pe-arm64-4core: Settings > Secrets and variables > Actions > Variables:CI_RUNNER_ARM=pe-arm64-4core,CI_RUNNER_EVENTS=merge_group,push,workflow_dispatch. To switch back: deleteCI_RUNNER_ARM. The next run usesubuntu-24.04-arm.Invariants
CI_RUNNER_EVENTSis empty or names the run's event as a whole name.GITHUB_TOKEN(id-token, used for GCP, aside).Test plan
policyengine_uk/tests/code_health/test_ci_runner_variables.py: invariants 1, 2 and 4 against the parsed workflows; invariant 3 exhaustively over 2 variable states × 6 event lists × 6 events (72 cases) on a Python mirror of the expression. 75 passed locally. Mutation check: makingPublishread a variable fails invariant 2, and removingChanges' read-only token fails invariant 4..github. The test skips itself at module level with the reason "the .github folder is not part of an installed package", astest_aggregate_impact_workflow.pydoes;.github/test_outcome_allowlist.tomlaccepts thatcollect-skippedoutcome forcode_health/*in wheel mode. Checked in a wheel-like layout: 1 skipped. Do not merge until "Release test / Result" is green on this PR.actionlinton every workflow: no findings.ruff format --checkandruff checkon the new test.changelog.d/2597.changed.md.Reproducer: n/a: CI workflows and a code-health test only; no files under policyengine_uk/ outside policyengine_uk/tests/ change.
axiom: n/a: CI infrastructure only
🤖 Generated with Claude Code