Skip to content

Register and check the uv.lock rules: use, install, test and relock from uv.lock - #3997

Draft
anth-volk wants to merge 12 commits into
auto-relock-dependencies-workflowfrom
uv-lock-rules
Draft

anth-volk wants to merge 12 commits into
auto-relock-dependencies-workflowfrom
uv-lock-rules

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #3996

Third pull request in a stack: #3995 (tests install from uv.lock) → #3994 (daily relock workflow) → this one.

What this adds

Four rules in rules/registry.toml, all mechanical and enforced, checked by a new locked-installs quality guard (scripts/rules/locked_installs.py, standard library only) in the pull request quality-guards job, each with bad and good proof fixtures under scripts/rules/tests/fixtures/locked_installs/:

Rule Checks
api.dependencies.lock-committed Every pyproject.toml has a committed uv.lock beside it, and a pull request job runs uv lock --check for it. The quality-guards job now installs uv 0.13.4 and runs uv lock --check.
api.dependencies.install-from-lock Every install in workflow run: blocks, tracked shell scripts, Makefile recipes, Dockerfiles and Modal image definitions uses uv sync --frozen/uv run --frozen (or --locked), unless it is a registered exception.
api.dependencies.test-from-lock Every pytest, coverage run, unittest, tox or nox command in those files runs through uv run --frozen.
api.dependencies.scheduled-relock A workflow on a schedule trigger relocks every uv.lock with upgrades and runs gh pr create/gh pr edit (static check of relock-dependencies.yml and the scripts it runs).

[[rule.exception]] can now name one site (match), and end on a removal condition, a review date (expires) with a tracking issue, or be a pinned-tool (the guard checks every package in the command is pinned with ==). An exception without match, without a removal condition or tracked review date, or that matches nothing, fails the guard. test_every_install_source_is_scanned checks, from git ls-files, that every workflow, shell script, Makefile, Dockerfile and Modal image module is scanned.

docs/engineering/skills/dependencies.md explains the four rules and how to add an exception. tests/unit/test_locked_test_environments.py (from #3995) now uses the guard's job tracing and keeps only the checks the guard does not make (Make targets, the Python version per test job, the operational jobs' explicit install).

Install sites found

Site Classification
Makefile (install, test, test-env-vars, debug-test, debug, debug-fastapi, typecheck, changelog), pr.yml, push.yml, alembic-v1-check.yml, alembic-v2-check.yml, v2-integration-check.yml, build_changelog.sh, run_live_staging_tests.sh, test_alembic_v2_lifecycle.sh, update-policyengine-package.sh, request-simulation-model-versions.sh, gcp/cloud_run/Dockerfile Locked
seed-v2-database.yml uv run python scripts/... (6 steps), migrate_v2_metadata_schema.sh uv run alembic (3) Moved to the lock: now uv run --frozen, after the job's uv sync --frozen. With a consistent lock this installs the same versions.
pr.yml quality-guards python -m unittest discover -s scripts/rules/tests Moved to the lock: now uv run --frozen --extra dev, with UV_PYTHON: "3.12" on the job
pr.yml, push.yml pip install "shellcheck-py==0.11.0.1" Pinned tool
install_locked_ruff.sh pip install "ruff==${version}" (version read from uv.lock) Pinned tool
migrate-v1-cloud-sql.yml, process-v1-household-mirror-event.yml pip install -e ".[dev]" Exception: scripts run with the bare python against the production and staging databases. Removal: the job uses uv sync --frozen/uv run --frozen after a staging run succeeds.
update-policyengine-bundle.yaml pip install requests uv Exception: the updater changes uv.lock, so it runs before a matching lock exists. Removal: install uv with astral-sh/setup-uv at a pinned version and run its Python through uv run --frozen.
verify_observability_collector.sh uv run --no-project --with grpcio --with opentelemetry-proto Exception: the deploy job does not install the project. Removal: the job syncs with uv sync --frozen and the check runs through uv run --frozen.
docker/Dockerfile pip install "policyengine[models]==6.7.5" ipython Exception: the image packages the .py bundle, not this project; ipython and the bundle's dependencies are unpinned. Removal: the image installs from uv.lock, or is removed. The match leaves out the version so bundle updates don't make it stale.

No Modal image definitions exist in this repository; policyengine_api/libs/simulation_api_modal.py is a client, and the scan of all non-test Python modules found no image install calls.

Red and green

Commit Red Green
Scanner (rule 2) python -m unittest scripts.rules.tests.test_locked_installs: import of scripts.rules.locked_installs fails 9 tests pass
Test commands (rule 3) Import of unlocked_test_commands/test_command_violations fails 11 pass
Lock committed (rule 1) Import of lock_violations fails 12 pass; then the guard reported no pull request job runs uv lock --check for . until the quality-guards step was added
Scheduled relock (rule 4) Import of relock_violations fails 14 pass
Registration test_repository_has_no_violations fails: rules/registry.toml has no rule ... 15 pass; run_quality_guards.py: locked-installs: passed
Fold of the lock test Refactor: same assertions tests/unit/test_locked_test_environments.py 16 pass

After the self-review fixes below, test_locked_installs runs 16 tests, all passing.

Checks run locally

  • ruff format --check ., ruff check . (locked ruff): pass.
  • python scripts/run_quality_guards.py: all guards pass, including locked-installs.
  • python -m scripts.rules guard --base-ref origin/auto-relock-dependencies-workflow: pass; python -m scripts.rules render: no changes.
  • uv lock --check: pass.
  • python -m unittest discover -s scripts/rules/tests -t . (in the locked environment): pass.
  • pytest tests/unit/v2/test_metadata_deployment.py tests/unit/test_locked_test_environments.py tests/unit/test_alembic_workflows.py tests/unit/test_relock_dependencies_workflow.py: pass.
  • # noqa count: 98 before and after. No new suppressions; the guard's import joins an existing suppressed import in run_quality_guards.py.

Environment variables

  • pr.yml quality-guards: UV_PYTHON is set in the job's env:. uv sync uses public PyPI (no [tool.uv.index]), so it needs no credentials. The guards read GITHUB_ACTIONS and GITHUB_BASE_REF, which the runner sets.
  • seed-v2-database.yml (only --frozen added): V2_MIGRATION_DATABASE_URL is a secret in the staging and production GitHub environments, and reaches each step through its env:. The job selects the environment with environment: ${{ inputs.deployment_environment }}, and push.yml calls the workflow with secrets: inherit. V2_SUPABASE_PROJECT_REF and V2_SUPABASE_ENVIRONMENT are variables in both environments, and PRODUCTION_V2_SUPABASE_PROJECT_REF is a repository variable; all three reach the job through its env:. migrate_v2_metadata_schema.sh reads V2_MIGRATION_DATABASE_URL from its step's env:.
  • seed-v2-database.yml runs only from push.yml on the default branch, or by manual dispatch, so pull request CI does not run it. Its --frozen change has not run.

Self-review fixes

A self-review of this branch found three defects, each fixed test first:

Finding Red Green
Composite actions were counted as scanned, but their runs: steps: were never read test_rule_proof_install_from_lock fails: the new composite action fixture's pip install requests and uv sync --extra dev are not reported Composite actions are read by their own run: reader; the test passes, and the coverage test includes action.yml
The relock check accepted an upgrade flag from any command in the job test_upgrade_flag_must_belong_to_the_relock fails: a plain uv lock next to pip install -U pip passes The flag must be on the uv lock command or in an array it expands ("${upgrade_arguments[@]}"); the test passes, and the repository's relock workflow still passes
uv run --frozen --no-sync pytest was reported as outside the lock test_rule_proof_test_from_lock fails on the good fixture's --no-sync run A uv run test command is locked when it has --frozen/--locked and no --with/--no-project; the test passes

Also fixed: the rules-registry skill's field table now lists the new exception fields, and the test-from-lock rule's scope includes Dockerfiles.

Not covered

  • The seed workflow and the API v2 type-check job still use the Python that .python-version selects (3.14), which is outside these rules.
  • The scan reads workflow run: blocks, not third-party actions' own installs.
  • A relock or lock check that loops over directories held in a shell variable is not resolved statically. There is one project here, so this doesn't apply yet.

Rebase onto master (2026-10-22)

Rebased onto the rebased #3994, now on master at ee46c7aca.

  • Conflicts in scripts/rules/load.py and scripts/rules/validate.py: Refactor tests, scripts and the rules tool, then select every ruff rule (ALL) #3934 split the registry loader and validator into one helper per field. Resolved by keeping Refactor tests, scripts and the rules tool, then select every ruff rule (ALL) #3934's structure. Its _exceptions helper now calls this PR's _exception parser (moved next to it, and accepting a missing rule id as the other helpers do), which reads match, removal, issue and category. _validate_expiry skips the expiry check for an exception without an expires date.
  • Refactor tests, scripts and the rules tool, then select every ruff rule (ALL) #3934's loader test: test_every_problem_is_reported_in_order expected the unknown-key message to list only expires, path and reason; it now lists all seven exception keys.
  • uv version: the quality-guards job pinned uv 0.12.1 as "the same uv version as the Cloud Run image and the release job", and Stop trusting the compromised eMudhra CA certificates #3999 raised both to 0.13.4. It now pins 0.13.4, and a new test, test_lock_check_uses_the_uv_version_that_locks_releases, checks that the two jobs' uv versions match. The test failed against 0.12.1 first.
  • Ruff: master's full rule set found two PLR0911 findings (too many return statements) in scripts/rules/locked_installs.py: _file_kind had 7 and classify_command had 8, where 6 are allowed. "Keep the locked-installs scanner within ruff's return limit (PLR0911)" fixes both without changing results: _file_kind picks from an ordered table of name tests, and the uv run branch moves into _classify_uv_run. It follows "Pin how each kind of install command is classified", which tests two results the fixtures never reached. No suppressions were added; the noqa count is unchanged at 96.
  • Checked locally: ruff format and lint, the quality guards (including locked-installs), the rules guard, uv lock --check, the rendered rules (regenerating them changed nothing), the rules tool's unittest suite (132 passed) and tests/unit/test_locked_test_environments.py with tests/unit/test_alembic_workflows.py (37 passed).

🤖 Generated with Claude Code

@anth-volk
anth-volk added this pull request to stack #3998 October 21, 2026 20:43
anth-volk and others added 2 commits October 22, 2026 17:00
scripts/rules/locked_installs.py scans workflow run blocks, shell
scripts, Makefile recipes, Dockerfiles and Modal image definitions for
commands and calls that install Python packages, and reports each one
that does not install exactly what uv.lock pins (uv sync/uv run without
--frozen or --locked, pip, uv pip, uvx, uv tool, pipx and the Modal
pip_install family). A [[rule.exception]] can now name one site with
`match`, and end on a `removal` condition, a review date with an
`issue`, or be a `pinned-tool` install the scanner checks is pinned with
==. The proof fixtures hold one bad example of each install kind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The scanner now also reports every pytest, coverage run, unittest, tox
or nox command in workflow run blocks, shell scripts, Makefile recipes
and Dockerfiles that does not run through uv run --frozen or --locked.
The proof fixtures hold a test step on setup-python's interpreter, a
script and a Make target that do the same. The scan-coverage test now
also skips the registry's own rule fixtures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk and others added 10 commits October 22, 2026 17:05
The scanner now reports a pyproject.toml without a committed uv.lock
beside it, and a project whose uv.lock no pull request job checks with
uv lock --check (following the scripts and Make targets each job runs).
The pull request quality-guards job now installs uv 0.13.4, the version
the Cloud Run image and the release job use, and runs uv lock --check;
a test checks that its uv version matches the release job's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The scanner now requires a workflow with a schedule trigger whose job,
through the scripts and Make targets it runs, relocks with upgrades
(--upgrade, --upgrade-package, -U or -P) and opens or updates a pull
request, and that the relock covers every committed uv.lock. It reads
only the workflow configuration; it does not run the workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Registers api.dependencies.lock-committed, install-from-lock,
test-from-lock and scheduled-relock, each enforced by the new
locked-installs quality guard with proof fixtures, and documents them
in docs/engineering/skills/dependencies.md.

Installs that bypass uv.lock are now registered exceptions, each naming
its command, a reason, and a removal condition or the pinned-tool
category: the pinned shellcheck and ruff installs, the two operational
workflows that pip install for a bare python, the bundle updater's
pip install, the collector check's uv run --with, and docker/Dockerfile.

Three sites move to the lock where that installs the same versions:
the seed workflow's uv run steps and migrate_v2_metadata_schema.sh run
with --frozen after the job's uv sync --frozen, and the rules tool's
tests run through uv run --frozen in the quality-guards job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/unit/test_locked_test_environments.py now uses the guard's
makefile_recipes and executed_lines instead of its own copies, and
drops its two checks that test jobs install nothing with pip and run
tests through uv run --frozen: the guard checks both for every file,
not only test jobs. It keeps the checks the guard does not make: the
Make targets CI calls, the Python version each test job selects, and
the operational jobs' explicit install. Adds the changelog fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A composite action keeps its steps under runs: steps:, not jobs:, so
the workflow parser found none of its commands: an install in one was
counted as scanned but never checked. Composite actions now have their
own kind, whose run: values and blocks are read directly, and the
scan-coverage test includes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The scheduled-relock check accepted an upgrade flag anywhere in the
job, so a plain uv lock next to pip install -U pip passed. The flag must
now be on the uv lock command, or in a shell array the command expands
("${arguments[@]}"), as relock_dependencies.sh builds its
--upgrade-package list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uv run --frozen --no-sync runs in the environment a uv sync --frozen
created, which rule test-from-lock allows, but the test-command check
treated every --no-sync run as outside the lock. A uv run test command
now counts as locked when it has --frozen or --locked and no --with or
--no-project.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rules-registry field table now lists match, removal, issue and
category for [[rule.exception]] and points to dependencies.md. The
test-from-lock rule's scope includes Dockerfiles, as its statement
says, and an exception without a match is reported as naming no site,
which fits both rules that read it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guard's tests reached classify_command only through fixtures, which
left two of its results untested: uv run --no-sync installs nothing, and
installers such as poetry install are neither locked nor pinned. Check
one command of every kind directly, before the function is restructured
for ruff's return-statement limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
master now selects every ruff rule, and PLR0911 allows six return
statements per function. _file_kind had seven and classify_command
eight. _file_kind now takes the first kind from an ordered table of
naming tests, and the uv run branch of classify_command moves into
_classify_uv_run. The results are unchanged, as the classification and
scan-coverage tests show.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Register and check rules for using, installing, testing and relocking from uv.lock

1 participant