diff --git a/.github/workflows/cooled-hooks.yaml b/.github/workflows/cooled-hooks.yaml index 833bacc..242293e 100644 --- a/.github/workflows/cooled-hooks.yaml +++ b/.github/workflows/cooled-hooks.yaml @@ -7,7 +7,7 @@ on: type: string default: 7 days exclude-newer-package: - description: Newline-separated name=date cutoffs that override exclude-newer for single packages + description: Newline-separated name=date cutoffs for single locked tools; hook environments ignore them type: string default: "" permissions: {} diff --git a/changelog.d/+cooled-prek-pins.bugfix.md b/changelog.d/+cooled-prek-pins.bugfix.md new file mode 100644 index 0000000..3e6340b --- /dev/null +++ b/changelog.d/+cooled-prek-pins.bugfix.md @@ -0,0 +1,2 @@ +`cooled-prek` admits a hook's exact pinned release up to its upload time while that release is younger than the +seven-day cutoff, so a hook bumped to a release younger than that installs instead of failing `cooled-hooks.yaml`. diff --git a/docs/locked-tools.md b/docs/locked-tools.md index e8f2382..8a74244 100644 --- a/docs/locked-tools.md +++ b/docs/locked-tools.md @@ -93,10 +93,15 @@ configurations, which no guard checks. Input `args` **(required)**: whitespace-separated prek arguments, such as `install-hooks`. prek must be on `PATH`. -A package with a `packages..uploaded-prior-to` exception in `pyproject.toml`'s `[tool.nab]` is admitted up to that -date in hook environments too. So a hook dependency pinned to a release newer than the cutoff, such as `sphinx` for -`docstrfmt`, needs the same temporary exception the release lock uses; delete it once the release is a week old. -Pitfall: hook environments read exceptions only from `pyproject.toml`, not from a standalone `nab.toml`. +A Python hook's exact pin, `name==version`, admits that release up to its upload time while it is younger than the +cutoff, so a hook bumped to a release younger than seven days installs. The pin comes from the hook's +`additional_dependencies` or from its repository's install requirements at the hook's `rev`: `install_requires` in +`setup.py` or `setup.cfg`, or `[project].dependencies` in `pyproject.toml`. Dev, test, and extra requirements are not +read. Once the release is a week old, nothing is written and the normal cutoff applies. A newer release of the pinned +package, uploaded after the pinned one, stays blocked in every hook environment. The pinned release's dependencies and +any ranges stay cooled. A pin and a `packages..uploaded-prior-to` exception for the same package: the later date +wins. `cooled-prek` fetches the hook repository and looks the pinned release up on PyPI; a failed fetch or lookup fails +the run. Pitfall: hook environments read exceptions only from `pyproject.toml`, not from a standalone `nab.toml`. ## Install locked tools: `install-locked-tools` diff --git a/docs/pr-checks.md b/docs/pr-checks.md index f859a69..a05b692 100644 --- a/docs/pr-checks.md +++ b/docs/pr-checks.md @@ -96,14 +96,16 @@ fails instead. needs: [test, hooks] ``` -| Input | Default | Meaning | -| ----------------------- | -------- | -------------------------------------------------------------------------------------- | -| `exclude-newer` | `7 days` | Upload cutoff for the lock reader that installs the caller's locked `release-prepare`. | -| `exclude-newer-package` | empty | Newline-separated `name=date` cutoffs that override `exclude-newer` for one package. | +| Input | Default | Meaning | +| ----------------------- | -------- | ----------------------------------------------------------------------------------------- | +| `exclude-newer` | `7 days` | Upload cutoff for the lock reader that installs the caller's locked `release-prepare`. | +| `exclude-newer-package` | empty | Newline-separated `name=date` cutoffs for one locked tool; hook environments ignore them. | prek comes from the caller's `release-prepare` lock group. Hook environments read cutoff exceptions from `[tool.nab]` in -`pyproject.toml`, as [`cooled-prek`](locked-tools.md#run-prek-under-the-cutoff-cooled-prek) describes. The prek cache is -keyed on `.pre-commit-config.yaml` and `pyproject.toml`, so a changed hook or exception rebuilds its environment. +`pyproject.toml`, and an exact hook pin admits its release while younger than the cutoff, as +[`cooled-prek`](locked-tools.md#run-prek-under-the-cutoff-cooled-prek) describes. The `exclude-newer-package` input only +reaches the locked release tools, never the hook environments. The prek cache is keyed on `.pre-commit-config.yaml` and +`pyproject.toml`, so a changed hook or exception rebuilds its environment. ## Release dry run diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index dc250c4..0231793 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -60,10 +60,13 @@ Run `nab lock --all-groups --all-extras` with the Nab version the lock records, The lock's `[tool.nab] created-at` is missing, has no time zone, or lies ahead of the runner's clock. Regenerate the lock. -**A hook fails to install during a release or maintenance run, with uv finding no matching version.** The hook pins a -release younger than seven days, and [`cooled-prek`](locked-tools.md#run-prek-under-the-cutoff-cooled-prek) builds hook -environments under that cutoff. Wait for the release to age, or add a dated `[tool.nab.packages.]` exception in -`pyproject.toml`; hook environments ignore exceptions in `nab.toml`. +**A hook fails to install during a release or maintenance run, with uv finding no matching version.** A package the hook +resolves to a release younger than seven days is not exactly pinned, and +[`cooled-prek`](locked-tools.md#run-prek-under-the-cutoff-cooled-prek) builds hook environments under that cutoff. An +exact pin (`name==version` in `additional_dependencies` or the hook repository's install requirements) admits its +release while that release is younger than the cutoff, so this is a range, a transitive dependency, or a release newer +than the pin. Wait for the release to age, or add a dated `[tool.nab.packages.]` exception in `pyproject.toml`; +hook environments ignore exceptions in `nab.toml`. ## Release preparation diff --git a/tests/cooled_prek/test_cooled_prek.py b/tests/cooled_prek/test_cooled_prek.py index ae2bb30..df732c8 100644 --- a/tests/cooled_prek/test_cooled_prek.py +++ b/tests/cooled_prek/test_cooled_prek.py @@ -2,12 +2,50 @@ import os import re +import shlex import subprocess +import sys +from datetime import UTC, datetime, timedelta from pathlib import Path +from textwrap import dedent import pytest import yaml from conftest import action +from pytest_httpserver import HTTPServer + +SCRIPT = Path(__file__).resolve().parents[2] / "tools/cooled_prek.sh" + + +def _fake_uv(directory: Path, httpserver: HTTPServer) -> None: + # The snippet runs as `uv run --with pyyaml python -I -`. The fake drops everything but the interpreter and + # imports a sitecustomize from a test-only directory, since -I ignores PYTHONPATH; it sends pypi.org to the server. + site = directory / "pypi-site" + site.mkdir() + base = httpserver.url_for("").rstrip("/") + (site / "sitecustomize.py").write_text( + dedent( + f""" + import urllib.request + from urllib.parse import urlsplit + + base = {base!r} + original_open = urllib.request.urlopen + + def route_open(request, *args, **kwargs): + parsed = urlsplit(request.full_url) + if parsed.hostname == "pypi.org": + request = urllib.request.Request(base + parsed.path, headers=dict(request.header_items())) + return original_open(request, *args, **kwargs) + + urllib.request.urlopen = route_open + """ + ) + ) + code = f"import sys; sys.path.insert(0, {str(site)!r}); import sitecustomize; exec(sys.stdin.read())" + fake = directory / "uv" + fake.write_text(f"#!/bin/sh\nexec {sys.executable!r} -I -c {shlex.quote(code)}\n") + fake.chmod(0o755) def test_no_hook_run_bypasses_cooled_prek() -> None: @@ -41,11 +79,12 @@ def test_prepare_installs_no_hook_it_does_not_run(name: str) -> None: assert "install-hooks" not in str(workflow["jobs"]["assemble"]["steps"]) -def test_cooled_prek_action_passes_each_argument(tmp_path: Path, action_scripts: Path) -> None: +def test_cooled_prek_action_passes_each_argument(tmp_path: Path, action_scripts: Path, httpserver: HTTPServer) -> None: fake = tmp_path / "bin/prek" fake.parent.mkdir() fake.write_text('#!/bin/sh\nprintf "%s\\n" "$@" >"$RUNNER_TEMP/args"\n') fake.chmod(0o755) + _fake_uv(fake.parent, httpserver) result = action( tmp_path, @@ -76,13 +115,14 @@ def test_cooled_prek_action_passes_each_argument(tmp_path: Path, action_scripts: ], ) def test_cooled_prek_builds_hooks_cooled_and_runs_them_with_the_caller_environment( - tmp_path: Path, arguments: list[str], install: list[str] + tmp_path: Path, arguments: list[str], install: list[str], httpserver: HTTPServer ) -> None: fake = tmp_path / "prek" fake.write_text( '#!/bin/sh\nn=$(ls calls 2>/dev/null | wc -l)\nmkdir -p calls/$n\nprintf "%s\\n" "$@" >calls/$n/args\nenv >calls/$n/env\n' ) fake.chmod(0o755) + _fake_uv(tmp_path, httpserver) leaked = dict.fromkeys(("UV_INDEX_URL", "UV_EXCLUDE_NEWER_PACKAGE", "UV_SYSTEM_CERTS", "SSL_CERT_FILE"), "leak") subprocess.run( @@ -113,10 +153,11 @@ def test_cooled_prek_builds_hooks_cooled_and_runs_them_with_the_caller_environme assert leaked.items() <= run_env.items() -def test_cooled_prek_admits_nab_cutoff_exceptions(tmp_path: Path) -> None: +def test_cooled_prek_admits_nab_cutoff_exceptions(tmp_path: Path, httpserver: HTTPServer) -> None: fake = tmp_path / "prek" fake.write_text('#!/bin/sh\ncat "$UV_CONFIG_FILE" >config\n') fake.chmod(0o755) + _fake_uv(tmp_path, httpserver) (tmp_path / "pyproject.toml").write_text( '[tool.nab]\nuploaded-prior-to = "P7D"\npackages.sphinx.uploaded-prior-to = 2026-10-21T00:05:13Z\n' 'packages.release-creator.uploaded-prior-to = "2026-10-26T18:24:46Z"\npackages.uv.decision-order = "stable"\n' @@ -136,10 +177,11 @@ def test_cooled_prek_admits_nab_cutoff_exceptions(tmp_path: Path) -> None: ] -def test_cooled_prek_without_exceptions_sets_no_config_file(tmp_path: Path) -> None: +def test_cooled_prek_without_exceptions_sets_no_config_file(tmp_path: Path, httpserver: HTTPServer) -> None: fake = tmp_path / "prek" fake.write_text('#!/bin/sh\nprintf "%s" "${UV_CONFIG_FILE-unset}" >config\n') fake.chmod(0o755) + _fake_uv(tmp_path, httpserver) (tmp_path / "pyproject.toml").write_text('[tool.nab]\nuploaded-prior-to = "P7D"\n') subprocess.run( @@ -150,3 +192,253 @@ def test_cooled_prek_without_exceptions_sets_no_config_file(tmp_path: Path) -> N ) assert (tmp_path / "config").read_text() == "unset" + + +def _hook_repo(root: Path, files: dict[str, str]) -> Path: + repo = root / "hook" + repo.mkdir() + for name, text in {"README.md": "hook\n", **files}.items(): + (repo / name).write_text(text) + git = ["git", "-C", str(repo), "-c", "user.name=test", "-c", "user.email=test@example.com"] + subprocess.run(["git", "init", "-q", str(repo)], check=True) + subprocess.run([*git, "add", "."], check=True) + subprocess.run([*git, "-c", "commit.gpgsign=false", "commit", "-q", "-m", "release"], check=True) + subprocess.run([*git, "tag", "v1.0.0"], check=True) + return repo + + +def _config(root: Path, repo: Path, hook: dict[str, object]) -> None: + (root / ".pre-commit-config.yaml").write_text( + yaml.safe_dump({"repos": [{"repo": str(repo), "rev": "v1.0.0", "hooks": [{"id": "simplint", **hook}]}]}) + ) + + +def _cooled_prek(root: Path, httpserver: HTTPServer) -> subprocess.CompletedProcess[str]: + fake = root / "prek" + fake.write_text('#!/bin/sh\ncat "${UV_CONFIG_FILE:-/dev/null}" >config\n') + fake.chmod(0o755) + _fake_uv(root, httpserver) + return subprocess.run( + [str(SCRIPT), "install-hooks"], + cwd=root, + env=os.environ | {"PATH": f"{root}{os.pathsep}{os.environ['PATH']}", "NO_PROXY": "localhost,127.0.0.1"}, + capture_output=True, + text=True, + check=False, + ) + + +def _uploaded(days: int) -> datetime: + return datetime.now(UTC).replace(microsecond=0) - timedelta(days=days) + + +def _stamp(value: datetime) -> str: + return value.strftime("%Y-%m-%dT%H:%M:%SZ") + + +def _simple(httpserver: HTTPServer, releases: dict[str, datetime]) -> None: + # One wheel per release on the Simple JSON page of simplint, uploaded at the given time. + files = [ + {"filename": f"simplint-{version}-py3-none-any.whl", "upload-time": uploaded.isoformat()} + for version, uploaded in releases.items() + ] + httpserver.expect_request("/simple/simplint/").respond_with_json({"meta": {"api-version": "1.0"}, "files": files}) + + +def _admitted(uploaded: datetime) -> str: + return f'"simplint" = "{_stamp(uploaded + timedelta(seconds=1))}"' + + +PIN_SOURCES = [ + pytest.param({"setup.py": 'setup(install_requires=["simplint==0.7.1"])\n'}, {"language": "python"}, id="setup-py"), + pytest.param( + {"setup.cfg": "[options]\ninstall_requires =\n simplint==0.7.1\n"}, {"language": "python"}, id="setup-cfg" + ), + pytest.param( + {"pyproject.toml": '[project]\nname = "hook"\ndependencies = ["simplint==0.7.1"]\n'}, + {"language": "python"}, + id="pyproject", + ), + pytest.param( + { + "setup.py": 'setup(install_requires=["simplint==0.7.1"])\n', + ".pre-commit-hooks.yaml": "- id: simplint\n language: python\n entry: simplint\n", + }, + {}, + id="language-from-manifest", + ), + pytest.param( + {}, {"language": "python", "additional_dependencies": ["simplint==0.7.1"]}, id="additional-dependencies" + ), + pytest.param( + {"setup.py": 'setup(install_requires=["simplint[socks]==0.7.1; python_version>=3.10"])\n'}, + {"language": "python"}, + id="extras-and-marker", + ), +] + + +@pytest.mark.parametrize(("files", "hook"), PIN_SOURCES) +def test_cooled_prek_admits_pin_up_to_its_upload( + tmp_path: Path, httpserver: HTTPServer, files: dict[str, str], hook: dict[str, object] +) -> None: + uploaded = _uploaded(1) + _simple(httpserver, {"0.7.1": uploaded}) + _config(tmp_path, _hook_repo(tmp_path, files), hook) + + result = _cooled_prek(tmp_path, httpserver) + + assert result.returncode == 0, result.stderr + assert (tmp_path / "config").read_text().splitlines() == ["[exclude-newer-package]", _admitted(uploaded)] + assert _admitted(uploaded) in result.stdout + + +@pytest.mark.parametrize( + ("files", "hook"), + [ + pytest.param({}, {"language": "python", "additional_dependencies": ["simplint>=0.7"]}, id="range"), + pytest.param( + {"setup.py": 'setup(install_requires=["simplint>=0.7"])\n'}, {"language": "python"}, id="metadata-range" + ), + pytest.param({}, {"language": "system", "additional_dependencies": ["simplint==0.7.1"]}, id="non-python-hook"), + pytest.param( + {"setup.py": 'setup(install_requires=["simplint==0.7.1"])\n'}, + {"language": "system"}, + id="non-python-metadata", + ), + pytest.param( + {"setup.py": 'setup(extras_require={"test": ["simplint==0.7.1"]}, tests_require=["simplint==0.7.1"])\n'}, + {"language": "python"}, + id="dev-extras", + ), + pytest.param( + {"setup.cfg": "[options.extras_require]\ntest =\n simplint==0.7.1\n"}, + {"language": "python"}, + id="dev-setup-cfg", + ), + pytest.param( + { + "pyproject.toml": ( + '[project.optional-dependencies]\ntest = ["simplint==0.7.1"]\n' + '[dependency-groups]\ndev = ["simplint==0.7.1"]\n' + ) + }, + {"language": "python"}, + id="dev-pyproject", + ), + ], +) +def test_cooled_prek_leaves_ranges_dev_pins_and_non_python_hooks_cooled( + tmp_path: Path, httpserver: HTTPServer, files: dict[str, str], hook: dict[str, object] +) -> None: + _config(tmp_path, _hook_repo(tmp_path, files), hook) + + result = _cooled_prek(tmp_path, httpserver) + + assert result.returncode == 0, result.stderr + assert (tmp_path / "config").read_text() == "" + assert httpserver.log == [] + + +def test_cooled_prek_does_not_admit_newer_release(tmp_path: Path, httpserver: HTTPServer) -> None: + pinned = _uploaded(2) + _simple(httpserver, {"0.7.1": pinned, "0.8.0": _uploaded(1)}) + _config( + tmp_path, + _hook_repo(tmp_path, {"setup.py": 'setup(install_requires=["simplint==0.7.1"])\n'}), + {"language": "python"}, + ) + + result = _cooled_prek(tmp_path, httpserver) + + assert result.returncode == 0, result.stderr + assert (tmp_path / "config").read_text().splitlines() == ["[exclude-newer-package]", _admitted(pinned)] + + +def test_cooled_prek_does_not_write_pin_older_than_window(tmp_path: Path, httpserver: HTTPServer) -> None: + _simple(httpserver, {"0.7.1": _uploaded(8)}) + _config( + tmp_path, + _hook_repo(tmp_path, {"setup.py": 'setup(install_requires=["simplint==0.7.1"])\n'}), + {"language": "python"}, + ) + + result = _cooled_prek(tmp_path, httpserver) + + assert result.returncode == 0, result.stderr + assert (tmp_path / "config").read_text() == "" + + +@pytest.mark.parametrize( + ("nab", "expected"), + [ + pytest.param("{later}", "{later}", id="toml-datetime-later-wins"), + pytest.param("{earlier}", "pin", id="toml-datetime-earlier-loses"), + pytest.param('"{later}"', "{later}", id="string-later-wins"), + pytest.param('"{earlier}"', "pin", id="string-earlier-loses"), + pytest.param('"P3D"', "pin", id="duration-loses-to-pin"), + ], +) +def test_cooled_prek_compares_nab_exception_with_pin( + tmp_path: Path, httpserver: HTTPServer, nab: str, expected: str +) -> None: + pinned = _uploaded(2) + later, earlier = _stamp(pinned + timedelta(days=1)), _stamp(pinned - timedelta(days=1)) + _simple(httpserver, {"0.7.1": pinned}) + (tmp_path / "pyproject.toml").write_text( + f"[tool.nab]\npackages.simplint.uploaded-prior-to = {nab.format(later=later, earlier=earlier)}\n" + ) + _config( + tmp_path, + _hook_repo(tmp_path, {"setup.py": 'setup(install_requires=["simplint==0.7.1"])\n'}), + {"language": "python"}, + ) + + result = _cooled_prek(tmp_path, httpserver) + + assert result.returncode == 0, result.stderr + value = _admitted(pinned) if expected == "pin" else f'"simplint" = "{expected.format(later=later)}"' + assert (tmp_path / "config").read_text().splitlines() == ["[exclude-newer-package]", value] + + +def test_cooled_prek_keeps_nab_duration_without_pin(tmp_path: Path, httpserver: HTTPServer) -> None: + (tmp_path / "pyproject.toml").write_text('[tool.nab]\npackages.sphinx.uploaded-prior-to = "P3D"\n') + _config(tmp_path, _hook_repo(tmp_path, {}), {"language": "python"}) + + result = _cooled_prek(tmp_path, httpserver) + + assert result.returncode == 0, result.stderr + assert (tmp_path / "config").read_text().splitlines() == ["[exclude-newer-package]", '"sphinx" = "P3D"'] + + +@pytest.mark.parametrize( + ("status", "error"), + [ + pytest.param(404, "cannot read simplint from PyPI", id="unknown-project"), + pytest.param(200, "PyPI has no files for simplint==0.7.1", id="version-missing"), + ], +) +def test_cooled_prek_fails_when_pin_cannot_be_looked_up( + tmp_path: Path, httpserver: HTTPServer, status: int, error: str +) -> None: + if status == 404: + httpserver.expect_request("/simple/simplint/").respond_with_data("", status=404) + else: + _simple(httpserver, {"0.6.0": _uploaded(1)}) + repo = _hook_repo(tmp_path, {"setup.py": 'setup(install_requires=["simplint==0.7.1"])\n'}) + _config(tmp_path, repo, {"language": "python"}) + + result = _cooled_prek(tmp_path, httpserver) + + assert result.returncode != 0 + assert error in result.stderr + assert str(repo) in result.stderr + + +def test_cooled_prek_fails_when_hook_repo_cannot_be_fetched(tmp_path: Path, httpserver: HTTPServer) -> None: + _config(tmp_path, tmp_path / "missing", {"language": "python"}) + + result = _cooled_prek(tmp_path, httpserver) + + assert result.returncode != 0 + assert "cannot fetch v1.0.0 of" in result.stderr diff --git a/tools/cooled_prek.sh b/tools/cooled_prek.sh index f2bac0b..ee350f8 100755 --- a/tools/cooled_prek.sh +++ b/tools/cooled_prek.sh @@ -8,18 +8,162 @@ cooled() ( unset UV_INSECURE_HOST UV_SYSTEM_CERTS SSL_CERT_FILE SSL_CERT_DIR export UV_EXCLUDE_NEWER="7 days" UV_NO_CONFIG=1 UV_NO_SOURCES=1 # uv pip, which prek calls, reads per-package cutoffs only from a config file, so the [tool.nab] exceptions the - # release lock already admits are written to one that holds nothing else. + # release lock already admits and the exact hook pins are written to one that holds nothing else. A pinned release + # stays admitted while it is younger than the cutoff, up to its upload time, so the pin installs and newer releases + # of the package stay blocked; its dependencies stay cooled. PyYAML comes from uv under the same cutoff because the + # runner's python3 may not have it and the hook manifests are YAML. The snippet runs under -I, which drops the + # runner's PYTHONPATH and keeps the repository directory off sys.path. exceptions=$( - python3 -I - <<'PY' + uv run --no-project --with pyyaml python -I - <<'PY' +import configparser +import datetime +import json import pathlib +import re +import subprocess +import sys +import tempfile import tomllib +import urllib.request + +import yaml + +HOOK_LANGUAGES = {"python", "python_venv"} +NAME = r"[A-Za-z0-9][A-Za-z0-9._-]*" +VERSION = r"[A-Za-z0-9][A-Za-z0-9.+!_-]*" +REQUIREMENT = re.compile(rf"\s*({NAME})(?:\[[^\]]*\])?==({VERSION})\s*") +SETUP_PY_INSTALL_REQUIRES = re.compile(r"install_requires\s*=\s*\[((?:[^\[\]]|\[[^\]]*\])*)\]") +QUOTED = re.compile(r"[\"']([^\"']+)[\"']") +SEVEN_DAYS_AGO = datetime.datetime.now(datetime.UTC) - datetime.timedelta(days=7) + + +def normalize(name): + return re.sub(r"[-_.]+", "-", name).lower() + + +def exact_pins(requirements): + pins = set() + for requirement in requirements: + if match := REQUIREMENT.fullmatch(requirement.split(";")[0]): + pins.add((normalize(match[1]), match[2])) + return pins + + +def setup_py_pins(text): + match = SETUP_PY_INSTALL_REQUIRES.search(text) + return exact_pins(QUOTED.findall(match[1])) if match else set() + + +def setup_cfg_pins(text): + parser = configparser.ConfigParser(interpolation=None) + parser.read_string(text) + return exact_pins(parser.get("options", "install_requires", fallback="").splitlines()) + + +def pyproject_pins(text): + return exact_pins(tomllib.loads(text).get("project", {}).get("dependencies", [])) + + +PACKAGING_READERS = {"setup.py": setup_py_pins, "setup.cfg": setup_cfg_pins, "pyproject.toml": pyproject_pins} + + +def repo_files(url, rev): + with tempfile.TemporaryDirectory() as directory: + git = ["git", "-C", directory] + try: + subprocess.run(["git", "init", "-q", directory], capture_output=True, text=True, check=True) + subprocess.run( + [*git, "fetch", "-q", "--depth", "1", "--filter=blob:none", url, rev], + capture_output=True, + text=True, + check=True, + ) + root = subprocess.run( + [*git, "ls-tree", "--name-only", "FETCH_HEAD"], capture_output=True, text=True, check=True + ).stdout.split() + wanted = {".pre-commit-hooks.yaml", *PACKAGING_READERS} + return { + name: subprocess.run( + [*git, "show", f"FETCH_HEAD:{name}"], capture_output=True, text=True, check=True + ).stdout + for name in root + if name in wanted + } + except subprocess.CalledProcessError as error: + sys.exit(f"cooled-prek: cannot fetch {rev} of {url}: {error.stderr.strip()}") + + +def hook_pins(url, rev, hooks): + files = repo_files(url, rev) + manifest = { + hook["id"]: hook.get("language") for hook in yaml.safe_load(files.get(".pre-commit-hooks.yaml", "")) or [] + } + python_hooks = [hook for hook in hooks if (hook.get("language") or manifest.get(hook["id"])) in HOOK_LANGUAGES] + if not python_hooks: + return set() + pins = exact_pins(requirement for hook in python_hooks for requirement in hook.get("additional_dependencies", [])) + for name, reader in PACKAGING_READERS.items(): + if name in files: + pins |= reader(files[name]) + return pins + + +def upload_time(url, name, version): + # Fixed PyPI origin, read with the same request as tools/verify_pypi_files.py. + request = urllib.request.Request( + f"https://pypi.org/simple/{name}/", + headers={"Accept": "application/vnd.pypi.simple.v1+json", "Cache-Control": "no-cache"}, + ) + parts = "[-_.]+".join(re.escape(part) for part in name.split("-")) + pattern = re.compile(rf"{parts}-{re.escape(version)}(?:-|\.tar\.gz$|\.zip$)", re.IGNORECASE) + try: + with urllib.request.urlopen( # ruff: ignore[suspicious-url-open-usage] # Fixed PyPI origin. + request, timeout=30 + ) as response: + files = json.load(response)["files"] + times = [ + datetime.datetime.fromisoformat(file["upload-time"]) for file in files if pattern.match(file["filename"]) + ] + except (OSError, ValueError, KeyError) as error: + sys.exit(f"cooled-prek: cannot read {name} from PyPI for the hook repository {url}: {error}") + if not times: + sys.exit(f"cooled-prek: PyPI has no files for {name}=={version}, pinned by the hook repository {url}") + return max(times) + + +def as_datetime(value): + if isinstance(value, str): + try: + value = datetime.datetime.fromisoformat(value) + except ValueError: + return None + return value if isinstance(value, datetime.datetime) and value.tzinfo is not None else None + path = pathlib.Path("pyproject.toml") packages = tomllib.loads(path.read_text()).get("tool", {}).get("nab", {}).get("packages", {}) if path.is_file() else {} -for name, settings in sorted(packages.items()): - if (cutoff := settings.get("uploaded-prior-to")) is not None: - value = cutoff.isoformat().replace("+00:00", "Z") if hasattr(cutoff, "isoformat") else str(cutoff) - print(f'"{name}" = "{value}"') +cutoffs = { + normalize(name): settings["uploaded-prior-to"] + for name, settings in packages.items() + if settings.get("uploaded-prior-to") is not None +} +path = pathlib.Path(".pre-commit-config.yaml") +config = (yaml.safe_load(path.read_text()) if path.is_file() else None) or {} +pins = {} +for repo in config.get("repos", []): + if repo["repo"] in {"local", "meta"}: + continue + for name, version in hook_pins(repo["repo"], repo["rev"], repo.get("hooks", [])): + cutoff = upload_time(repo["repo"], name, version).replace(microsecond=0) + datetime.timedelta(seconds=1) + if cutoff > SEVEN_DAYS_AGO: + pins[name] = max(pins.get(name, cutoff), cutoff) +for name, cutoff in pins.items(): + current = as_datetime(cutoffs.get(name)) + if current is None or current < cutoff: + cutoffs[name] = cutoff +for name, value in sorted(cutoffs.items()): + value = value.isoformat().replace("+00:00", "Z") if hasattr(value, "isoformat") else str(value) + print(f'"{name}" = "{value}"') PY ) if [[ -n "$exceptions" ]]; then @@ -27,7 +171,7 @@ PY printf '[exclude-newer-package]\n%s\n' "$exceptions" >"$config" export UV_CONFIG_FILE="$config" echo "Running prek $* with hook environments built under the 7 days cutoff, repository uv settings ignored," \ - "[tool.nab] cutoff exceptions admitted: $(tr '\n' ' ' <<<"$exceptions")" + "[tool.nab] exceptions and exact hook pins admitted up to their dates: $(tr '\n' ' ' <<<"$exceptions")" else echo "Running prek $* with hook environments built under the 7 days cutoff, repository uv settings ignored" fi