Skip to content

ci: cache native dependency builds in Verify jobs #1177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kvlonge merged 3 commits into from
Oct 20, 2026
Merged

ci: cache native dependency builds in Verify jobs #1177

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bd4eafe
ci: cache native dependency builds in Verify jobs
kvlonge Oct 20, 2026
57ef954
Merge remote-tracking branch 'origin/main' into ci/rust-build-cache
kvlonge Oct 20, 2026
c26e5a2
Merge branch 'main' into ci/rust-build-cache
kvlonge Oct 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions .github/workflows/build-distributions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,16 +127,29 @@ jobs:
if: inputs.prebuilt-run-id == ''
with:
command: build
args: --release --out dist --features extension-module
# One platform records a cargo build timing report for build time work.
args: >-
--release --out dist --features extension-module
${{ matrix.platform.name == 'linux-x86_64' && '--timings' || '' }}
target: ${{ matrix.platform.target }}
manylinux: auto
- uses: PyO3/maturin-action@v1
if: inputs.prebuilt-run-id == ''
with:
command: build
args: --release --out dist --manifest-path crates/sqlbuild-cli/Cargo.toml
args: >-
--release --out dist --manifest-path crates/sqlbuild-cli/Cargo.toml
${{ matrix.platform.name == 'linux-x86_64' && '--timings' || '' }}
target: ${{ matrix.platform.target }}
manylinux: auto
- name: Upload the cargo build timing reports
if: inputs.prebuilt-run-id == '' && matrix.platform.name == 'linux-x86_64'
uses: actions/upload-artifact@v4
with:
name: cargo-timings-${{ matrix.platform.name }}
path: target/cargo-timings/cargo-timing-*.html
if-no-files-found: warn
retention-days: 30
- name: Download prebuilt wheels
if: inputs.prebuilt-run-id != ''
uses: actions/download-artifact@v4
Expand Down
126 changes: 126 additions & 0 deletions .github/workflows/rust-cache.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
name: Rust Build Cache

# Saves the native dependency builds that Verify restores. Verify never saves its own: a cache saved
# by a pull request or merge queue run is visible only to that ref, while one saved on main is
# visible to every pull request. Each job runs the same cargo builds as the Verify jobs sharing its
# key, so the cached artifacts match theirs, and builds only when main has no cache for the current
# lockfile and toolchain. Release distributions never use these caches.
on:
push:
branches: [main]
# A new stable toolchain changes the key without a push to main.
schedule:
- cron: "30 4 * * *"
workflow_dispatch:

permissions:
contents: read

concurrency:
group: rust-cache-${{ github.ref }}
cancel-in-progress: false

jobs:
linux:
name: Linux extension
runs-on: ubuntu-24.04
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
version: "0.12.3"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- name: Look up the cache
id: lookup
uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
lookup-only: true
save-if: false
- name: Restore the closest cache
if: steps.lookup.outputs.cache-hit != 'true'
uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
- name: Install Python
if: steps.lookup.outputs.cache-hit != 'true'
run: uv python install 3.12
- name: Build the extension as Verify does
if: steps.lookup.outputs.cache-hit != 'true'
run: uv sync --locked --all-groups --all-extras

checks:
name: Linux extension and native checks
runs-on: ubuntu-24.04
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
version: "0.12.3"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- name: Look up the cache
id: lookup
uses: Swatinem/rust-cache@v2
with:
shared-key: verify-checks
lookup-only: true
save-if: false
- name: Restore the closest cache
if: steps.lookup.outputs.cache-hit != 'true'
uses: Swatinem/rust-cache@v2
with:
shared-key: verify-checks
- name: Install Python
if: steps.lookup.outputs.cache-hit != 'true'
run: uv python install 3.12
- name: Build the extension as Verify does
if: steps.lookup.outputs.cache-hit != 'true'
run: uv sync --locked --all-groups --all-extras
- name: Build the native checks as make rust-check does
if: steps.lookup.outputs.cache-hit != 'true'
run: |
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace --all-features --no-run

windows:
name: Windows extension
runs-on: windows-2025
timeout-minutes: 60
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
version: "0.12.3"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- name: Look up the cache
id: lookup
uses: Swatinem/rust-cache@v2
with:
shared-key: verify-windows
lookup-only: true
save-if: false
- name: Restore the closest cache
if: steps.lookup.outputs.cache-hit != 'true'
uses: Swatinem/rust-cache@v2
with:
shared-key: verify-windows
- name: Build the native discovery tests as Verify does
if: steps.lookup.outputs.cache-hit != 'true'
run: cargo test --locked -p sqlbuild-discovery --no-run
- name: Install Python
if: steps.lookup.outputs.cache-hit != 'true'
run: uv python install 3.12
- name: Build the extension as Verify does
if: steps.lookup.outputs.cache-hit != 'true'
run: uv sync --locked --no-dev --python 3.12
64 changes: 64 additions & 0 deletions .github/workflows/verify.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,13 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
# Restores the native dependency builds that rust-cache.yml saves on main; workspace crates
# always build from source. Verify never saves: a pull request or merge queue cache is
# visible only to its own ref. Jobs that run the same cargo builds share a key.
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-checks
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -315,6 +322,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -355,6 +366,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -398,6 +413,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install ${{ matrix.python }}
- name: Install dependencies
Expand Down Expand Up @@ -431,6 +450,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-windows
save-if: false
- name: Test the native discovery crate (walk, junctions, ordering, read errors)
run: cargo test --locked -p sqlbuild-discovery
- name: Install Python
Expand Down Expand Up @@ -481,6 +504,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -512,6 +539,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -541,6 +572,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -589,6 +624,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -668,8 +707,17 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Seed the base build directory with the cached dependency builds
run: |
if [[ -d target ]]; then
cp -a target base/target
fi
- name: Install head dependencies
run: uv sync --locked --all-groups --all-extras
- name: Install base dependencies
Expand Down Expand Up @@ -734,6 +782,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -801,6 +853,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -847,6 +903,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down Expand Up @@ -880,6 +940,10 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
with:
shared-key: verify-linux
save-if: false
- name: Install Python
run: uv python install 3.12
- name: Install dependencies
Expand Down
Loading