Repository navigation
build: compile only the Python module and the sqlquack binary for the Python tests - #172
Merged
Merged
Conversation
… Python tests python-module depended on gate-build, so make python-api compiled every workspace crate, tool and test target, polyglot-sql included, before running tests that take about a second and a half. A new python-build target builds only sqlquack-python's library and the sqlquack binary, in the gate profile with the gate features; python-module, and so python-api and verify-recorded, use it by default. CI's checks job sets PYTHON_MODULE_BUILD=gate-build, because check-merge has already built the whole gate there and its Python tests and replay should use that one build. Measured on one 8-core orb, fresh target directory each time, one run: make python-api cold went from 444 s wall (3,107 s user CPU, 4.8 GB peak RSS, 3.1 GB target) to 134 s wall (733 s user CPU, 2.4 GB peak RSS, 1.3 GB target). Warm went from 3.5 s to 2.3 s.
kvlonge
force-pushed
the
ci/python-module-build
branch
from
November 11, 2026 12:54
85be27f to
3f853df
Compare
kvlonge
enabled auto-merge
November 11, 2026 12:54
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Nov 11, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Nov 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
make python-apiran 22 tests that take about 1.5 s, butpython-moduledepended ongate-build, which compiles every workspace crate, tool and test target, polyglot-sql included. A cold run took minutes and needed several GB of memory, enough to fail on small machines.Changes
python-buildtarget: builds onlysqlquack-python's library and thesqlquackbinary, with the gate profile and the gate features, into the same target directory.python-module(and sopython-apiandverify-recorded) uses$(PYTHON_MODULE_BUILD), which defaults topython-build.PYTHON_MODULE_BUILD=gate-build:check-mergehas already built the whole gate there, so its Python tests and replay keep using that one build.Verification
Same 8-core orb, fresh target directory each time, one run each (
/usr/bin/time):make python-api22 passed in every run. After a full
gate-build, the first narrow build recompiles the 7 workspace crates whose features the rest of the workspace widens (about 75 s); later switches between the two builds take under a second.make check-cipassed, withpython-apion the narrow build.make verify-recordedon the narrow binary: 12,236 cases, 0 failed, 0 unproven, the same counts as main.