fix(toolchain): fix crash on Windows when precompiling is enabled - #4082
Open
rdesgroppes wants to merge 1 commit into
Open
fix(toolchain): fix crash on Windows when precompiling is enabled#4082rdesgroppes wants to merge 1 commit into
rdesgroppes wants to merge 1 commit into
Conversation
`python/private/py_executable.bzl`'s `_maybe_add_test_main_validation` fix (bazel-contrib#4079) noted `precompile.bzl` as a remaining user of the same `exec_interpreter` relocation issue, needing the same migration. There was no existing test exercising `_precompile`'s action at all: `tests/base_rules/precompile`'s suite is `analysis_test`-only, checking declared providers, never actually running the precompiler. Reproducing this on Windows therefore required a real `bazel build`, via the new `test_precompile_enabled_succeeds`: ``` bazel test \ //tests/base_rules/precompile:test_precompile_enabled_succeeds ... ERROR: .../tests/base_rules/precompile/BUILD.bazel:3:22: Python precompiling .../test_precompile_enabled_succeeds_main.py into .../test_precompile_enabled_succeeds_main.cpython-311.pyc failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../multiplex-worker-1-PyCompile.log ---8<---8<--- (empty) ---8<---8<--- End of log ---8<---8<--- ``` The worker crashes at startup, unable to find its DLLs, before it can write anything to its own log or respond over the worker protocol. `_precompile` now uses `actions_run()` with `exec_runtime`, exactly as `_maybe_add_test_main_validation` does, instead of `exec_tools_info.exec_interpreter[DefaultInfo].files_to_run`. Reproducing and fixing this also uncovered two more problems, both specific to the precompiler's worker mode and unrelated to `exec_interpreter`. First, `tools/precompiler/precompiler.py`'s persistent worker reads each JSON request as a single line via `asyncio.StreamReader`, whose default 64KiB limit is exceeded once every interpreter distribution file, previously hidden by relocation into a much smaller symlink tree, shows up as an actual, individually-digested action input: ``` ValueError: Separator is not found, and chunk exceed the limit ``` A CPython 3.11 distribution's ~2,260 inputs measure ~470KiB this way; `1 << 22` (4MiB) leaves ample headroom. Second, the worker's default implementation, `_AsyncPersistentWorker`, can't start on Windows at all: `asyncio`'s `ProactorEventLoop` fails to wrap `stdin`/`stdout` as pipe transports, with: ``` OSError: [WinError 6] The handle is invalid ``` Bazel gives workers anonymous pipes (`CreatePipe`) for stdio, which never support overlapped I/O, so `asyncio`'s `ProactorEventLoop` can't register them with an I/O completion port. This is unrelated to precompiling's relocation bug: nothing exercises this worker on Windows today. `_SerialPersistentWorker`, the blocking-I/O alternative already present in the file, has no such issue, so `--worker_impl` now defaults to `serial` on Windows. `tests/base_rules/precompile:test_precompile_enabled_succeeds` is a real, executing `py_test` with `precompile = "enabled"`, added alongside the analysis-only suite to close this gap: it forces the precompiler action to actually run, and needs no CI wiring since it carries no tag excluding it from the existing Windows job's default test sweep.
rdesgroppes
force-pushed
the
fix-precompile-exec-interpreter-windows
branch
from
August 19, 2026 09:37
3869654 to
b208186
Compare
rdesgroppes
marked this pull request as ready for review
August 19, 2026 09:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
python/private/py_executable.bzl's_maybe_add_test_main_validationfix (#4079) notedprecompile.bzlas a remaining user of the sameexec_interpreterrelocation issue, needing the same migration.There was no existing test exercising
_precompile's action at all:tests/base_rules/precompile's suite isanalysis_test-only, checking declared providers, never actually running the precompiler.Reproducing this on Windows therefore required a real
bazel build, via the newtest_precompile_enabled_succeeds:The worker crashes at startup, unable to find its DLLs, before it can write anything to its own log or respond over the worker protocol.
_precompilenow usesactions_run()withexec_runtime, exactly as_maybe_add_test_main_validationdoes, instead ofexec_tools_info.exec_interpreter[DefaultInfo].files_to_run.Reproducing and fixing this also uncovered two more problems, both specific to the precompiler's worker mode and unrelated to
exec_interpreter.First,
tools/precompiler/precompiler.py's persistent worker reads each JSON request as a single line viaasyncio.StreamReader, whose default 64KiB limit is exceeded once every interpreter distribution file, previously hidden by relocation into a much smaller symlink tree, shows up as an actual, individually-digested action input:A CPython 3.11 distribution's ~2,260 inputs measure ~470KiB this way;
1 << 22(4MiB) leaves ample headroom.Second, the worker's default implementation,
_AsyncPersistentWorker, can't start on Windows at all:asyncio'sProactorEventLoopfails to wrapstdin/stdoutas pipe transports, with:Bazel gives workers anonymous pipes (
CreatePipe) for stdio, which never support overlapped I/O, soasyncio'sProactorEventLoopcan't register them with an I/O completion port.This is unrelated to precompiling's relocation bug: nothing exercises this worker on Windows today.
_SerialPersistentWorker, the blocking-I/O alternative already present in the file, has no such issue, so--worker_implnow defaults toserialon Windows.tests/base_rules/precompile:test_precompile_enabled_succeedsis a real, executingpy_testwithprecompile = "enabled", added alongside the analysis-only suite to close this gap: it forces the precompiler action to actually run, and needs no CI wiring since it carries no tag excluding it from the existing Windows job's default test sweep.