Skip to content

fix(toolchain): fix crash on Windows when precompiling is enabled - #4082

Open
rdesgroppes wants to merge 1 commit into
bazel-contrib:mainfrom
rdesgroppes:fix-precompile-exec-interpreter-windows
Open

fix(toolchain): fix crash on Windows when precompiling is enabled#4082
rdesgroppes wants to merge 1 commit into
bazel-contrib:mainfrom
rdesgroppes:fix-precompile-exec-interpreter-windows

Conversation

@rdesgroppes

@rdesgroppes rdesgroppes commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

python/private/py_executable.bzl's _maybe_add_test_main_validation fix (#4079) noted precompile.bzl as a remaining user of the same exec_interpreter relocation issue, needing the same migration.

There was no existing test exercising _precompile's action at all: tests/base_rules/precompile's suite is analysis_test-only, checking declared providers, never actually running the precompiler.
Reproducing this on Windows therefore required a real bazel build, via the new test_precompile_enabled_succeeds:

bazel test \
  //tests/base_rules/precompile:test_precompile_enabled_succeeds
...
ERROR: .../tests/base_rules/precompile/BUILD.bazel:3:22: Python
 precompiling .../test_precompile_enabled_succeeds_main.py into
 .../test_precompile_enabled_succeeds_main.cpython-311.pyc
 failed: Worker process did not return a WorkResponse:
---8<---8<--- Start of log, file at
 .../multiplex-worker-1-PyCompile.log ---8<---8<---
(empty)
---8<---8<--- End of log ---8<---8<---

The worker crashes at startup, unable to find its DLLs, before it can write anything to its own log or respond over the worker protocol.

_precompile now uses actions_run() with exec_runtime, exactly as _maybe_add_test_main_validation does, instead of exec_tools_info.exec_interpreter[DefaultInfo].files_to_run.

Reproducing and fixing this also uncovered two more problems, both specific to the precompiler's worker mode and unrelated to exec_interpreter.

First, tools/precompiler/precompiler.py's persistent worker reads each JSON request as a single line via asyncio.StreamReader, whose default 64KiB limit is exceeded once every interpreter distribution file, previously hidden by relocation into a much smaller symlink tree, shows up as an actual, individually-digested action input:

ValueError: Separator is not found, and chunk exceed the limit

A CPython 3.11 distribution's ~2,260 inputs measure ~470KiB this way; 1 << 22 (4MiB) leaves ample headroom.

Second, the worker's default implementation, _AsyncPersistentWorker, can't start on Windows at all: asyncio's ProactorEventLoop fails to wrap stdin/stdout as pipe transports, with:

OSError: [WinError 6] The handle is invalid

Bazel gives workers anonymous pipes (CreatePipe) for stdio, which never support overlapped I/O, so asyncio's ProactorEventLoop can't register them with an I/O completion port.
This is unrelated to precompiling's relocation bug: nothing exercises this worker on Windows today.
_SerialPersistentWorker, the blocking-I/O alternative already present in the file, has no such issue, so --worker_impl now defaults to serial on Windows.

tests/base_rules/precompile:test_precompile_enabled_succeeds is a real, executing py_test with precompile = "enabled", added alongside the analysis-only suite to close this gap: it forces the precompiler action to actually run, and needs no CI wiring since it carries no tag excluding it from the existing Windows job's default test sweep.

`python/private/py_executable.bzl`'s `_maybe_add_test_main_validation`
fix (bazel-contrib#4079) noted `precompile.bzl` as a remaining user of the same
`exec_interpreter` relocation issue, needing the same migration.

There was no existing test exercising `_precompile`'s action at all:
`tests/base_rules/precompile`'s suite is `analysis_test`-only, checking
declared providers, never actually running the precompiler.
Reproducing this on Windows therefore required a real `bazel build`,
via the new `test_precompile_enabled_succeeds`:
```
bazel test \
  //tests/base_rules/precompile:test_precompile_enabled_succeeds
...
ERROR: .../tests/base_rules/precompile/BUILD.bazel:3:22: Python
 precompiling .../test_precompile_enabled_succeeds_main.py into
 .../test_precompile_enabled_succeeds_main.cpython-311.pyc
 failed: Worker process did not return a WorkResponse:
---8<---8<--- Start of log, file at
 .../multiplex-worker-1-PyCompile.log ---8<---8<---
(empty)
---8<---8<--- End of log ---8<---8<---
```
The worker crashes at startup, unable to find its DLLs, before it can
write anything to its own log or respond over the worker protocol.

`_precompile` now uses `actions_run()` with `exec_runtime`, exactly as
`_maybe_add_test_main_validation` does, instead of
`exec_tools_info.exec_interpreter[DefaultInfo].files_to_run`.

Reproducing and fixing this also uncovered two more problems, both
specific to the precompiler's worker mode and unrelated to
`exec_interpreter`.

First, `tools/precompiler/precompiler.py`'s persistent worker reads
each JSON request as a single line via `asyncio.StreamReader`, whose
default 64KiB limit is exceeded once every interpreter distribution
file, previously hidden by relocation into a much smaller symlink
tree, shows up as an actual, individually-digested action input:
```
ValueError: Separator is not found, and chunk exceed the limit
```
A CPython 3.11 distribution's ~2,260 inputs measure ~470KiB this way;
`1 << 22` (4MiB) leaves ample headroom.

Second, the worker's default implementation, `_AsyncPersistentWorker`,
can't start on Windows at all: `asyncio`'s `ProactorEventLoop` fails to
wrap `stdin`/`stdout` as pipe transports, with:
```
OSError: [WinError 6] The handle is invalid
```
Bazel gives workers anonymous pipes (`CreatePipe`) for stdio, which
never support overlapped I/O, so `asyncio`'s `ProactorEventLoop` can't
register them with an I/O completion port.
This is unrelated to precompiling's relocation bug: nothing exercises
this worker on Windows today.
`_SerialPersistentWorker`, the blocking-I/O alternative already
present in the file, has no such issue, so `--worker_impl` now
defaults to `serial` on Windows.

`tests/base_rules/precompile:test_precompile_enabled_succeeds` is a
real, executing `py_test` with `precompile = "enabled"`, added
alongside the analysis-only suite to close this gap: it forces the
precompiler action to actually run, and needs no CI wiring since it
carries no tag excluding it from the existing Windows job's default
test sweep.
@rdesgroppes
rdesgroppes force-pushed the fix-precompile-exec-interpreter-windows branch from 3869654 to b208186 Compare August 19, 2026 09:37
@rdesgroppes rdesgroppes changed the title fix(precompiling): fix crash on Windows when precompiling is enabled fix(toolchain): fix crash on Windows when precompiling is enabled Aug 19, 2026
@rdesgroppes
rdesgroppes marked this pull request as ready for review August 19, 2026 09:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant