Skip to content

fix: do not fix the multiprocessing start method at import time on macOS - #1657

Merged
Jammy2211 merged 1 commit into
PyAutoLabs:mainfrom
samlange04:fix/multiprocessing-context-import-time
Oct 7, 2026
Merged

Jammy2211 merged 1 commit into
PyAutoLabs:mainfrom
samlange04:fix/multiprocessing-context-import-time

Conversation

@samlange04

Copy link
Copy Markdown
Contributor

Summary

Discussion: https://github.com/orgs/PyAutoLabs/discussions/27

Since e6279c5, fork_context() in autofit/non_linear/parallel/context.py returns multiprocessing.get_context() (no argument) on macOS and Windows. That call permanently fixes the interpreter's default start method, and fork_context() runs at import time via class Process(fork_context().Process) in process.py. Any code that imports autofit and then calls multiprocessing.set_start_method(...) raises RuntimeError: context has already been set.

test_autofit/conftest.py does exactly this on macOS, so plain python -m pytest test_autofit has been unable to load the conftest on a Mac since 2026-07-31.

This PR returns the multiprocessing module on those platforms instead. It exposes the same Process / Queue / Pool API but resolves the start method when a process is created, which is the pre-3.14 behaviour the helper exists to preserve. Non-macOS POSIX behaviour (pinned "fork", the point of e6279c5) is unchanged. The docstring explains the reasoning.

API Changes

  • None. fork_context() is internal. Its return type on macOS/Windows changes from a BaseContext instance to the multiprocessing module, which supports every attribute autofit uses on it (Process, Queue, Pool). No downstream library calls fork_context().

Test Plan

  • Cause confirmed by patching multiprocessing.context._default_context.get_context to print a stack on the first call while the context was unset: fires from parallel/process.py:47 -> parallel/context.py:33 during import autofit.
  • On macOS 15 / Python 3.13.7, before: python -m pytest test_autofit fails with ImportError while loading conftest ... RuntimeError: context has already been set. After: the conftest loads.
  • Full suite after the change on macOS: python -m pytest test_autofit, 2924 passed, 39 skipped, 1 failed. The failure is test_autofit/non_linear/samples/test_pdf.py::test_save_covariance_matrix, a last-digit floating-point text comparison that fails identically on unmodified main.
  • Nested-sampling and parallel tests (test_autofit/non_linear/search/nest/): 38 passed, 13 skipped.
  • Linux CI: the sys.platform != "darwin" branch is untouched, so behaviour there should be identical. Needs CI to confirm.

🤖 Generated with Claude Code

`fork_context()` (added in e6279c5) returned `multiprocessing.get_context()`
on macOS/Windows. Calling `get_context()` with no argument permanently
fixes the interpreter's default start method, and `fork_context()` runs at
import time via `class Process(fork_context().Process)`. Any code that
imports autofit and then calls `multiprocessing.set_start_method(...)`
therefore raised `RuntimeError: context has already been set`. The autofit
test suite does exactly this on macOS in `test_autofit/conftest.py`, so
`pytest` has failed to load the conftest on macOS since that commit.

Return the `multiprocessing` module on those platforms instead. It exposes
the same `Process` / `Queue` / `Pool` API but resolves the start method
when a process is created, which is the pre-3.14 behaviour this helper
exists to preserve. Non-macOS POSIX behaviour (pinned "fork") is
unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Jammy2211
Jammy2211 merged commit e807cc5 into PyAutoLabs:main Oct 7, 2026
4 checks passed
samlange04 pushed a commit to samlange04/PyAutoFit that referenced this pull request Oct 7, 2026
…t import)

Monkeypatch sys.platform to darwin/win32 and assert fork_context()
returns the multiprocessing module (Process/Queue/Pool) and never calls
multiprocessing.get_context() with no argument, the import-time call
that fixed the default start method before PyAutoLabs#1657.

Closes PyAutoLabs#1661

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@samlange04
samlange04 deleted the fix/multiprocessing-context-import-time branch October 16, 2026 04:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants