Skip to content

test: isolate EP output directories for parallel runs #1726

Closed

Description

@Jammy2211

Overview

test: isolate EP output directories for parallel runs

Plan

Override the graphical suite configuration fixture with per-test tmp_path output and balanced config push/pop. Ensure explicit output-directory consumers and teardown use the isolated location. Run graphical tests serially and under xdist with identical collected counts; preserve production identifiers and behavior.

Tier: glance — merge mode: human /prm; this session stops at PR-open by explicit agreement.

Detailed implementation plan

Affected repository: PyAutoLabs/PyAutoFit. Files: test_autofit/graphical/conftest.py and explicit shared-output call sites if needed.
Override the graphical suite configuration fixture with per-test tmp_path output and balanced config push/pop. Ensure explicit output-directory consumers and teardown use the isolated location. Run graphical tests serially and under xdist with identical collected counts; preserve production identifiers and behavior.
Run focused regression tests and the full PyAutoFit suite; compare failures against main rather than claiming unverified green. No public signature changes planned.

Branch survey: main clean; no existing PyAutoFit claim in current Mind active.md; no open PyAutoFit PRs or matching open issue found. Branch: feature/ep-test-output-isolation. These four separately reviewed tasks are coordinated in one user-authorized group, with isolated worktrees and sequential edits.

Authorization and Heart RED

User request: "Find 4 grouped tasks and do them". After the exact Heart RED reasons were disclosed and development-only authorization requested, the user replied "Yes I authorise". Applies to selecting, implementing and testing these four tasks and opening reviewable PRs; no merge or release. Tests and author diff review will be recorded at PR-open, not claimed in advance.

Published RED reasons:

  • install verification FAILED (testpypi; checks F)
  • release validation FAILED (stage integrate)
  • PyAutoHeart/workflows/Release Integrate: failure
  • manifest_drift/checks/workspace checkouts (manifest ↔ disk): red
  • Release readiness: release validation FAILED (stage integrate)
  • Release readiness: install verification FAILED (testpypi; checks F)
  • PyAutoArray/test_autoarray/inversion/inversion/interferometer/test_interferometer.py::test__interferometer_sparse_operator__mapped_reconstructed_data_dict_matc…
  • Unit-test timing: 1 test regressions (>3× baseline), 11 slow (>1.5×)
  • Release validation: NOT release_ready — v2026.10.25.1.dev83701 profile=release (2026-10-25T07:21:05+00:00)
  • validation_report/stages/integrate: fail
  • verify_install FAILED: verify_install FAILED
  • F: colab gate: finufft (declared by autoarray, absent on Colab, imported)
  • Install verification: FAILED (testpypi; F) (2026-10-25T07:42:48.134987+00:00)
  • verify_install: red
  • PyAutoBrain: red
  • PyAutoLens: red
  • autolens_profiling: red
  • Worktree drift: 1 orphan / 0 missing / 3 dirty

Original prompt

EP test suite is not xdist-safe: tests share one on-disk output directory

Type: bug
Target: PyAutoFit
Repos:

  • PyAutoFit
    Difficulty: small
    Autonomy: safe
    Priority: medium
    Status: formalised
    Consequence: glance
    Witness: 'python -m pytest test_autofit/graphical -q -n auto' passes with the same count as the serial run (279 on main e331e33, 281 after PyAutoFit#1608).
    Review-minutes: 3
    Unattended: ready

EP test suite is not xdist-safe: tests share one on-disk output directory
Type: bug
Target: PyAutoFit
Difficulty: small
Autonomy: safe
Priority: medium
Witness: 'python -m pytest test_autofit/graphical -q -n auto' passes with the same count as the serial run (279 on main e331e33, 281 after PyAutoFit#1608).

Measured 2026-09-11 on clean main e331e33: 'python -m pytest test_autofit/graphical -q -n auto' gives 17 failed / 268 passed, while the same suite serial gives 279 passed. Signature: FileNotFoundError on test_autofit/output//mean_field_evolution.png inside PIL Image.save (e.g. test_autofit/graphical/gaussian/test_optimizer.py::test_default; also hierarchical/test_optimise.py::test_optimise, regression/test_linear_regression.py::test_exact_updates and test_laplace, stochastic/test_regression.py::test_stochastic_linear_regression, gaussian/test_optimizer.py::TestDynesty::test_null_paths). Several EP tests write their visualisation output under the same search-identifier path in test_autofit/output/, so one xdist worker's teardown removes the directory another worker is still writing into. The remote-session doctrine says to run suites with -n auto, so this suite silently cannot follow it and PyAutoFit#1608 had to be judged serially.

Fix: give every EP test that writes output its own tmp_path-rooted output directory (a conftest fixture pointing the conf output path at tmp_path, or a unique_tag per test), so the suite is parallel-safe.

Related: complete/2026/10/prior-stable-test-ids.md covers the other xdist breakage (collection-id mismatch in test_prior_properties), a separate cause.

Implementation and validation — 2026-10-25

PR: #1728

Full library suite: 3913 passed, 1 skipped, 1 xfailed. Graphical suite: 329 passed serially and 329 passed with four xdist workers; unchanged main reproduced 13 failures / 316 passed under xdist. Python 3.12.14. Author diff review passed; production code and output identifier generation untouched.

Published vitals: organism RED / PyAutoFit scope GREEN. Local readiness STALE. Existing development-only authorization applies; no merge or release. Awaiting required CI and human review.

Activity

  1. Jammy2211 commented on Oct 25, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped in #1728, merged after every head CI workflow and job passed. Human Prm authorized merge; the checked head was unchanged. Completion and validation are being recorded in PyAutoMind. Pending release; no release performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions