Skip to content

test: replace Emcee timing gate with JAX dispatch regression #1727

Closed

Description

@Jammy2211

Overview

test: replace Emcee timing gate with JAX dispatch regression

Plan

Test that Emcee call_wrap invokes its cached compiled scalar objective exactly once per evaluation with equivalent values and no retracing for same-shaped inputs. Retain the timing comparison only behind an explicit local profiling environment flag. Run focused JAX dispatch tests and the library suite.

Tier: glance — merge mode: human /prm; this session stops at PR-open by explicit agreement.

Detailed implementation plan

Affected repository: PyAutoLabs/PyAutoFit. Files: test_autofit/non_linear/search/mcmc/test_emcee_jax_timing.py.
Test that Emcee call_wrap invokes its cached compiled scalar objective exactly once per evaluation with equivalent values and no retracing for same-shaped inputs. Retain the timing comparison only behind an explicit local profiling environment flag. Run focused JAX dispatch tests and the library suite.
Run focused regression tests and the full PyAutoFit suite; compare failures against main rather than claiming unverified green. No public signature changes planned.

Branch survey: main clean; no existing PyAutoFit claim in current Mind active.md; no open PyAutoFit PRs or matching open issue found. Branch: feature/emcee-jax-dispatch-guard. These four separately reviewed tasks are coordinated in one user-authorized group, with isolated worktrees and sequential edits.

Authorization and Heart RED

User request: "Find 4 grouped tasks and do them". After the exact Heart RED reasons were disclosed and development-only authorization requested, the user replied "Yes I authorise". Applies to selecting, implementing and testing these four tasks and opening reviewable PRs; no merge or release. Tests and author diff review will be recorded at PR-open, not claimed in advance.

Published RED reasons:

  • install verification FAILED (testpypi; checks F)
  • release validation FAILED (stage integrate)
  • PyAutoHeart/workflows/Release Integrate: failure
  • manifest_drift/checks/workspace checkouts (manifest ↔ disk): red
  • Release readiness: release validation FAILED (stage integrate)
  • Release readiness: install verification FAILED (testpypi; checks F)
  • PyAutoArray/test_autoarray/inversion/inversion/interferometer/test_interferometer.py::test__interferometer_sparse_operator__mapped_reconstructed_data_dict_matc…
  • Unit-test timing: 1 test regressions (>3× baseline), 11 slow (>1.5×)
  • Release validation: NOT release_ready — v2026.10.25.1.dev83701 profile=release (2026-10-25T07:21:05+00:00)
  • validation_report/stages/integrate: fail
  • verify_install FAILED: verify_install FAILED
  • F: colab gate: finufft (declared by autoarray, absent on Colab, imported)
  • Install verification: FAILED (testpypi; F) (2026-10-25T07:42:48.134987+00:00)
  • verify_install: red
  • PyAutoBrain: red
  • PyAutoLens: red
  • autolens_profiling: red
  • Worktree drift: 1 orphan / 0 missing / 3 dirty

Original prompt

@PyAutoFit test_emcee_call_path_runs_at_jitted_speed is a wall-clock ratio assertion that flakes on shared CI runners

Type: test
Target: autofit
Repos:

  • PyAutoFit
    Difficulty: trivial
    Autonomy: safe
    Priority: normal
    Status: formalised
    Consequence: glance
    Witness: the test no longer asserts a wall-clock ratio between the wrapped call path and the raw jitted function on CI; it asserts structurally that the Emcee call path dispatches to the compiled objective (e.g. the wrapper calls the jitted callable exactly once per evaluation with no Python-side re-tracing), with the timing comparison kept only under an opt-in env flag for local profiling.
    Filed: 2026-10-14

test_autofit/non_linear/search/mcmc/test_emcee_jax_timing.py::test_emcee_call_path_runs_at_jitted_speed failed on PyAutoFit main df6d218 (run 38577574188, unittest (3.12) only: "call_wrap 0.116 ms vs jit 0.055 ms") while the same code passed on #1687's PR CI and on the previous main run 5bf057f; gh run rerun --failed on the same run went green. A failure on main turns Heart RED and stops every --auto ship gate (it parked B4b on 2026-10-14). Sub-millisecond ratio assertions cannot be stable on shared runners; replace with a structural assertion.

Implementation and validation — 2026-10-25

PR: #1729

Focused JAX dispatch tests: 3 passed, 1 skipped (opt-in profiling). Full library suite: 3912 passed, 2 skipped, 1 xfailed, with one /tmp/summary.csv collision from concurrently running isolated checkouts (the existing fixture hardcodes that global filename). Re-ran the entire affected aggregate-CSV module alone: 17 passed. No remaining failing tests. Python 3.12.14. Author diff review passed. No production changes.

Published vitals: organism RED / PyAutoFit scope GREEN. Local readiness STALE. Existing development-only authorization applies; no merge or release. Awaiting required CI and human review.

Activity

  1. Jammy2211 commented on Oct 25, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped in #1729, merged after every head CI workflow and job passed. Human Prm authorized merge; the checked head was unchanged. Completion and validation are being recorded in PyAutoMind. Pending release; no release performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions