Skip to content

feat(heart): Colab gate — check F seeds from Google's manifest and audits every lazy import - #228

Merged
Jammy2211 merged 2 commits into
mainfrom
feature/colab-notebook-release-gate
Sep 15, 2026
Merged

Jammy2211 merged 2 commits into
mainfrom
feature/colab-notebook-release-gate

Conversation

@Jammy2211

@Jammy2211 Jammy2211 commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes #227.

Heart check F is the release-time "Colab simulation", and until now it emulated Colab by pip install autolens jax with dependencies. That venv already held every package the real --no-deps bootstrap in autonerves.setup_colab would miss, so the check could never see a missing dependency. On 2026-09-15 corner — a base autofit dependency imported lazily inside corner_cornerpy — was absent on Colab and HowToFit tutorial 5 died mid-workshop after its emcee run; workspace smoke (PYAUTO_TEST_MODE=2, samplers never constructed) and check F both reported green.

This PR makes check F a faithful Colab gate:

  • Ground truth instead of an approximation. The sim venv (python3.12, Colab's interpreter) is seeded from Google's published Colab manifest (googlecolab/backend-info pip-freeze.txt, fetched live → $HEART_STATE_DIR cache → vendored snapshot), installing only the manifest packages the stack's resolved closure needs, completed to a self-consistent subset. Nothing absent from Colab's manifest is ever installed.
  • The real bootstrap. The injected setup cell runs verbatim (pip install autonerves --no-deps → setup_colab.setup("autolens") → workspace clone), unchanged.
  • Prove what the libraries import. After the bootstrap, the gate walks every declared requirement of the installed PyAuto packages, AST-scans them for every third-party import at any depth (recording whether the site is guarded by a try/except ImportError), imports each module for real, and constructs af.Emcee/DynestyStatic/Nautilus/LBFGS. Unguarded import failure → FAIL (readiness RED, stage-3 report fails). Guarded misses, version conflicts and never-imported declared deps → WARN.
  • Scoping decision recorded on the issue: this is rung 2 of the prompt's ladder (~70 s, in the existing release-only verify_install_release lane); running notebooks end to end at real settings (rungs 3/4) tests a different bug class and is declined for this gate.

Witness (both legs run locally against live PyPI, manifest fetched live 2026-09-15)

Leg Setup Result Wall
1 PyPI as-is F FAIL colab gate: blackjax (…/blackjax/nuts/search.py:260); colossus (autolens/lens/los.py:207); corner (autofit/non_linear/plot/samples_plotters.py:95); +6 more 74 s
2 COLAB_GATE_AUTONERVES_SRC = PyAutoNerves feature/colab-bootstrap-lazy-deps (PyAutoNerves#167) F FAIL colab gate: colossus (autolens/lens/los.py:207); hmf (autolens/lens/los.py:147); jax_zero_contour (autogalaxy/operate/lens_calc.py:1452); +3 more 66 s

Leg 1 reproduces the 2026-09-15 failure that every existing gate missed. Leg 2 shows the gate flips on exactly the _SHARED_EXTRAS change: PyAutoNerves#167 clears corner and blackjax and five of seven version conflicts.

Merge consequence and how the holes were closed

The gate found five library-side unguarded lazy imports of packages Colab does not ship. They were resolved on 2026-09-15, in two places:

  • Real Colab breakages → _SHARED_EXTRAS on the PyAutoNerves branch (PyAutoNerves#167 follow-up PR): jax_zero_contour>=2.0.0,<3.0.0 (an autogalaxy base dependency reached by autogalaxy/operate/lens_calc.py:1452,1884, guarded at plot_utils.py:429 but not there) and zeus-mcmc==2.5.4 (af.Zeus, run by autofit_workspace notebooks). Every runtime requirement of both is already shipped by Colab.
  • Not notebook paths → heart/config/colab_gate.yaml accepted_missing, each with a written reason (second commit on this PR): colossus and hmf (autolens/lens/los.py line-of-sight tooling, test/dev extra; hmf is declared in no pyproject at all, a library follow-up), and mcp (autofit/mcp/server.py, the MCP server entry point, unconditional import is correct there).

Leg 3 (this PR + the extended Nerves branch overlaid):

F      PASS    Colab manifest live 2026-09-15; 87 Colab-provided, 11 extras, 39 imports probed

179 s, 0 FAILs, 32 WARNs (the three accepted names with reasons echoed; networkx==3.1 / psutil==6.1.0 autofit pins vs Colab 3.6.1 / 5.9.5; guarded coolest, nufftax; five declared-but-never-imported deps; the 20 closure packages Colab does not ship, all now arriving via extras).

The release-integrate run stays RED until the autonerves carrying PyAutoNerves#167 and its follow-up is on PyPI — check F installs autonerves from the index. That is the gate working; the ordering is: merge this, merge the Nerves PR, release autonerves.

Scripts Changed

  • heart/checks/colab_gate.py (new) — seed / verify gate. seed resolves the with-deps closure via pip install --dry-run --report - without installing it, installs only the part Colab also ships at Colab's pins, then completes that set to a self-consistent Colab subset. verify walks declared requirements, AST-scans every import (guarded = inside a try whose handlers catch ImportError/ModuleNotFoundError/Exception/bare), probes each module for real (a failing module whose distribution Colab ships is installed at Colab's pin and re-probed), constructs the four tutorial searches, and applies the verdict. Stdlib + packaging; pure functions throughout; JSON report.
  • heart/checks/colab_pip_freeze.snapshot.txt (new) — vendored googlecolab/backend-info manifest, fetched 2026-09-15; last fallback behind live fetch → $HEART_STATE_DIR/colab_pip_freeze.txt cache.
  • heart/config/colab_gate.yaml (new) — accepted_missing: exemptions keyed by module name, each with a one-line reason: colossus, hmf, mcp (see above). A shipped-config test asserts exactly this set.
  • heart/checks/verify_install.sh — check F rebuilt: python3.12 venv (missing interpreter → FAIL, the check B/E rule), the with-deps pip install <stack> jax deleted, gate seed before the setup cell and verify between the setup cell and the notebook cell. Setup cell, google.colab stub, exit-3 SKIP and the cosmos_web_ring notebook cell unchanged. New COLAB_GATE_AUTONERVES_SRC dev/witness overlay. Gate JSON nested additively at checks[F].colab_gate in the readiness sidecar; gate temp files swept via ARTEFACTS (honours --keep).
  • tests/test_colab_gate.py (new, 34 tests) + tests/test_verify_install_script.py (+9) — manifest parsing/normalisation, live→cache→snapshot fallback, closure intersection, requirement walk, AST scan, verdict rules incl. accepted_missing and the imported-missing FAIL, and a round-trip of the real sidecar writer through readiness.compute. No network, no venv.
  • skills/verify_install/verify_install.md, docs/release_validation.md, health_agent/capabilities.yaml, bin/pyauto-heart — check F description rewritten (the old row still named autoconf and dataset/imaging/simple), with an explicit statement of what the simulation does not cover: GPU, OS, and the manifest's 1–2 day lag.

Test Plan

  • python3 -m pytest -q -n auto in PyAutoHeart: 1015 passed (baseline 1013 + the new modules' own tests; the two late fixes — seed completion and import heal — brought their own tests)
  • Witness leg 1: check F against PyPI → F|FAIL naming corner (74 s)
  • Witness leg 2: PyAutoNerves#167 branch overlaid → corner, blackjax cleared; five library-side FAILs remain (66 s)
  • Sidecar with nested colab_gate parses through readiness.compute
  • Witness leg 3: this PR + extended Nerves branch → F|PASS (179 s), 1016 tests
  • Human: release autonerves after both PRs merge (check F installs it from the index)

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

…st (#227)

Check F emulated Colab with `pip install autolens jax` WITH dependencies, so
its venv already held every package the real --no-deps bootstrap would miss;
`corner` was absent on Colab on 2026-09-15 and no gate saw it.

- heart/checks/colab_gate.py: `seed` builds a python3.12 venv holding only
  the packages Google's Colab manifest (googlecolab/backend-info) ships that
  the stack's resolved closure needs; `verify` walks declared requirements,
  AST-scans every third-party import in the installed libraries (guarded or
  not), imports each for real, constructs the tutorial searches, and FAILs on
  any unguarded miss.
- verify_install.sh: check F runs seed -> verbatim setup cell -> verify ->
  notebook cell; with-deps stack install removed; COLAB_GATE_AUTONERVES_SRC
  overlay for witness runs; gate JSON nested under checks[F].colab_gate.
- Vendored manifest snapshot, empty heart/config/colab_gate.yaml
  accepted_missing, 43 new tests, check F docs rewritten.

Witness: PyPI as-is -> F|FAIL naming corner (74 s); PyAutoNerves#167
overlaid -> corner/blackjax cleared, five library-side unguarded imports
remain (jax_zero_contour, zeus, colossus, hmf, mcp).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…with Nerves#167 + extras

Three of the five library-side holes are not notebook paths: colossus and hmf
(autolens/lens/los.py line-of-sight tooling, test/dev extra) and mcp
(autofit/mcp/server.py, the MCP server entry point). They are accepted with
written reasons in heart/config/colab_gate.yaml. The other two
(jax_zero_contour, zeus-mcmc) are real Colab breakages and join
_SHARED_EXTRAS on the PyAutoNerves branch. Check F with that branch overlaid
now reports F|PASS.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Jammy2211

Copy link
Copy Markdown
Contributor Author

Leg 3: with the extended PyAutoNerves branch (jax_zero_contour + zeus-mcmc in _SHARED_EXTRAS; colossus/hmf/mcp accepted with reasons in colab_gate.yaml) check F reports F|PASS in 179 s — 87 Colab-provided, 11 extras, 39 imports probed, 0 FAILs / 32 WARNs. 1016 unit tests green.

@Jammy2211

Copy link
Copy Markdown
Contributor Author

Nerves PR: PyAutoLabs/PyAutoNerves#169

@Jammy2211
Jammy2211 merged commit c7659ae into main Sep 15, 2026
2 checks passed
@Jammy2211
Jammy2211 deleted the feature/colab-notebook-release-gate branch September 15, 2026 20:00
@Jammy2211 Jammy2211 removed the pending-release Awaiting a published release label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(heart): Colab gate — prove the --no-deps bootstrap can import what the libraries import

1 participant