diff --git a/.github/AGENTS.md b/.github/AGENTS.md index 902a560..88c1fa9 100644 --- a/.github/AGENTS.md +++ b/.github/AGENTS.md @@ -1,12 +1,15 @@ # CI guidance The workflow installs `uv.lock`, then calls the arms of -`scripts/release_check.py` instead of repeating commands. The `test` matrix job -runs `--arm test-full`; the `gate` job runs lint, types, docs, manuscript, -smoke, and package once on 3.12; the `gnn-conformance` job provisions a Python -3.12 runtime from the GNN revision pinned in `specs/gnn-pin.json` and runs -`--arm gnn`. It is a required check and must not be converted into a skip when -the runtime cannot be provisioned. +`scripts/release_check.py` instead of repeating commands. The `test` matrix +contains only Python 3.11 and runs `--arm test-full`. The `gnn-conformance` job +(display name `gnn conformance (pinned)`) runs `--arm all` +on Python 3.12 with `GNN_RUNTIME_PYTHON` set to the separate runtime provisioned from `specs/gnn-pin.json`. +There is no separate `gate` job. Pinned conformance always runs in the `gnn` +arm; missing runtime dependencies must fail, never skip. PDF rendering remains +an explicit local `--arm render` selection. The workflow uses pinned +`astral-sh/setup-uv` directly, not the Institute composite; see +[`../docs/gate.md`](../docs/gate.md) for the exact action and command map. Manuscript figure bytes are written by the host toolchain, and `uv.lock` resolves different matplotlib versions for different Python versions, so the diff --git a/.github/GNN.md b/.github/GNN.md index de56a18..86aa03e 100644 --- a/.github/GNN.md +++ b/.github/GNN.md @@ -7,11 +7,16 @@ (`8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1`) and records SHA-256 digests for the five parser files under `src/gnn/`. -The job installs that commit into a separate runtime, then runs -`python scripts/validate_gnn_spec.py --gnn-runtime --json` and checks -that the resolved `gnn.__version__` matches the pin tag. The validator verifies -the installed parser file digests, parses `specs/gridworld_3x3.gnn.md` with the -pinned `MarkdownGNNParser`, and compares the declared parameters and the `A`, +The job installs that commit into a separate Python 3.12 runtime, then invokes +the canonical release gate with `--arm all` and `GNN_RUNTIME_PYTHON` set to that runtime. Its `gnn` +arm always performs both the offline drift check and real pinned validation; +a missing runtime fails instead of skipping conformance. See +[`docs/gate.md`](../docs/gate.md) for the complete job and arm map, and +[`docs/gnn.md`](../docs/gnn.md) for standalone validation commands. + +The validator checks that the resolved `gnn.__version__` matches the pin tag, +verifies the installed parser file digests, parses `specs/gridworld_3x3.gnn.md` +with the pinned `MarkdownGNNParser`, and compares the declared parameters and the `A`, `B`, `C`, `D`, and `E` matrices against the runtime model built by `blockference.gnn`; it also requires the pinned semantic validator, the B-orientation check, and the consistency checker to report no errors or diff --git a/.github/README.md b/.github/README.md index cb89b8a..3e239a5 100644 --- a/.github/README.md +++ b/.github/README.md @@ -27,6 +27,7 @@ Choose the shortest path for your goal: | Read the generative-model specification | [`docs/gnn.md`](../docs/gnn.md) and [`specs/gridworld_3x3.gnn.md`](../specs/gridworld_3x3.gnn.md) | | Explore executable examples | [`notebooks/README.md`](../notebooks/README.md) | | Read the publication source | [`docs/manuscript/README.md`](../docs/manuscript/README.md) | +| Trace publication figures to their sources | [`docs/manuscript/figure-registry.json`](../docs/manuscript/figure-registry.json) | | Contribute or report a bug | [`docs/contributing.md`](../docs/contributing.md) | | Report a security issue | [`SECURITY.md`](../SECURITY.md) | @@ -77,32 +78,29 @@ The GitHub Actions workflow runs on pushes and pull requests targeting `main`, using the committed `uv.lock` on Python 3.11 and 3.12. CI calls the arms of the canonical gate in [`scripts/release_check.py`](../scripts/release_check.py): -* the `test` matrix job runs `--arm test-full` (full suite, 90% coverage floor); -* the `gate` job runs `--arm lint --arm types --arm docs --arm manuscript - --arm smoke --arm package` once on Python 3.12; -* the `gnn-conformance` job runs `--arm gnn` with a runtime provisioned from - `specs/gnn-pin.json`. +* the `test` matrix contains only Python 3.11 and runs `--arm test-full`; +* the Python 3.12 `gnn-conformance` job (display name + `gnn conformance (pinned)`) runs `--arm all` with `GNN_RUNTIME_PYTHON` set to a + separate pinned runtime provisioned from `specs/gnn-pin.json`. The arms cover Ruff linting, repository-wide Markdown link/anchor validation, mypy, the test suite, notebook hygiene and execution, real pipeline and CLI -smoke checks, manuscript validation/build/render drift, and a wheel import -smoke. [`docs/gate.md`](../docs/gate.md) lists them and +smoke checks, manuscript validation/build drift, and a wheel import smoke. +The full test suite therefore runs on both Python versions without a separate +`gate` job. PDF rendering remains an explicit local `--arm render` selection. +[`docs/gate.md`](../docs/gate.md) defines the arms, runtime-aware local +commands, and exact pinned `astral-sh/setup-uv` action; this workflow does not +use the Institute setup composite. [`docs/_generated/gate-metrics.md`](../docs/_generated/gate-metrics.md) holds -the measured numbers. - -Run the release gate locally with: - -```bash -uv run python scripts/release_check.py --arm all -``` - -The fast default (`uv run python scripts/release_check.py`) runs lint, types, -the fast test selection, and docs. A separate `gnn conformance (pinned)` job -(job key `gnn-conformance`) installs the GNN revision pinned in -`specs/gnn-pin.json` into its own Python 3.12 runtime and validates -`specs/gridworld_3x3.gnn.md` against the declared runtime model. The gate is -intentionally fail-closed: a required check is not converted into a skip when -an optional backend or generated artefact is unavailable. See +the measured numbers and the statuses of the one-run before/after receipts. +Those receipts are not a claimed speedup; the failed offline package baseline +cache miss remains a failure. + +The no-argument fast default runs lint, types, the fast test selection, and +docs. The release gate's `gnn` arm always validates +`specs/gridworld_3x3.gnn.md` against the declared runtime model using the pinned +parser. The gate is intentionally fail-closed: a missing GNN runtime, required +backend, or generated artifact cannot turn a required check into a skip. See [`docs/development.md`](../docs/development.md) for the development workflow and [`docs/design-contract.md`](../docs/design-contract.md) for the project scope and invariants. diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 2ec1369..145cbfc 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -1,4 +1,20 @@ # workflows -The workflow installs the locked environment, runs Ruff and pytest, and checks -the complete smoke artefact tree through the package CLI. +[`ci.yml`](ci.yml) installs the locked environment and calls the canonical +release gate in two jobs: + +| Job key | Python | Selection | +| --- | --- | --- | +| `test` | matrix: 3.11 only | `--arm test-full` | +| `gnn-conformance` | 3.12 | `--arm all` with `GNN_RUNTIME_PYTHON` set to the provisioned runtime | + +The second job retains the display name `gnn conformance (pinned)` and installs +the revision in [`../../specs/gnn-pin.json`](../../specs/gnn-pin.json) into a +separate runtime. The full suite runs on both Python versions; all other release +arms run with conformance on Python 3.12. Missing pinned-runtime validation +fails the gate. There is no separate `gate` job, and PDF rendering remains an +explicit local arm. + +The workflow uses pinned `astral-sh/setup-uv` directly, not the Institute +composite. [`../../docs/gate.md`](../../docs/gate.md) is authoritative for the +exact action pin, gate commands, and measured-receipt workflow. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7341525..fdba67d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -22,7 +22,7 @@ jobs: strategy: fail-fast: false matrix: - python-version: ["3.11", "3.12"] + python-version: ["3.11"] steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 @@ -52,36 +52,6 @@ jobs: path: coverage.xml if-no-files-found: ignore - gate: - timeout-minutes: 60 - # Lint, types, docs, manuscript, smoke, and package work runs once; the - # test matrix above repeats only the suite across supported Python versions. - name: gate (lint, types, docs, manuscript, smoke, package) - runs-on: ubuntu-latest - - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - name: Install uv - # v10.3.3; setup-uv publishes no floating v10 major tag, so @v10 is unresolvable - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 - with: - version: "0.13.7" - enable-cache: true - cache-dependency-glob: "uv.lock" - - - name: Set up Python 3.12 - run: uv python install 3.12 - - - name: Install locked package and development dependencies - run: uv sync --locked --extra dev --python 3.12 - - - name: Run the release arms - run: >- - uv run python scripts/release_check.py - --arm lint --arm types --arm docs --arm manuscript - --arm smoke --arm package - gnn-conformance: timeout-minutes: 120 # Validates the committed GNN specification with the parser pinned in @@ -106,8 +76,8 @@ jobs: - name: Set up Python 3.12 run: uv python install 3.12 - - name: Install locked package without development extras - run: uv sync --locked --python 3.12 + - name: Install locked package and development dependencies + run: uv sync --locked --extra dev --python 3.12 - name: Provision the pinned GNN parser runtime id: runtime @@ -123,8 +93,7 @@ jobs: "generalized-notation-notation @ git+${REPO}@${COMMIT}" echo "python=$RUNNER_TEMP/gnn-runtime/bin/python" >> "$GITHUB_OUTPUT" - - name: GNN conformance arm - run: >- - uv run python scripts/release_check.py - --arm gnn --gnn-runtime "${{ steps.runtime.outputs.python }}" - --require-gnn + - name: Release arms (including Python 3.12 full tests and GNN conformance) + env: + GNN_RUNTIME_PYTHON: ${{ steps.runtime.outputs.python }} + run: uv run python scripts/release_check.py --arm all diff --git a/README.md b/README.md index 0526d29..f6b6b22 100644 --- a/README.md +++ b/README.md @@ -23,10 +23,14 @@ artefacts, and fail-closed validation. ```bash uv sync --locked --extra dev uv run python scripts/release_check.py # fast arms (default) -uv run python scripts/release_check.py --arm all # release gate +uv run python scripts/release_check.py --arm all --gnn-runtime /path/to/gnn-runtime/bin/python uv run blockference pipeline --config configs/smoke.yml --output-root output ``` +The release gate requires a separate pinned GNN runtime; replace the example +interpreter path above with yours. [`docs/gate.md`](docs/gate.md) defines the +arms and CI mapping, and [`docs/gnn.md`](docs/gnn.md) explains runtime setup. + The CLI has three explicit commands: ```bash @@ -109,7 +113,8 @@ The grid-world generative model is declared in [Generalized Notation Notation](https://github.com/ActiveInferenceInstitute/Generalized_Notation_Notation) (GNN). The document is generated from the implementation and validated against the upstream parser pinned in [`specs/gnn-pin.json`](specs/gnn-pin.json) -(GNN v5.0.3, commit `8f293d9ba`): the validator reads the declared parameters +(schema `gnn-importer-pin/v1`, GNN v5.0.3, commit +`8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1`): the validator reads the declared parameters back, builds the runtime model from them, and requires the declared matrices to match exactly. See [`docs/gnn.md`](docs/gnn.md) for the commands and the conformance contract. diff --git a/TODO.md b/TODO.md index 4f8701b..e60cf1b 100644 --- a/TODO.md +++ b/TODO.md @@ -1,10 +1,10 @@ # ActiveBlockference TODO Owner: Active Inference Institute -Last reviewed: 2026-11-11 +Last reviewed: 2026-11-12 This file records documentation findings from the 2026-08-02 DOCS-DEEP pass and -the 2026-11-11 completeness pass. +the 2026-11-11 completeness pass, plus the subsequent estate alignment. Items are grouped by the size of the documentation change required: * **Minor** — typo, broken link, stale path, or formatting correction. @@ -46,6 +46,15 @@ Items are grouped by the size of the documentation change required: the manuscript and package floor can no longer disagree. The `python_version < '3.11'` `tomli` fallback in `requirements.txt` remains a data-file pin, not a second supported floor. +- [x] Align the documentation hub and flat index with long-lived guides, + generated receipts, and the manuscript figure registry; keep measured + values in generated metrics rather than human-maintained prose. +- [x] Document the fail-closed GNN arm, the canonical full parser pin, the + Python 3.11 test-only/Python 3.12 full-gate CI mapping, and the directly + pinned uv setup action without claiming use of the Institute composite. +- [x] Document receipt imports, including the failed offline package baseline + cache miss, without claiming a speedup; distinguish the declared NumPy + range from the actual lockfile resolution. ## Major @@ -55,13 +64,18 @@ an executable publication validator. ## Open / deferred -- The Python-version claim under **Medium** is open pending an owner decision on - which side to change. All 2026-08-02 docs-deep items remain complete. +No unresolved item remains from the listed documentation findings. The Python +floor reconciliation under **Medium** is complete, not deferred. ## Verification notes -The pass verified the documented single-agent and multi-agent Python examples, -the `docs/api.md` pipeline example, the tutorial snippets, `uv run ruff check -blockference tests scripts GRTs`, manuscript validation/build drift checks, and -the repository-wide relative Markdown links and anchors. Full release verification -remains the canonical `uv run python scripts/release_check.py --arm all` command. +The earlier completeness pass verified the documented single-agent and +multi-agent Python examples, the `docs/api.md` pipeline example, the tutorial +snippets, Ruff, manuscript validation/build drift checks, and repository-wide +relative Markdown links and anchors. These are historical verification notes, +not results from the estate alignment. + +The estate documentation edits were reviewed by reasoning only; integrated +verification belongs to the canonical release selection in +[`docs/gate.md`](docs/gate.md), with a configured pinned GNN runtime. The +generated metrics and receipts record actual runs, including failures. diff --git a/docs/README.md b/docs/README.md index 650b77d..8dde1ea 100644 --- a/docs/README.md +++ b/docs/README.md @@ -13,6 +13,8 @@ Start here if you are new to the repository: Generated facts live under [`_generated/`](_generated/README.md). Measured gate numbers are in [`_generated/gate-metrics.md`](_generated/gate-metrics.md); do not copy those numbers into prose. +The receipts retain failed and non-equivalent workloads, including the offline +package baseline cache miss; they do not establish a speedup. ## Concepts and design @@ -37,12 +39,22 @@ not copy those numbers into prose. schema notes. * [`../notebooks/README.md`](../notebooks/README.md) — executable notebook examples. +* [`../scripts/README.md`](../scripts/README.md) — release and publication tools. +* [`../tests/README.md`](../tests/README.md) — test layout and gate coverage. +* [`../.github/workflows/README.md`](../.github/workflows/README.md) — workflow + entry point; [`gate.md`](gate.md) owns the exact two-job CI map. +* [`../SECURITY.md`](../SECURITY.md) — private security reporting. +* [`../TODO.md`](../TODO.md) — documentation completion record. +* [`../GRTs/README.md`](../GRTs/README.md) — optional local research orchestration. +* [`../.aii/README.md`](../.aii/README.md) — InstituteOS metadata sidecar. ## Publication source -* [`manuscript/`](manuscript/) — the ordered publication source, formal - definitions, numbered equations, figures, bibliography, and reproducible - build commands. +* [`manuscript/README.md`](manuscript/README.md) — ordered publication source, + formal definitions, numbered equations, bibliography, and build commands. +* [`manuscript/SYNTAX.md`](manuscript/SYNTAX.md) — publication syntax and variables. +* [`manuscript/figure-registry.json`](manuscript/figure-registry.json) — figure + sources and artifact mapping. The software verdicts described throughout these documents certify software and artefact integrity only; they do not establish empirical adequacy of a diff --git a/docs/_generated/README.md b/docs/_generated/README.md index afc23b5..6f79a8f 100644 --- a/docs/_generated/README.md +++ b/docs/_generated/README.md @@ -1,14 +1,23 @@ # `docs/_generated/` -Generated documentation. Every file here is written by a named script and -checked by a gate arm; do not edit these files by hand. +Generated facts here are written by named scripts and checked by gate arms; +do not edit them by hand. This README explains the producers. | File | Producer | Check | | --- | --- | --- | | [`gate-metrics.md`](gate-metrics.md) | `scripts/gate_metrics.py` from [`gate-timings.json`](gate-timings.json) and [`gate-baseline.json`](gate-baseline.json) | `scripts/release_check.py --arm docs` | +| [`gate-baseline.json`](gate-baseline.json) | `scripts/gate_metrics.py --record-baseline PATH` imports real pre-change gate receipts | `scripts/gate_metrics.py --check` | +| [`gate-timings.json`](gate-timings.json) | `scripts/gate_metrics.py --record-timings PATH` imports real current gate receipts | `scripts/gate_metrics.py --check` | | [`manuscript-variables.json`](manuscript-variables.json) | `scripts/manuscript_variables.py` | `scripts/release_check.py --arm manuscript` | -`gate-timings.json` is a recorded receipt from one `--timings` run of the gate; -`gate-baseline.json` records the pre-change arm commands and their measured wall -times. Both are evidence for the measured numbers, not claims about another -host. +`release_check.py --timings PATH` records actual gate commands, statuses, and +elapsed times. Repeat `--record-baseline PATH` for disjoint pre-change selections; +`--record-timings PATH` imports one combined current receipt into the canonical +record before generating `gate-metrics.md`. See +[`../gate.md`](../gate.md) for the authoritative recording and import commands. + +The before/after records are one-run, host-specific evidence, not a controlled +benchmark or a claimed speedup. The offline package baseline cache miss is a +failed measurement, not a successful comparison. Missing arms remain unrecorded; +failed and non-equivalent workloads keep their status and qualifications. +Measured values belong in generated metrics only. diff --git a/docs/_generated/gate-baseline.json b/docs/_generated/gate-baseline.json index aa4d6aa..fb037ae 100644 --- a/docs/_generated/gate-baseline.json +++ b/docs/_generated/gate-baseline.json @@ -1,77 +1,184 @@ { - "schema": "activeblockference-gate-baseline/v1", - "measured": "2026-11-12", - "host": "macOS-27.0.1-arm64-arm-64bit", - "python": "3.12.16", - "conditions": "One manual run per command before the composable gate landed, on the same host. Other estate jobs drove the load average to about 100 and all caches were warm, so these are historical evidence, not a controlled benchmark.", "arms": { - "lint": { - "seconds": 0.34, - "comparable": true, - "commands": [ - {"command": "uv run ruff check blockference tests GRTs scripts", "seconds": 0.34} - ] - }, - "types": { - "seconds": 10.35, - "comparable": false, - "note": "The baseline mypy run targeted Python 3.12; the arm now targets the requires-python floor (3.11).", + "docs": { "commands": [ - {"command": "uv run mypy blockference", "seconds": 10.35} - ] + "uv run python scripts/check_docs_links.py", + "uv run python scripts/gate_metrics.py --check" + ], + "seconds": 7.45, + "status": "passed" }, - "test-full": { - "seconds": 88.77, - "comparable": false, - "note": "The baseline was `uv run pytest` without `--cov` or `--cov-fail-under=90`; the arm runs coverage. Recorded result: 84 passed, 1 skipped.", + "gnn": { "commands": [ - {"command": "uv run pytest", "seconds": 88.77} - ] + "uv run python scripts/export_gnn_spec.py --check", + "uv run python scripts/validate_gnn_spec.py --gnn-runtime /private/tmp/gnn-rt-503/bin/python --json" + ], + "seconds": 2.78, + "status": "passed" }, - "docs": { - "seconds": 0.22, - "comparable": false, - "note": "The baseline ran only the link/anchor checker; the arm also verifies the generated metrics document.", + "lint": { "commands": [ - {"command": "uv run python scripts/check_docs_links.py", "seconds": 0.22} - ] + "uv run ruff check blockference tests GRTs scripts" + ], + "seconds": 0.11, + "status": "passed" }, "manuscript": { - "seconds": 18.81, - "comparable": false, - "note": "The baseline ran validation, build, and the build check; the arm also verifies the generated variables file.", "commands": [ - {"command": "uv run python scripts/validate_manuscript.py", "seconds": 3.85}, - {"command": "uv run python scripts/build_manuscript.py", "seconds": 7.24}, - {"command": "uv run python scripts/build_manuscript.py --check", "seconds": 7.72} - ] + "uv run python scripts/manuscript_variables.py --check", + "uv run python scripts/validate_manuscript.py", + "uv run python scripts/build_manuscript.py", + "uv run python scripts/build_manuscript.py --check" + ], + "seconds": 6.04, + "status": "passed" }, - "gnn": { - "seconds": 14.8, - "comparable": true, + "package": { + "commands": [], + "error": "Command '['uv', 'build']' returned non-zero exit status 2.", + "seconds": 0.08, + "status": "failed" + }, + "render": { "commands": [ - {"command": "uv run python scripts/export_gnn_spec.py --check", "seconds": 3.08}, - {"command": "uv run python scripts/validate_gnn_spec.py --json", "seconds": 11.72} - ] + "uv run python scripts/build_manuscript.py --render pdf" + ], + "seconds": 4.04, + "status": "passed" }, "smoke": { - "seconds": 15.42, - "comparable": true, "commands": [ - {"command": "uv run blockference pipeline --config configs/smoke.yml --output-root /output --run-name gate_smoke", "seconds": 7.57}, - {"command": "uv run blockference validation --run-dir /output/gate_smoke", "seconds": 3.66}, - {"command": "uv run blockference run --config configs/smoke.yml --output /run.csv", "seconds": 4.19} - ] + "uv run blockference pipeline --config configs/smoke.yml --output-root /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-gzxa03z4/output --run-name gate_smoke", + "uv run blockference validation --run-dir /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-gzxa03z4/output/gate_smoke", + "uv run blockference run --config configs/smoke.yml --output /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-gzxa03z4/run.csv" + ], + "seconds": 3.48, + "status": "passed" }, - "package": { - "seconds": 17.67, - "comparable": true, + "test": { + "commands": [ + "uv run pytest -m not slow" + ], + "seconds": 33.27, + "status": "passed" + }, + "test-full": { + "commands": [ + "uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90" + ], + "seconds": 55.38, + "status": "passed" + }, + "types": { "commands": [ - {"command": "uv build", "seconds": 4.98}, - {"command": "uv venv --python /.venv/bin/python /venv", "seconds": 0.04}, - {"command": "uv pip install --python /venv/bin/python dist/*.whl", "seconds": 1.01}, - {"command": "/venv/bin/python -c \"from blockference import GridWorld; ...\"", "seconds": 11.64} - ] + "uv run mypy blockference" + ], + "seconds": 0.61, + "status": "passed" + } + }, + "conditions": "One run per arm before this estate change on the same host and interpreter. Warm caches and host load are uncontrolled; failed observations are retained. The render arm was measured separately after the package cache miss.", + "host": "macOS-27.0.1-arm64-arm-64bit", + "measured": "2026-11-12T13:46:28Z", + "python": "3.12.16", + "runs": [ + { + "arms": { + "docs": { + "commands": [ + "uv run python scripts/check_docs_links.py", + "uv run python scripts/gate_metrics.py --check" + ], + "seconds": 7.45, + "status": "passed" + }, + "gnn": { + "commands": [ + "uv run python scripts/export_gnn_spec.py --check", + "uv run python scripts/validate_gnn_spec.py --gnn-runtime /private/tmp/gnn-rt-503/bin/python --json" + ], + "seconds": 2.78, + "status": "passed" + }, + "lint": { + "commands": [ + "uv run ruff check blockference tests GRTs scripts" + ], + "seconds": 0.11, + "status": "passed" + }, + "manuscript": { + "commands": [ + "uv run python scripts/manuscript_variables.py --check", + "uv run python scripts/validate_manuscript.py", + "uv run python scripts/build_manuscript.py", + "uv run python scripts/build_manuscript.py --check" + ], + "seconds": 6.04, + "status": "passed" + }, + "package": { + "commands": [], + "error": "Command '['uv', 'build']' returned non-zero exit status 2.", + "seconds": 0.08, + "status": "failed" + }, + "smoke": { + "commands": [ + "uv run blockference pipeline --config configs/smoke.yml --output-root /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-gzxa03z4/output --run-name gate_smoke", + "uv run blockference validation --run-dir /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-gzxa03z4/output/gate_smoke", + "uv run blockference run --config configs/smoke.yml --output /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-gzxa03z4/run.csv" + ], + "seconds": 3.48, + "status": "passed" + }, + "test": { + "commands": [ + "uv run pytest -m not slow" + ], + "seconds": 33.27, + "status": "passed" + }, + "test-full": { + "commands": [ + "uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90" + ], + "seconds": 55.38, + "status": "passed" + }, + "types": { + "commands": [ + "uv run mypy blockference" + ], + "seconds": 0.61, + "status": "passed" + } + }, + "generated": "2026-11-12T13:46:28Z", + "host": "macOS-27.0.1-arm64-arm-64bit", + "python": "3.12.16", + "schema": "activeblockference-gate/v1", + "tests": { + "coverage_percent": 90.94, + "passed": 85, + "skipped": 0 + } + }, + { + "arms": { + "render": { + "commands": [ + "uv run python scripts/build_manuscript.py --render pdf" + ], + "seconds": 4.04, + "status": "passed" + } + }, + "generated": "2026-11-12T13:48:41Z", + "host": "macOS-27.0.1-arm64-arm-64bit", + "python": "3.12.16", + "schema": "activeblockference-gate/v1" } - } + ], + "schema": "activeblockference-gate-baseline/v1" } diff --git a/docs/_generated/gate-metrics.md b/docs/_generated/gate-metrics.md index b7f15ae..e128c9f 100644 --- a/docs/_generated/gate-metrics.md +++ b/docs/_generated/gate-metrics.md @@ -2,41 +2,40 @@ -Measured numbers from one recorded run of the canonical gate on `macOS-27.0.1-arm64-arm-64bit` with Python 3.12.16 at `2026-11-12T13:20:11Z`. +Measured wall times from recorded canonical gate observations on `macOS-27.0.1-arm64-arm-64bit` with Python 3.12.16 at `2026-11-12T13:56:34Z`. + +First successful measurements retained for each arm. Package initially failed twice because offline pandas wheel metadata was unavailable; only package and the not-yet-run render arm were then run after SHA-256-verified acquisition of the exact pandas wheel in uv.lock. Those failures remain in this receipt’s runs. UV_FIND_LINKS pointed to that verified local wheelhouse for the successful package run. Baseline and after test selections differ by the new regression cases. No speedup is claimed. Refresh with: ```bash uv run python scripts/release_check.py --arm all --arm test --arm render \ - --timings docs/_generated/gate-timings.json -uv run python scripts/gate_metrics.py + --timings /tmp/gate-after.json +uv run python scripts/gate_metrics.py --record-timings /tmp/gate-after.json ``` ## Before and after -`n/a` means the arm or selection did not exist before the composable gate. - -| Arm | Before (s) | After (s) | After status | After command | -| --- | --- | --- | --- | --- | -| `lint` | 0.34 | 0.05 | passed | `uv run ruff check blockference tests GRTs scripts` | -| `types` | 10.35 | 0.28 | passed | `uv run mypy blockference` | -| `test` | n/a | 15.18 | passed | `uv run pytest -m not slow` | -| `test-full` | 88.77 | 77.03 | passed | `uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90` | -| `docs` | 0.22 | 10.25 | passed | `uv run python scripts/check_docs_links.py` | -| `manuscript` | 18.81 | 7.73 | passed | `uv run python scripts/manuscript_variables.py --check` | -| `gnn` | 14.8 | 5.74 | passed | `uv run python scripts/export_gnn_spec.py --check` | -| `smoke` | 15.42 | 6.1 | passed | `uv run blockference pipeline --config configs/smoke.yml --output-root /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-1k9b70t9/output --run-name gate_smoke` | -| `package` | 17.67 | 12.87 | passed | `uv build` | -| `render` | n/a | 11.65 | passed | `uv run python scripts/build_manuscript.py --render pdf` | +`n/a` means no baseline observation was recorded. Failed baselines are not comparable to passing runs. No speedup is inferred from these single samples. + +| Arm | Before (s) | Before status | After (s) | After status | After command | +| --- | --- | --- | --- | --- | --- | +| `lint` | 0.11 | passed | 0.08 | passed | `uv run ruff check blockference tests GRTs scripts` | +| `types` | 0.61 | passed | 0.3 | passed | `uv run mypy blockference` | +| `test` | 33.27 | passed | 4.82 | passed | `uv run pytest -m not slow` | +| `test-full` | 55.38 | passed | 31.17 | passed | `uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90` | +| `docs` | 7.45 | passed | 3.56 | passed | `uv run python scripts/check_docs_links.py` | +| `manuscript` | 6.04 | passed | 3.24 | passed | `uv run python scripts/manuscript_variables.py --check` | +| `gnn` | 2.78 | passed | 1.51 | passed | `uv run python scripts/export_gnn_spec.py --check` | +| `smoke` | 3.48 | passed | 2.55 | passed | `uv run blockference pipeline --config configs/smoke.yml --output-root /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-vx7lf7ds/output --run-name gate_smoke` | +| `package` | 0.08 | failed | 14.86 | passed | `uv build` | +| `render` | 4.04 | passed | 6.18 | passed | `uv run python scripts/build_manuscript.py --render pdf` | Baseline workloads that are not equivalent to their arms: -- `types`: The baseline mypy run targeted Python 3.12; the arm now targets the requires-python floor (3.11). -- `test-full`: The baseline was `uv run pytest` without `--cov` or `--cov-fail-under=90`; the arm runs coverage. Recorded result: 84 passed, 1 skipped. -- `docs`: The baseline ran only the link/anchor checker; the arm also verifies the generated metrics document. -- `manuscript`: The baseline ran validation, build, and the build check; the arm also verifies the generated variables file. +- `package` failed: Command '['uv', 'build']' returned non-zero exit status 2. -Baseline conditions: One manual run per command before the composable gate landed, on the same host. Other estate jobs drove the load average to about 100 and all caches were warm, so these are historical evidence, not a controlled benchmark. +Baseline conditions: One run per arm before this estate change on the same host and interpreter. Warm caches and host load are uncontrolled; failed observations are retained. The render arm was measured separately after the package cache miss. The baseline and current receipts are historical evidence. Timings include host load and warm caches; they are not a controlled benchmark. @@ -44,14 +43,14 @@ The baseline and current receipts are historical evidence. Timings include host | Selection | Collected tests | | --- | --- | -| full suite | 85 | -| fast (`-m "not slow"`) | 72 | +| full suite | 92 | +| fast (`-m "not slow"`) | 79 | | slow (`-m slow`) | 13 | -Recorded run: 85 passed, 0 skipped. +Recorded run: 92 passed, 0 skipped. `test-full` recorded 90.94% line coverage of `blockference` (floor 90%). ## Receipts -The current numbers come from [`gate-timings.json`](gate-timings.json), written by `scripts/release_check.py --timings`; the before numbers come from [`gate-baseline.json`](gate-baseline.json), recorded once before the composable gate landed. An arm absent from a receipt is reported as not recorded, never as passing. +The current numbers come from [`gate-timings.json`](gate-timings.json), written by `scripts/release_check.py --timings`; the before numbers come from [`gate-baseline.json`](gate-baseline.json), imported from actual runs with `scripts/gate_metrics.py --record-baseline ` (repeat for disjoint arms). An arm absent from a receipt is reported as not recorded, never as passing. diff --git a/docs/_generated/gate-timings.json b/docs/_generated/gate-timings.json index 970375c..b7ab436 100644 --- a/docs/_generated/gate-timings.json +++ b/docs/_generated/gate-timings.json @@ -5,22 +5,22 @@ "uv run python scripts/check_docs_links.py", "uv run python scripts/gate_metrics.py --check" ], - "seconds": 10.25, + "seconds": 3.56, "status": "passed" }, "gnn": { "commands": [ "uv run python scripts/export_gnn_spec.py --check", - "uv run python scripts/validate_gnn_spec.py --json" + "uv run python scripts/validate_gnn_spec.py --gnn-runtime /private/tmp/gnn-rt-503/bin/python --json" ], - "seconds": 5.74, + "seconds": 1.51, "status": "passed" }, "lint": { "commands": [ "uv run ruff check blockference tests GRTs scripts" ], - "seconds": 0.05, + "seconds": 0.08, "status": "passed" }, "manuscript": { @@ -30,64 +30,191 @@ "uv run python scripts/build_manuscript.py", "uv run python scripts/build_manuscript.py --check" ], - "seconds": 7.73, + "seconds": 3.24, "status": "passed" }, "package": { "commands": [ "uv build", - "uv venv --python /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/.venv/bin/python /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-f6ywi5t7/venv", - "uv pip install --python /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-f6ywi5t7/venv/bin/python /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/dist/active_blockference-1.0.0-py3-none-any.whl", - "/var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-f6ywi5t7/venv/bin/python -c from blockference import GridWorld; assert GridWorld(2, {'agent': (0, 0)}).current_state == {'agent': (0, 0)}" + "uv export --locked --no-dev --no-emit-project --no-hashes --output-file /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/constraints.txt", + "uv venv --python /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/.venv/bin/python /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/venv", + "uv pip install --python /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/venv/bin/python --constraint /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/constraints.txt /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/dist/active_blockference-1.0.0-py3-none-any.whl", + "/private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/venv/bin/python -I -c from blockference import GridWorld; assert GridWorld(2, {'agent': (0, 0)}).current_state == {'agent': (0, 0)}" ], - "seconds": 12.87, + "seconds": 14.86, "status": "passed" }, "render": { "commands": [ "uv run python scripts/build_manuscript.py --render pdf" ], - "seconds": 11.65, + "seconds": 6.18, "status": "passed" }, "smoke": { "commands": [ - "uv run blockference pipeline --config configs/smoke.yml --output-root /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-1k9b70t9/output --run-name gate_smoke", - "uv run blockference validation --run-dir /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-1k9b70t9/output/gate_smoke", - "uv run blockference run --config configs/smoke.yml --output /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-1k9b70t9/run.csv" + "uv run blockference pipeline --config configs/smoke.yml --output-root /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-vx7lf7ds/output --run-name gate_smoke", + "uv run blockference validation --run-dir /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-vx7lf7ds/output/gate_smoke", + "uv run blockference run --config configs/smoke.yml --output /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-vx7lf7ds/run.csv" ], - "seconds": 6.1, + "seconds": 2.55, "status": "passed" }, "test": { "commands": [ "uv run pytest -m not slow" ], - "seconds": 15.18, + "seconds": 4.82, "status": "passed" }, "test-full": { "commands": [ "uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90" ], - "seconds": 77.03, + "seconds": 31.17, "status": "passed" }, "types": { "commands": [ "uv run mypy blockference" ], - "seconds": 0.28, + "seconds": 0.3, "status": "passed" } }, - "generated": "2026-11-12T13:20:11Z", + "completed": "2026-11-12T14:02:13Z", + "conditions": "First successful measurements retained for each arm. Package initially failed twice because offline pandas wheel metadata was unavailable; only package and the not-yet-run render arm were then run after SHA-256-verified acquisition of the exact pandas wheel in uv.lock. Those failures remain in this receipt\u2019s runs. UV_FIND_LINKS pointed to that verified local wheelhouse for the successful package run. Baseline and after test selections differ by the new regression cases. No speedup is claimed.", + "generated": "2026-11-12T13:56:34Z", "host": "macOS-27.0.1-arm64-arm-64bit", "python": "3.12.16", + "runs": [ + { + "arms": { + "docs": { + "commands": [ + "uv run python scripts/check_docs_links.py", + "uv run python scripts/gate_metrics.py --check" + ], + "seconds": 3.56, + "status": "passed" + }, + "gnn": { + "commands": [ + "uv run python scripts/export_gnn_spec.py --check", + "uv run python scripts/validate_gnn_spec.py --gnn-runtime /private/tmp/gnn-rt-503/bin/python --json" + ], + "seconds": 1.51, + "status": "passed" + }, + "lint": { + "commands": [ + "uv run ruff check blockference tests GRTs scripts" + ], + "seconds": 0.08, + "status": "passed" + }, + "manuscript": { + "commands": [ + "uv run python scripts/manuscript_variables.py --check", + "uv run python scripts/validate_manuscript.py", + "uv run python scripts/build_manuscript.py", + "uv run python scripts/build_manuscript.py --check" + ], + "seconds": 3.24, + "status": "passed" + }, + "package": { + "commands": [], + "error": "Command '['uv', 'pip', 'install', '--python', '/private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-cppzgeqk/venv/bin/python', '--constraint', '/private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-cppzgeqk/constraints.txt', '/Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/dist/active_blockference-1.0.0-py3-none-any.whl']' returned non-zero exit status 1.", + "seconds": 1.67, + "status": "failed" + }, + "smoke": { + "commands": [ + "uv run blockference pipeline --config configs/smoke.yml --output-root /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-vx7lf7ds/output --run-name gate_smoke", + "uv run blockference validation --run-dir /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-vx7lf7ds/output/gate_smoke", + "uv run blockference run --config configs/smoke.yml --output /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-gate-vx7lf7ds/run.csv" + ], + "seconds": 2.55, + "status": "passed" + }, + "test": { + "commands": [ + "uv run pytest -m not slow" + ], + "seconds": 4.82, + "status": "passed" + }, + "test-full": { + "commands": [ + "uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90" + ], + "seconds": 31.17, + "status": "passed" + }, + "types": { + "commands": [ + "uv run mypy blockference" + ], + "seconds": 0.3, + "status": "passed" + } + }, + "generated": "2026-11-12T13:56:34Z", + "host": "macOS-27.0.1-arm64-arm-64bit", + "python": "3.12.16", + "schema": "activeblockference-gate/v1", + "tests": { + "coverage_percent": 90.94, + "passed": 92, + "skipped": 0 + } + }, + { + "arms": { + "package": { + "commands": [], + "error": "Command '['uv', 'pip', 'install', '--python', '/private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-uftvgkxi/venv/bin/python', '--constraint', '/private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-uftvgkxi/constraints.txt', '/Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/dist/active_blockference-1.0.0-py3-none-any.whl']' returned non-zero exit status 1.", + "seconds": 1.81, + "status": "failed" + } + }, + "generated": "2026-11-12T13:59:29Z", + "host": "macOS-27.0.1-arm64-arm-64bit", + "python": "3.12.16", + "schema": "activeblockference-gate/v1" + }, + { + "arms": { + "package": { + "commands": [ + "uv build", + "uv export --locked --no-dev --no-emit-project --no-hashes --output-file /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/constraints.txt", + "uv venv --python /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/.venv/bin/python /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/venv", + "uv pip install --python /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/venv/bin/python --constraint /private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/constraints.txt /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/dist/active_blockference-1.0.0-py3-none-any.whl", + "/private/tmp/activeblockference-estate.8EbtJ4/tmp/activeblockference-package-kki5eqz6/venv/bin/python -I -c from blockference import GridWorld; assert GridWorld(2, {'agent': (0, 0)}).current_state == {'agent': (0, 0)}" + ], + "seconds": 14.86, + "status": "passed" + }, + "render": { + "commands": [ + "uv run python scripts/build_manuscript.py --render pdf" + ], + "seconds": 6.18, + "status": "passed" + } + }, + "generated": "2026-11-12T14:02:13Z", + "host": "macOS-27.0.1-arm64-arm-64bit", + "python": "3.12.16", + "schema": "activeblockference-gate/v1" + } + ], "schema": "activeblockference-gate/v1", "tests": { "coverage_percent": 90.94, - "passed": 85, + "passed": 92, "skipped": 0 } } diff --git a/docs/_generated/manuscript-variables.json b/docs/_generated/manuscript-variables.json index ca3b579..c5c5fb8 100644 --- a/docs/_generated/manuscript-variables.json +++ b/docs/_generated/manuscript-variables.json @@ -1,4 +1,13 @@ { + "experiment_affordance_count": 5, + "experiment_affordances": "`UP`, `DOWN`, `LEFT`, `RIGHT`, `STAY`", + "experiment_agents": 2, + "experiment_config_path": "configs/manuscript.yml", + "experiment_dimension": 3, + "experiment_engine": "radcad", + "experiment_planning_length": 2, + "experiment_seed": 17, + "experiment_timesteps": 6, "gate_entry_point": "scripts/release_check.py", "gnn_commit": "8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1", "gnn_job_key": "gnn-conformance", @@ -11,6 +20,15 @@ "python_requires": "3.11+", "schema": "manuscript-variables/v1", "sources": { + "experiment_affordance_count": "configs/manuscript.yml len(grid.affordances)", + "experiment_affordances": "configs/manuscript.yml grid.affordances", + "experiment_agents": "configs/manuscript.yml simulation.n_agents", + "experiment_config_path": "docs/manuscript/config.yaml experiment.config_path", + "experiment_dimension": "configs/manuscript.yml grid.dimension", + "experiment_engine": "configs/manuscript.yml engine", + "experiment_planning_length": "configs/manuscript.yml grid.planning_length", + "experiment_seed": "configs/manuscript.yml seed", + "experiment_timesteps": "configs/manuscript.yml simulation.timesteps", "gate_entry_point": "scripts/release_check.py", "gnn_commit": "specs/gnn-pin.json commit", "gnn_job_key": ".github/workflows/ci.yml GNN job key", diff --git a/docs/contributing.md b/docs/contributing.md index 7f10c08..eb8ce9a 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -8,8 +8,9 @@ contributions from researchers, engineers, and AI agents alike. 1. Open or comment on an issue describing what you want to do. 2. Fork → branch → code → test → PR. -3. Run `uv run python scripts/release_check.py --arm all` before pushing; the - no-argument default is the fast selection. +3. Run the full release selection described in [`gate.md`](gate.md) before + pushing, with a pinned GNN runtime configured; the no-argument default is + only the fast selection. 4. Keep PRs scoped: one logical change per PR. ## Detailed flow @@ -53,6 +54,14 @@ If you change a public surface, update: * `docs/api.md`. * `docs/architecture.md` if you change layering. * The folder-local `README.md` table. +* [`documentation-index.md`](documentation-index.md) when adding a long-lived guide. +* [`manuscript/figure-registry.json`](manuscript/figure-registry.json) when changing + a publication figure's source or artifact mapping. + +Gate timings and counts belong only in generated metrics. Import real +before receipts with repeatable `--record-baseline` and one combined after receipt +with `--record-timings`, as documented in [`gate.md`](gate.md); preserve failed +or missing measurements rather than reporting a speedup. ### 6. Open the PR diff --git a/docs/development.md b/docs/development.md index 7481ac4..494964b 100644 --- a/docs/development.md +++ b/docs/development.md @@ -5,7 +5,7 @@ Use the committed lockfile for the exact environment: ```bash uv sync --locked --extra dev uv run python scripts/release_check.py # fast arms (default) -uv run python scripts/release_check.py --arm all # release gate +uv run python scripts/release_check.py --arm all --gnn-runtime /path/to/gnn-runtime/bin/python ``` [`gate.md`](gate.md) lists every arm and CI job; `--list` prints the exact @@ -21,10 +21,12 @@ the `slow` marker and are excluded from `--arm test`. The recorded before/after arm times, including the render arm, are in [`_generated/gate-metrics.md`](_generated/gate-metrics.md). -The pinned GNN parser conformance check (`scripts/validate_gnn_spec.py`) is -part of the `gnn` arm only when `GNN_ROOT` or `GNN_RUNTIME_PYTHON` is set, -because it needs an external runtime; CI runs the same arm as its own required -job. See [`gnn.md`](gnn.md). +The `gnn` arm always runs pinned parser conformance +(`scripts/validate_gnn_spec.py`); it fails without the external runtime rather +than skipping validation. Replace the runtime path above with an installed +pinned interpreter, or configure the source-checkout route in [`gnn.md`](gnn.md). +CI's `gnn-conformance` job runs `all` on Python 3.12 with that separate runtime; +the Python 3.11 `test` job runs `test-full`. There is no separate `gate` job. Markdown links and anchors across all tracked documentation are validated by `scripts/check_docs_links.py`, which is also part of the canonical release @@ -35,11 +37,11 @@ Python kernel. The runtime package includes both radCAD and cadCAD so backend integration tests run rather than being omitted. Dependency bounds are kept to what the code and its runtime requirements -actually need. The `numpy` bound `>=1.25,<1.26.4` mirrors the requirement of -`radcad` 0.14.0, which is the current release of the simulation backend: a -`numpy` 2.x resolution fails with "no solution found" while `radcad` is a -required dependency, so the bound is a required pin rather than a habit. -Remove it only when `radcad` widens its own requirement. +actually need. `pyproject.toml` declares `numpy>=1.25,<2.2.7`; the committed +`uv.lock` currently selects NumPy `1.26.3` with radCAD `0.14.0`. The declared +range does not itself exclude all NumPy 2.x versions, and the locked resolution +must not be confused with that range. Pinned GNN requires NumPy 2.x, so use its +separate runtime rather than changing the library's locked environment. Ruff uses a 100-column line length and import sorting. New public functions need docstrings and tests. Do not commit generated output trees, caches, or @@ -47,8 +49,11 @@ virtual environments; `uv.lock` is intentionally tracked. The manuscript source is under `docs/manuscript/`. Its ordered Markdown files, `config.yaml`, `references.bib`, and `figures/` are the publication record. -`scripts/build_manuscript.py` composes a numbered Markdown manuscript and can -be passed to Pandoc for PDF or HTML rendering; `docs/_build/` is disposable. +[`figure-registry.json`](manuscript/figure-registry.json) records the figure +sources and artifacts. `scripts/build_manuscript.py` composes a numbered +Markdown manuscript; its authoritative PDF command is +`uv run python scripts/build_manuscript.py --render pdf`, also selected by the +explicit local gate arm `--arm render`. `docs/_build/` is disposable. Figure SVG and PNG output is written by the host toolchain, so its bytes legitimately differ between the maintainer host and CI: matplotlib changes diff --git a/docs/documentation-index.md b/docs/documentation-index.md index 5108db7..8e2c1b1 100644 --- a/docs/documentation-index.md +++ b/docs/documentation-index.md @@ -9,6 +9,10 @@ Flat index of the repository's long-lived documents. Generated facts live under - [`README.md`](README.md) — documentation hub and reading order. - [`../.github/README.md`](../.github/README.md) — GitHub-facing signpost. - [`contributing.md`](contributing.md) — contribution flow. +- [`../AGENTS.md`](../AGENTS.md) — repository contributor guidance; folder-local `AGENTS.md` files refine it. +- [`../SECURITY.md`](../SECURITY.md) — private security reporting. +- [`../LICENSE`](../LICENSE) — license terms. +- [`../TODO.md`](../TODO.md) — documentation findings and completion record. ## Concepts and contracts @@ -25,6 +29,8 @@ Flat index of the repository's long-lived documents. Generated facts live under - [`gnn.md`](gnn.md) — GNN declaration, pinned parser validation, and commands. - [`../specs/README.md`](../specs/README.md) — GNN document and pin record. - [`../.github/GNN.md`](../.github/GNN.md) — what the conformance job parses. +- [`../specs/gridworld_3x3.gnn.md`](../specs/gridworld_3x3.gnn.md) — canonical generated model declaration. +- [`../specs/gnn-pin.json`](../specs/gnn-pin.json) — authoritative upstream parser pin and digests. ## Development and operations @@ -32,8 +38,14 @@ Flat index of the repository's long-lived documents. Generated facts live under - [`gate.md`](gate.md) — gate arms, CI job map, and generated metrics. - [`../scripts/README.md`](../scripts/README.md) — gate and generator scripts. - [`../tests/README.md`](../tests/README.md) — test layout and gate equivalence. +- [`../.github/workflows/README.md`](../.github/workflows/README.md) — workflow entry point. +- [`../tests/envs/README.md`](../tests/envs/README.md) — environment test conventions. +- [`../tests/utils/README.md`](../tests/utils/README.md) — utility test conventions. +- [`_generated/README.md`](_generated/README.md) — generated-document producers and receipt imports. - [`_generated/gate-metrics.md`](_generated/gate-metrics.md) — measured arm runtimes and test counts (generated). - [`_generated/manuscript-variables.json`](_generated/manuscript-variables.json) — declared manuscript variables (generated). +- [`_generated/gate-baseline.json`](_generated/gate-baseline.json) — one-run historical baseline receipt, including failures. +- [`_generated/gate-timings.json`](_generated/gate-timings.json) — recorded current gate receipts. ## Tutorials and examples @@ -42,9 +54,45 @@ Flat index of the repository's long-lived documents. Generated facts live under - [`tutorials/02_multi_agent_dict.md`](tutorials/02_multi_agent_dict.md) - [`tutorials/03_cadcad_pipeline.md`](tutorials/03_cadcad_pipeline.md) - [`../notebooks/README.md`](../notebooks/README.md) — executable notebooks. +- [`../notebooks/simple_gridworld/README.md`](../notebooks/simple_gridworld/README.md) — grid-world notebook guide. +- [`../notebooks/ants/README.md`](../notebooks/ants/README.md) — multi-agent notebook guide. - [`../configs/README.md`](../configs/README.md) — configuration examples. +- [`../GRTs/README.md`](../GRTs/README.md) — explicit local and optional provider-based research. +- [`../GRTs/data3/README.md`](../GRTs/data3/README.md) — research workspace contents. ## Publication source - [`manuscript/README.md`](manuscript/README.md) — ordered manuscript source, build, and render. - [`manuscript/SYNTAX.md`](manuscript/SYNTAX.md) — labels, cross-references, citations, variables. +- [`manuscript/preamble.md`](manuscript/preamble.md) — renderer expectations. +- [`manuscript/00_abstract.md`](manuscript/00_abstract.md) — abstract. +- [`manuscript/01_introduction.md`](manuscript/01_introduction.md) — introduction. +- [`manuscript/02_formal_model.md`](manuscript/02_formal_model.md) — formal model. +- [`manuscript/03_deterministic_dynamics.md`](manuscript/03_deterministic_dynamics.md) — deterministic dynamics. +- [`manuscript/04_implementation.md`](manuscript/04_implementation.md) — implementation architecture. +- [`manuscript/05_validation.md`](manuscript/05_validation.md) — validation. +- [`manuscript/06_results.md`](manuscript/06_results.md) — results. +- [`manuscript/07_related_work.md`](manuscript/07_related_work.md) — related work. +- [`manuscript/08_limitations.md`](manuscript/08_limitations.md) — limitations. +- [`manuscript/09_reproducibility.md`](manuscript/09_reproducibility.md) — reproducibility. +- [`manuscript/99_references.md`](manuscript/99_references.md) — references heading. +- [`manuscript/config.yaml`](manuscript/config.yaml) — publication metadata and experiment configuration. +- [`manuscript/references.bib`](manuscript/references.bib) — citation database. +- [`manuscript/figure-registry.json`](manuscript/figure-registry.json) — figure sources and artifact mapping. + +## Package guides + +- [`../blockference/README.md`](../blockference/README.md) — package layout. +- [`../blockference/config/README.md`](../blockference/config/README.md) — configuration helpers. +- [`../blockference/envs/README.md`](../blockference/envs/README.md) — environments. +- [`../blockference/io/README.md`](../blockference/io/README.md) — persistence. +- [`../blockference/simulations/README.md`](../blockference/simulations/README.md) — backend adapters. +- [`../blockference/utils/README.md`](../blockference/utils/README.md) — utility helpers. +- [`../blockference/viz/README.md`](../blockference/viz/README.md) — visualization. + +## InstituteOS metadata and historical review + +- [`../.aii/README.md`](../.aii/README.md) — metadata sidecar. +- [`../.aii/docs/README.md`](../.aii/docs/README.md) — sidecar document index. +- [`../.aii/docs/INTEGRATION.md`](../.aii/docs/INTEGRATION.md) — federation and metadata operations. +- [`../REVIEW_LOG_2026-08-02.md`](../REVIEW_LOG_2026-08-02.md) — historical review record, not current gate evidence. diff --git a/docs/gate.md b/docs/gate.md index a83790d..1ac3dd5 100644 --- a/docs/gate.md +++ b/docs/gate.md @@ -7,15 +7,22 @@ second copy of any command. ```bash uv run python scripts/release_check.py # fast arms (default) -uv run python scripts/release_check.py --arm all # release gate +uv run python scripts/release_check.py --arm all --gnn-runtime /path/to/gnn-runtime/bin/python uv run python scripts/release_check.py --list # arms and their commands ``` +Run `uv sync --locked --extra dev` before the gate. Its child commands set +`UV_NO_SYNC=1` to reuse that prepared project environment instead of repeating +dependency synchronization. + Measured wall times and test counts live in the generated [`_generated/gate-metrics.md`](_generated/gate-metrics.md), which shows the recorded before/after arm times and labels baseline workloads that are not -equivalent to their arms. Do not hand-edit those numbers: refresh them with the -commands the generated page records. +equivalent to their arms. These are one-run, host-specific receipts, not a +controlled benchmark or a claimed speedup. The offline package baseline failed +because its required package was absent from the local cache; preserve that +failure rather than comparing it as a successful workload. Do not hand-edit +measurements. ## Arms @@ -27,10 +34,10 @@ commands the generated page records. | `test-full` | `uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90` | yes | | `docs` | `scripts/check_docs_links.py`, `scripts/gate_metrics.py --check` | yes | | `manuscript` | `scripts/manuscript_variables.py --check`, `scripts/validate_manuscript.py`, `scripts/build_manuscript.py`, `scripts/build_manuscript.py --check` | yes | -| `gnn` | `scripts/export_gnn_spec.py --check`, pinned `scripts/validate_gnn_spec.py` when a runtime is configured | yes | +| `gnn` | `scripts/export_gnn_spec.py --check` and real pinned `scripts/validate_gnn_spec.py` validation; missing runtime fails | yes | | `smoke` | pipeline, validation, and standalone `run` JSON smoke checks in a temporary directory | yes | -| `package` | `uv build` and a clean virtualenv import of the wheel | yes | -| `render` | Pandoc + XeLaTeX PDF render of the manuscript | no | +| `package` | `uv build`, locked `uv export` constraints for clean-wheel dependencies, and isolated import outside the source tree | yes | +| `render` | `scripts/build_manuscript.py --render pdf` (Pandoc + XeLaTeX) | no | `--arm` is repeatable and arms always run in the canonical order above. `fast` is the default because it is the selection used while editing; `all` is the @@ -51,10 +58,11 @@ job parses. The conformance job's key is `gnn-conformance` and its display name is `gnn conformance (pinned)`. -The `gnn` arm always runs the offline drift check. Pinned-parser conformance -runs when `GNN_ROOT` or `GNN_RUNTIME_PYTHON` is set, or when `--gnn-runtime` is -passed; otherwise the arm reports the run as not run, never as a pass. CI -provisions a runtime and passes `--require-gnn`. +The `gnn` arm always runs both the offline drift check and real pinned-parser +conformance. Select an installed pinned runtime with `--gnn-runtime` or +`GNN_RUNTIME_PYTHON`, or supply a pinned source checkout via `GNN_ROOT` as +described in [`gnn.md`](gnn.md). Missing or invalid runtime dependencies fail the +arm and therefore `all`; conformance is never silently skipped. ## GitHub Actions @@ -62,15 +70,41 @@ provisions a runtime and passes `--require-gnn`. | Job | Arms | | --- | --- | -| `test` (matrix 3.11, 3.12) | `--arm test-full` | -| `gate` (3.12) | `--arm lint --arm types --arm docs --arm manuscript --arm smoke --arm package` | -| `gnn-conformance` | `--arm gnn --gnn-runtime --require-gnn` | +| `test` (matrix: Python 3.11 only) | `--arm test-full` | +| `gnn-conformance` (Python 3.12; display name `gnn conformance (pinned)`) | `--arm all`, with `GNN_RUNTIME_PYTHON` set to the provisioned runtime | + +The full suite therefore runs on both supported CI Python versions. The +remaining release arms run in `gnn-conformance` on Python 3.12, alongside the +real pinned conformance check; there is no separate `gate` job. The pinned GNN +runtime remains separate from the locked library environment because their +NumPy requirements differ. PDF rendering is an explicit local `--arm render` +selection, not part of `all` or CI. -The matrix job therefore repeats only the test suite across supported Python -versions; lint, types, docs, manuscript, smoke, and package work run once. The -workflow installs uv with +The workflow installs uv directly with `astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2` (`v10.3.3`, pinned by commit because setup-uv publishes no floating `v10` tag). It does not -use the composite action +use the Institute composite action `ActiveInferenceInstitute/.github/.github/actions/setup-python-uv@7159e67315330aac8e414aebae0ad1e999a41448`; the workflow installs the locked environment directly. + +## Recording measurements + +`release_check.py --timings PATH` writes the actual commands, statuses, and +elapsed times for the selected run, including a failed arm. Use a fresh receipt +path for each run; selecting `render` separately records the explicit local PDF +build without adding it to CI. + +Import receipts before regenerating the checked metrics page: + +```bash +uv run python scripts/gate_metrics.py \ + --record-baseline /path/to/before-receipt.json \ + --record-baseline /path/to/before-render-receipt.json \ + --record-timings /path/to/after-receipt.json +uv run python scripts/gate_metrics.py --check +``` + +Replace these paths with receipts produced by real gate runs. `--record-baseline` +is repeatable for disjoint selections; `--record-timings` accepts one combined +`--arm all --arm test --arm render` receipt. Do not synthesize timings or turn +failed, missing, or non-equivalent measurements into passes. diff --git a/docs/gnn.md b/docs/gnn.md index 7634c17..f64d505 100644 --- a/docs/gnn.md +++ b/docs/gnn.md @@ -47,11 +47,14 @@ works. Exit codes are `0` pass, `1` conformance failure, and `2` pinned runtime or checkout unavailable — an unavailable run is reported as not run, never as a pass. -CI runs the same validation in the `gnn-conformance` job (display name -`gnn conformance (pinned)`), provisioning a separate Python 3.12 runtime from -the pinned revision because the GNN distribution requires `numpy>=2` while this -package resolves `numpy` 1.x. Locally, `scripts/release_check.py --arm gnn` -runs it when `GNN_ROOT` or `GNN_RUNTIME_PYTHON` is set. +CI's `gnn-conformance` job (display name `gnn conformance (pinned)`) runs the +full `all` selection on Python 3.12, provisioning a separate pinned runtime +because GNN requires NumPy 2.x while the library's committed lock resolves +NumPy 1.x. Locally, the `gnn` arm always runs both specification drift and real +parser conformance, using `--gnn-runtime`, `GNN_RUNTIME_PYTHON`, or `GNN_ROOT`. +An unavailable runtime fails the arm and `all`; no opt-in requirement flag or +skip mode is needed. See [`gate.md`](gate.md) for the canonical gate commands +and exact CI map. ## What the validation checks diff --git a/docs/manuscript/06_results.md b/docs/manuscript/06_results.md index 9160b5f..3a43c68 100644 --- a/docs/manuscript/06_results.md +++ b/docs/manuscript/06_results.md @@ -3,11 +3,14 @@ ## Seeded reference run The reference figure is generated from the same package used by the pipeline. -It configures a $3\times3$ grid, two agents, a planning horizon of two, six -simulation timesteps, the five default affordances, and seed 17. No trajectory -values are transcribed by hand: `scripts/build_manuscript.py` invokes -`run_experiment`, extracts the returned coordinates and EFE vectors, and saves -the figure after closing the Matplotlib figure in a `finally` block. +It reads `{{experiment_config_path}}`, configuring a +${{experiment_dimension}}\times{{experiment_dimension}}$ grid, +{{experiment_agents}} agents, a planning horizon of {{experiment_planning_length}}, +{{experiment_timesteps}} simulation timesteps, {{experiment_affordance_count}} +configured affordances, and seed {{experiment_seed}}. No trajectory values are +transcribed by hand: `scripts/build_manuscript.py` invokes `run_experiment`, +extracts the returned coordinates and EFE vectors, and saves the figure after +closing the Matplotlib figure in a `finally` block. ![Seeded trajectories and persisted minimum policy EFE.](figures/fig3_seeded_run.png){#fig:seeded-run width=95%} @@ -18,13 +21,13 @@ the actual diagnostic object, not a proxy derived from action entropy. | Parameter | Reference value | |---|---| -| Backend | radCAD; cadCAD uses the same state-update contract | -| Grid | $3\times3$ | -| Agents | 2 | -| Planning length | 2 | -| Timesteps | 6 | -| Seed | 17 | -| Affordances | `UP`, `DOWN`, `LEFT`, `RIGHT`, `STAY` | +| Backend | `{{experiment_engine}}`; both supported engines use the same state-update contract | +| Grid | ${{experiment_dimension}}\times{{experiment_dimension}}$ | +| Agents | {{experiment_agents}} | +| Planning length | {{experiment_planning_length}} | +| Timesteps | {{experiment_timesteps}} | +| Seed | {{experiment_seed}} | +| Affordances | {{experiment_affordances}} | : Reference configuration used for the reproducible manuscript figure. {#tbl:reference-run} diff --git a/docs/manuscript/09_reproducibility.md b/docs/manuscript/09_reproducibility.md index e156d3b..bcf7d12 100644 --- a/docs/manuscript/09_reproducibility.md +++ b/docs/manuscript/09_reproducibility.md @@ -5,11 +5,16 @@ uses `uv sync --locked --extra dev` for development. The release procedure is: ```bash uv sync --locked --extra dev +GNN_RUNTIME_PYTHON="${GNN_RUNTIME_PYTHON:?Set the external pinned GNN runtime Python}" \ uv run python {{gate_entry_point}} --arm all ``` -The gate has separately runnable arms (`--list` shows them); `--arm all` is the -release selection and the fast default is used while editing. +The release selection requires a separately configured external GNN runtime at +the pinned revision, named by `GNN_RUNTIME_PYTHON` (or a pinned checkout named by +`GNN_ROOT`); see [`../gnn.md`](../gnn.md) for setup. The `gnn` arm always performs +real pinned validation and fails when that runtime is unavailable. The gate has +separately runnable arms (`--list` shows them); `--arm all` is the release +selection and the fast default is used while editing. The generative-model declaration is checked against the pinned upstream parser: `{{gnn_pin_path}}` records schema `{{gnn_pin_schema}}`, tag `{{gnn_tag}}`, and @@ -31,12 +36,15 @@ be distinct or otherwise explicit. ## Provenance of the figures -Figures 1, 2, and 3 are structural diagrams generated by the manuscript -builder. Figure 4 is computed from a seeded `run_experiment` call with the -configuration recorded in [@tbl:reference-run]. The figure generator imports -the public package API, uses the same action vocabulary and simulation -semantics as the release pipeline, and checks the generated figures in the -manuscript validation step. +The loop in [@fig:loop], collision diagram in [@fig:collision], and pipeline +diagram in [@fig:release-contract] are structural diagrams generated by the +manuscript builder. The seeded-run figure [@fig:seeded-run] is computed from a +seeded `run_experiment` call with the configuration recorded in +[@tbl:reference-run]. The figure generator imports the public package API and +uses the same action vocabulary and simulation semantics as the release +pipeline. `figure-registry.json` declares each source label and its SVG/PNG +companions; source validation and both build/check paths reject registry, +reference, or generated-filename drift. ## Software citation diff --git a/docs/manuscript/AGENTS.md b/docs/manuscript/AGENTS.md index e0886ea..8539397 100644 --- a/docs/manuscript/AGENTS.md +++ b/docs/manuscript/AGENTS.md @@ -16,9 +16,12 @@ Use the following conventions: 6. Cite scholarship with `[@BibTeXKey]` and add the key to `references.bib`. 7. Generate figures from code and close every figure on success and failure. 8. Do not hardcode a generated result when it can be derived by the builder. + Declare each figure's label, ordered source, and SVG/PNG companions in + `figure-registry.json`; do not leave undeclared figures in `figures/`. 9. Use `{{token}}` placeholders from `docs/_generated/manuscript-variables.json` - for pins, job names, versions, and the Python floor; never type those values - into prose. Regenerate with `scripts/manuscript_variables.py`. + for pins, job names, versions, the Python floor, and reference-run parameters; + never type those values into prose. Regenerate with + `scripts/manuscript_variables.py` after changing their source configuration. Required checks are: diff --git a/docs/manuscript/README.md b/docs/manuscript/README.md index 25c5c94..1249d27 100644 --- a/docs/manuscript/README.md +++ b/docs/manuscript/README.md @@ -12,6 +12,9 @@ the ordered manuscript convention used by the Institute template code project: - `preamble.md` documents renderer expectations and the render command. - `figures/` contains the publication figures generated by `scripts/build_manuscript.py`. +- `figure-registry.json` declares each figure label, ordered source, and SVG/PNG + filenames; validation and the builder reject missing, orphaned, or mismatched + references and assets. - `../_generated/manuscript-variables.json` (generated) declares the values injected into `{{token}}` placeholders. @@ -29,8 +32,9 @@ uv run python scripts/build_manuscript.py --render pdf The builder injects declared `{{token}}` variables from `docs/_generated/manuscript-variables.json`; that file is generated from -`pyproject.toml`, `specs/gnn-pin.json`, and `.github/workflows/ci.yml`, so a pin, -job name, or Python floor is never typed into prose. The composed Markdown and +`pyproject.toml`, `specs/gnn-pin.json`, `.github/workflows/ci.yml`, and the +executable configuration selected by `config.yaml`. Pins, job names, the Python +floor, and reference-run parameters are not typed into prose. The composed Markdown and the rendered PDF/HTML live under `docs/_build/`, which is reproducible and excluded from version control. Change the ordered source, then rerun the builder and validation commands. @@ -47,3 +51,8 @@ structurally by `--check` (well-formed SVG, positive viewBox, version-free Dublin Core title, text drawn as paths, plus the PNG canvas checks) and their bytes are excluded from the release gate's generated-drift diff. Every other generated artifact stays byte-compared. + +The figure registry checks names and source references independently of these +toolchain-dependent bytes. Each declaration must have SVG and PNG companions, +match the ordered source's image label and filename, and be used by that source. +Generated and committed filename sets must both match the registry. diff --git a/docs/manuscript/SYNTAX.md b/docs/manuscript/SYNTAX.md index 65ed4a2..1f97b32 100644 --- a/docs/manuscript/SYNTAX.md +++ b/docs/manuscript/SYNTAX.md @@ -31,7 +31,9 @@ their type and number in source order and resolves a reference such as ``` Figure and table numbers are also assigned by source order. The image path is -relative to this directory and must exist when validation runs. +relative to this directory. Declare the figure label, ordered source, and its +SVG/PNG companion filenames in `figure-registry.json`; both companions must +exist, and no undeclared SVG or PNG may remain in `figures/`. ## Variables @@ -39,10 +41,11 @@ Write `{{token}}` where a value must come from the generated variables file (`docs/_generated/manuscript-variables.json`), for example `Python {{python_requires}}` or the pinned GNN commit. The file is produced by `scripts/manuscript_variables.py` from `pyproject.toml`, `specs/gnn-pin.json`, -and `.github/workflows/ci.yml`. Do not type a pin, job name, or supported Python -version into prose; add a key to the generator instead. The builder fails on an -undeclared token, and validation reports tokens the variables file does not -declare. +`.github/workflows/ci.yml`, and the experiment configuration selected by +`config.yaml`. Do not type pins, job names, supported Python versions, or +reference-run parameters into prose; add a key to the generator instead. The +builder fails on an undeclared token, and validation reports tokens the +variables file does not declare. ## Citations diff --git a/docs/manuscript/figure-registry.json b/docs/manuscript/figure-registry.json new file mode 100644 index 0000000..7e70281 --- /dev/null +++ b/docs/manuscript/figure-registry.json @@ -0,0 +1,29 @@ +{ + "schema": "manuscript-figures/v1", + "figures": [ + { + "label": "fig:loop", + "source": "01_introduction.md", + "svg": "fig1_active_inference_loop.svg", + "png": "fig1_active_inference_loop.png" + }, + { + "label": "fig:collision", + "source": "03_deterministic_dynamics.md", + "svg": "fig2_collision_resolution.svg", + "png": "fig2_collision_resolution.png" + }, + { + "label": "fig:seeded-run", + "source": "06_results.md", + "svg": "fig3_seeded_run.svg", + "png": "fig3_seeded_run.png" + }, + { + "label": "fig:release-contract", + "source": "05_validation.md", + "svg": "fig4_release_contract.svg", + "png": "fig4_release_contract.png" + } + ] +} diff --git a/pyproject.toml b/pyproject.toml index 8cf1ca7..17676a8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -32,8 +32,8 @@ classifiers = [ dependencies = [ "cadCAD>=0.4.28", "radcad>=0.12", - # radcad 0.14.0 requires numpy>=1.25,<2.2.7; numpy 2.x is unresolvable - # while radcad is a required runtime dependency (verified 2026-11-01). + # Preserve the simulation backend's compatible range; uv.lock selects the + # concrete version used by the project environment. "numpy>=1.25,<2.2.7", "pandas>=2.0", "matplotlib>=3.7", diff --git a/scripts/README.md b/scripts/README.md index 1729c20..ad543eb 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -5,7 +5,7 @@ Repository gates and generators, all run from the repository root: | Script | Purpose | |---|---| | `release_check.py` | The canonical gate: composable arms for lint, types, fast/full tests, docs, manuscript, GNN, smoke, package, and render. | -| `gate_metrics.py` | Writes `docs/_generated/gate-metrics.md` from a recorded gate receipt; `--check` gates drift. | +| `gate_metrics.py` | Imports disjoint baseline receipts with repeatable `--record-baseline PATH` and one current receipt with `--record-timings PATH`, writes `docs/_generated/gate-metrics.md`, and checks drift with `--check`. | | `manuscript_variables.py` | Derives `docs/_generated/manuscript-variables.json` from `pyproject.toml`, the GNN pin, and CI; `--check` gates drift. | | `check_docs_links.py` | Validates internal Markdown links and anchors across tracked documentation. | | `build_manuscript.py` | Builds the manuscript and its figures, injects declared variables, and renders with Pandoc (`--render pdf\|html`). | @@ -13,9 +13,18 @@ Repository gates and generators, all run from the repository root: | `execute_notebooks.py` | Parses and executes every tracked notebook from the repository root. | | `validate_notebooks.py` | Notebook hygiene checks before execution. | | `export_gnn_spec.py` | Writes or verifies `specs/gridworld_3x3.gnn.md` from the implementation. | -| `validate_gnn_spec.py` | Validates the committed specification with the pinned parser. | +| `validate_gnn_spec.py` | Validates the committed specification with the pinned parser, using a separate interpreter selected by `--gnn-runtime` or the documented environment variables. | | `gnn_parser_probe.py` | The only module that imports the upstream `gnn` package; runs as a subprocess under the pin. | -Run `uv run python scripts/release_check.py --arm all` before publishing; the -no-argument default is the fast selection. See -[`../docs/gate.md`](../docs/gate.md) for the arm table and measured numbers. +Run the release selection with a pinned GNN runtime configured before +publishing; the no-argument default is the fast selection. The `gnn` arm always +runs real pinned conformance and fails if its runtime is unavailable. +[`../docs/gate.md`](../docs/gate.md) owns the arm commands, CI mapping, +`--timings PATH` recording and metrics-import workflow. + +The gate reuses the already-synced locked project environment by setting +`UV_NO_SYNC=1` for its commands. Its package check constrains clean-wheel +dependencies using a locked `uv export` and imports the installed wheel from +outside the source checkout. PDF rendering remains an explicit local +`--arm render` selection of `build_manuscript.py --render pdf`; it is not part +of `all` or CI. diff --git a/scripts/build_manuscript.py b/scripts/build_manuscript.py index 2d23e23..da88d59 100644 --- a/scripts/build_manuscript.py +++ b/scripts/build_manuscript.py @@ -20,6 +20,11 @@ from collections import defaultdict from pathlib import Path +if __package__: + from scripts.validate_manuscript import figure_registry_errors +else: + from validate_manuscript import figure_registry_errors + REPO_ROOT = Path(__file__).resolve().parents[1] os.environ.setdefault( "MPLCONFIGDIR", str(Path(os.getenv("TMPDIR", "/tmp")) / "activeblockference-mpl") @@ -190,8 +195,8 @@ def _manuscript_variables() -> dict[str, str]: """Return the declared manuscript variables as substitution text. The values are produced by ``scripts/manuscript_variables.py`` from - ``pyproject.toml``, the GNN pin, and the CI workflow, so prose never carries - a hand-copied pin, job name, or Python floor. + ``pyproject.toml``, the executable experiment configuration, the GNN pin, + and the CI workflow rather than hand-copied prose values. """ if not VARIABLES_PATH.is_file(): @@ -514,6 +519,9 @@ def as_mapping(value): path = output_dir / "fig4_release_contract.svg" _save_figure(fig, path) paths.append(path) + errors = figure_registry_errors(MANUSCRIPT_DIR, output_dir) + if errors: + raise ValueError("; ".join(errors)) return paths @@ -707,31 +715,26 @@ def main(argv: list[str] | None = None) -> int: if args.check and args.render != "none": parser.error("--check cannot be combined with --render") try: + registry_errors = figure_registry_errors(MANUSCRIPT_DIR, check_files=False) + if registry_errors: + raise ValueError("; ".join(registry_errors)) compiled, labels = compose_manuscript() if not args.check: if not args.no_figures: build_figures() + else: + registry_errors = figure_registry_errors(MANUSCRIPT_DIR, FIGURE_DIR) + if registry_errors: + raise ValueError("; ".join(registry_errors)) output = _write_outputs(compiled) print(f"wrote {output}") if args.render != "none": rendered = render_manuscript(output, args.render, args.output) print(f"rendered {rendered}") else: - missing = [ - str(path) - for path in ( - FIGURE_DIR / f"fig{i}_{name}.svg" - for i, name in ( - (1, "active_inference_loop"), - (2, "collision_resolution"), - (3, "seeded_run"), - (4, "release_contract"), - ) - ) - if not path.is_file() - ] - if missing: - raise FileNotFoundError("missing manuscript figures: " + ", ".join(missing)) + registry_errors = figure_registry_errors(MANUSCRIPT_DIR, FIGURE_DIR) + if registry_errors: + raise ValueError("; ".join(registry_errors)) if not (BUILD_DIR / "manuscript.md").is_file(): raise FileNotFoundError( f"missing composed manuscript: {BUILD_DIR / 'manuscript.md'}" diff --git a/scripts/gate_metrics.py b/scripts/gate_metrics.py index 95506d4..a756381 100644 --- a/scripts/gate_metrics.py +++ b/scripts/gate_metrics.py @@ -3,8 +3,8 @@ The numbers in ``docs/_generated/gate-metrics.md`` come from recorded receipts: * ``gate-timings.json`` -- one run of the canonical gate (``--timings``); -* ``gate-baseline.json`` -- the pre-change arm commands and their measured wall - times, recorded once when the composable gate landed. +* ``gate-baseline.json`` -- pre-change receipts imported with + ``--record-baseline`` (repeat for disjoint arm selections). The producer never invents a runtime: an arm missing from a receipt is reported as not recorded. Test counts are derived from the current suite with @@ -13,14 +13,15 @@ Refresh after changing the gate or the test selection:: uv run python scripts/release_check.py --arm all --arm test --arm render \ - --timings docs/_generated/gate-timings.json - uv run python scripts/gate_metrics.py + --timings /tmp/gate-after.json + uv run python scripts/gate_metrics.py --record-timings /tmp/gate-after.json """ from __future__ import annotations import argparse import json +import shutil import subprocess import sys from pathlib import Path @@ -85,6 +86,48 @@ def _baseline() -> dict[str, object]: return payload +def _load_run(path: Path) -> dict[str, object]: + """Read an actual gate run, retaining both passing and failing observations.""" + + payload = json.loads(path.read_text(encoding="utf-8")) + if payload.get("schema") != SCHEMA or not isinstance(payload.get("arms"), dict): + raise ValueError(f"not a gate run receipt: {path}") + for arm, recorded in payload["arms"].items(): + if arm not in ARM_ORDER or recorded.get("status") not in {"passed", "failed"}: + raise ValueError(f"invalid arm observation in {path}: {arm}") + if not isinstance(recorded.get("seconds"), (int, float)) or recorded["seconds"] < 0: + raise ValueError(f"invalid arm wall time in {path}: {arm}") + return payload + + +def _record_baseline(paths: list[Path]) -> None: + """Combine disjoint measured selections without changing their observations.""" + + runs = [_load_run(path) for path in paths] + first = runs[0] + arms: dict[str, object] = {} + for run in runs: + if (run.get("host"), run.get("python")) != (first.get("host"), first.get("python")): + raise ValueError("baseline selections must share a host and interpreter") + if arms.keys() & run["arms"].keys(): + raise ValueError("baseline selections overlap; retain one observation per arm") + arms.update(run["arms"]) + baseline = { + "schema": BASELINE_SCHEMA, + "measured": first["generated"], + "host": first["host"], + "python": first["python"], + "conditions": ( + "One run per arm before this estate change on the same host and interpreter. " + "Warm caches and host load are uncontrolled; failed observations are retained. " + "The render arm was measured separately after the package cache miss." + ), + "runs": runs, + "arms": arms, + } + BASELINE.write_text(json.dumps(baseline, indent=2, sort_keys=True) + "\n", encoding="utf-8") + + def render(payload: dict[str, object], baseline: dict[str, object]) -> str: """Render the metrics document from the recorded receipts.""" @@ -96,41 +139,50 @@ def render(payload: dict[str, object], baseline: dict[str, object]) -> str: "", "", "", - "Measured numbers from one recorded run of the canonical gate on " + "Measured wall times from recorded canonical gate observations on " f"`{payload.get('host')}` with Python {payload.get('python')} at " f"`{payload.get('generated')}`.", "", + str(payload.get("conditions", "One observation per arm; host load and caches are uncontrolled.")), + "", "Refresh with:", "", "```bash", "uv run python scripts/release_check.py --arm all --arm test --arm render \\", - " --timings docs/_generated/gate-timings.json", - "uv run python scripts/gate_metrics.py", + " --timings /tmp/gate-after.json", + "uv run python scripts/gate_metrics.py --record-timings /tmp/gate-after.json", "```", "", "## Before and after", "", - "`n/a` means the arm or selection did not exist before the composable gate.", + "`n/a` means no baseline observation was recorded. Failed baselines are not " + "comparable to passing runs. No speedup is inferred from these single samples.", "", - "| Arm | Before (s) | After (s) | After status | After command |", - "| --- | --- | --- | --- | --- |", + "| Arm | Before (s) | Before status | After (s) | After status | After command |", + "| --- | --- | --- | --- | --- | --- |", ] for arm in ARM_ORDER: recorded = before.get(arm) or {} current = arms.get(arm) or {} before_seconds = recorded.get("seconds", "n/a") + before_status = recorded.get("status", "not recorded") after_seconds = current.get("seconds", "not recorded") status = current.get("status", "—") commands = current.get("commands") or [] command = f"`{commands[0]}`" if commands else "—" lines.append( - f"| `{arm}` | {before_seconds} | {after_seconds} | {status} | {command} |" + f"| `{arm}` | {before_seconds} | {before_status} | {after_seconds} | {status} | {command} |" ) notes = [ f"- `{arm}`: {recorded['note']}" for arm, recorded in before.items() if recorded.get("note") and not recorded.get("comparable", True) ] + notes.extend( + f"- `{arm}` failed: {recorded['error']}" + for arm, recorded in before.items() + if recorded.get("status") == "failed" and recorded.get("error") + ) if notes: lines += [ "", @@ -171,9 +223,9 @@ def render(payload: dict[str, object], baseline: dict[str, object]) -> str: "", "The current numbers come from [`gate-timings.json`](gate-timings.json), " "written by `scripts/release_check.py --timings`; the before numbers come " - "from [`gate-baseline.json`](gate-baseline.json), recorded once before the " - "composable gate landed. An arm absent from a receipt is reported as not " - "recorded, never as passing.", + "from [`gate-baseline.json`](gate-baseline.json), imported from actual runs with " + "`scripts/gate_metrics.py --record-baseline ` (repeat for disjoint arms). " + "An arm absent from a receipt is reported as not recorded, never as passing.", "", ] return "\n".join(lines) @@ -184,8 +236,22 @@ def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--check", action="store_true", help="fail when the document is stale") + parser.add_argument( + "--record-baseline", type=Path, action="append", default=[], + help="import a pre-change gate receipt; repeat for disjoint arm selections", + ) + parser.add_argument("--record-timings", type=Path, help="import a post-change gate receipt") args = parser.parse_args(argv) + if args.check and (args.record_baseline or args.record_timings): + parser.error("--check cannot import receipts") try: + OUTPUT.parent.mkdir(parents=True, exist_ok=True) + if args.record_baseline: + _record_baseline(args.record_baseline) + if args.record_timings: + _load_run(args.record_timings) + if args.record_timings.resolve() != RECEIPT.resolve(): + shutil.copyfile(args.record_timings, RECEIPT) content = render(_receipt(), _baseline()) except (FileNotFoundError, ValueError, subprocess.CalledProcessError) as exc: print(f"gate metrics failed: {exc}", file=sys.stderr) diff --git a/scripts/manuscript_variables.py b/scripts/manuscript_variables.py index e5ad46d..1c7831a 100644 --- a/scripts/manuscript_variables.py +++ b/scripts/manuscript_variables.py @@ -2,8 +2,8 @@ Manuscript prose uses ``{{token}}`` placeholders; the declared values come from this generated file rather than from hand-copied literals. The values are read -from ``pyproject.toml`` (Python floor and package version), the canonical GNN -pin, and the CI workflow (the conformance job key and display name). +from ``pyproject.toml`` (Python floor and package version), the executable +manuscript configuration, the canonical GNN pin, and the CI workflow. """ from __future__ import annotations @@ -21,6 +21,7 @@ PIN_PATH = ROOT / "specs" / "gnn-pin.json" WORKFLOW_PATH = ROOT / ".github" / "workflows" / "ci.yml" SCHEMA = "manuscript-variables/v1" +MANUSCRIPT_CONFIG_PATH = ROOT / "docs" / "manuscript" / "config.yaml" def _project() -> dict[str, object]: @@ -49,6 +50,31 @@ def build() -> dict[str, object]: project = _project() pin = json.loads(PIN_PATH.read_text(encoding="utf-8")) job_key, job_name = _gnn_job() + metadata = yaml.safe_load(MANUSCRIPT_CONFIG_PATH.read_text(encoding="utf-8")) + config_path = metadata["experiment"]["config_path"] + config = yaml.safe_load((ROOT / config_path).read_text(encoding="utf-8")) + experiment_values = { + "experiment_config_path": config_path, + "experiment_engine": config["engine"], + "experiment_dimension": config["grid"]["dimension"], + "experiment_agents": config["simulation"]["n_agents"], + "experiment_planning_length": config["grid"]["planning_length"], + "experiment_timesteps": config["simulation"]["timesteps"], + "experiment_seed": config["seed"], + "experiment_affordance_count": len(config["grid"]["affordances"]), + "experiment_affordances": ", ".join(f"`{value}`" for value in config["grid"]["affordances"]), + } + experiment_sources = { + "experiment_config_path": "docs/manuscript/config.yaml experiment.config_path", + "experiment_engine": f"{config_path} engine", + "experiment_dimension": f"{config_path} grid.dimension", + "experiment_agents": f"{config_path} simulation.n_agents", + "experiment_planning_length": f"{config_path} grid.planning_length", + "experiment_timesteps": f"{config_path} simulation.timesteps", + "experiment_seed": f"{config_path} seed", + "experiment_affordance_count": f"{config_path} len(grid.affordances)", + "experiment_affordances": f"{config_path} grid.affordances", + } return { "schema": SCHEMA, "python_requires": _python_requires(str(project["requires-python"])), @@ -61,6 +87,7 @@ def build() -> dict[str, object]: "gnn_commit": str(pin["commit"]), "gnn_job_key": job_key, "gnn_job_name": job_name, + **experiment_values, "sources": { "python_requires": "pyproject.toml [project].requires-python", "package_version": "pyproject.toml [project].version", @@ -72,6 +99,7 @@ def build() -> dict[str, object]: "gnn_commit": "specs/gnn-pin.json commit", "gnn_job_key": ".github/workflows/ci.yml GNN job key", "gnn_job_name": ".github/workflows/ci.yml GNN job name", + **experiment_sources, }, } diff --git a/scripts/release_check.py b/scripts/release_check.py index dd26633..5bdf138 100644 --- a/scripts/release_check.py +++ b/scripts/release_check.py @@ -21,8 +21,8 @@ * ``all`` -- every release arm (``test`` and ``render`` are excluded). ``fast`` is the default; ``all`` is the release gate. Refresh the measured -numbers with ``--arm all --arm test --timings docs/_generated/gate-timings.json`` -and then ``uv run python scripts/gate_metrics.py``. +numbers with ``--arm all --arm test --arm render --timings /tmp/gate-after.json`` +and then ``uv run python scripts/gate_metrics.py --record-timings /tmp/gate-after.json``. """ from __future__ import annotations @@ -76,7 +76,7 @@ def _run( command: list[str], *, cwd: Path = ROOT, capture: bool = False ) -> subprocess.CompletedProcess[str]: - """Run one gate command with a consistent environment.""" + """Run one gate command without re-synchronizing the prepared locked environment.""" print("$", " ".join(command), flush=True) return subprocess.run( @@ -85,7 +85,8 @@ def _run( check=True, text=True, capture_output=capture, - env={**os.environ, "PYTHONUNBUFFERED": "1"}, + env={**os.environ, "PYTHONUNBUFFERED": "1", "UV_NO_SYNC": "1"}, + stdin=subprocess.DEVNULL, ) @@ -103,7 +104,8 @@ def _changed_paths() -> set[str]: """ result = subprocess.run( - ["git", "diff", "--name-only"], cwd=ROOT, check=True, text=True, capture_output=True + ["git", "diff", "--name-only"], cwd=ROOT, check=True, text=True, + capture_output=True, stdin=subprocess.DEVNULL, ) return { line @@ -129,7 +131,7 @@ def _assert_json(command: list[str]) -> None: def _plan( - arm: str, *, workdir: Path, gnn_runtime: str | None, gnn_available: bool + arm: str, *, workdir: Path, gnn_runtime: str | None ) -> list[list[str]] | None: """Return the commands for one arm, or ``None`` for a Python-driven arm.""" @@ -165,11 +167,10 @@ def _plan( ] if arm == "gnn": commands = [["uv", "run", "python", "scripts/export_gnn_spec.py", "--check"]] - if gnn_available: - validator = ["uv", "run", "python", "scripts/validate_gnn_spec.py", "--json"] - if gnn_runtime: - validator[-1:-1] = ["--gnn-runtime", gnn_runtime] - commands.append(validator) + validator = ["uv", "run", "python", "scripts/validate_gnn_spec.py", "--json"] + if gnn_runtime: + validator[-1:-1] = ["--gnn-runtime", gnn_runtime] + commands.append(validator) return commands if arm == "smoke": output_root = workdir / "output" @@ -214,21 +215,35 @@ def _package_smoke() -> list[list[str]]: if not wheels: raise FileNotFoundError("uv build produced no wheel") with tempfile.TemporaryDirectory(prefix="activeblockference-package-") as directory: + constraints = Path(directory) / "constraints.txt" + commands.append( + [ + "uv", "export", "--locked", "--no-dev", "--no-emit-project", "--no-hashes", + "--output-file", str(constraints), + ] + ) + _run(commands[-1], capture=True) venv = Path(directory) / "venv" commands.append(["uv", "venv", "--python", sys.executable, str(venv)]) _run(commands[-1]) python = venv / ("Scripts/python.exe" if os.name == "nt" else "bin/python") - commands.append(["uv", "pip", "install", "--python", str(python), str(wheels[-1])]) + commands.append( + [ + "uv", "pip", "install", "--python", str(python), + "--constraint", str(constraints), str(wheels[-1]), + ] + ) _run(commands[-1]) commands.append( [ str(python), + "-I", "-c", "from blockference import GridWorld; " "assert GridWorld(2, {'agent': (0, 0)}).current_state == {'agent': (0, 0)}", ] ) - _run(commands[-1]) + _run(commands[-1], cwd=Path(directory)) return commands @@ -283,16 +298,10 @@ def main(argv: list[str] | None = None) -> int: parser.add_argument("--timings", type=Path, help="write a JSON runtime receipt") parser.add_argument("--json", action="store_true", help="print a JSON summary") parser.add_argument("--gnn-runtime", help="interpreter with the pinned GNN distribution") - parser.add_argument( - "--require-gnn", - action="store_true", - help="fail when no pinned GNN runtime or checkout is available", - ) args = parser.parse_args(argv) selected = _selected(args.arm or []) gnn_runtime = args.gnn_runtime or os.environ.get("GNN_RUNTIME_PYTHON") - gnn_available = bool(gnn_runtime or os.environ.get("GNN_ROOT")) if args.list: for arm in (*ALIASES, *ARM_ORDER): print(f"{arm:10s} {ARM_HELP[arm]}") @@ -302,21 +311,11 @@ def main(argv: list[str] | None = None) -> int: arm, workdir=Path(""), gnn_runtime=gnn_runtime, - gnn_available=gnn_available, ) for command in plan or [["(python-driven arm)"]]: print(" $", " ".join(command)) return 0 - if "gnn" in selected and not gnn_available: - if args.require_gnn: - parser.error("--require-gnn needs --gnn-runtime, GNN_RUNTIME_PYTHON, or GNN_ROOT") - print( - "gnn validation not run: set GNN_RUNTIME_PYTHON or GNN_ROOT to run the pinned " - "parser conformance check (see docs/gnn.md)", - flush=True, - ) - receipt: dict[str, object] = { "schema": SCHEMA, "generated": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), @@ -341,7 +340,6 @@ def main(argv: list[str] | None = None) -> int: arm, workdir=workdir, gnn_runtime=gnn_runtime, - gnn_available=gnn_available, ) or [] ) diff --git a/scripts/validate_manuscript.py b/scripts/validate_manuscript.py index 7219eff..b4396b5 100644 --- a/scripts/validate_manuscript.py +++ b/scripts/validate_manuscript.py @@ -33,6 +33,79 @@ def _bibliography_keys() -> set[str]: return set(re.findall(r"@\w+\{([^,]+),", content)) +def figure_registry_errors( + manuscript: Path, figure_dir: Path | None = None, *, check_files: bool = True +) -> list[str]: + """Compare the figure declaration with source references and asset names.""" + + try: + registry = json.loads((manuscript / "figure-registry.json").read_text(encoding="utf-8")) + if not isinstance(registry, dict) or registry.get("schema") != "manuscript-figures/v1": + raise ValueError("expected manuscript-figures/v1 schema") + entries = registry.get("figures") + if not isinstance(entries, list) or not entries: + raise ValueError("figures must be a non-empty list") + declared: dict[str, dict[str, str]] = {} + filenames: set[str] = set() + for entry in entries: + if not isinstance(entry, dict) or any( + not isinstance(entry.get(key), str) for key in ("label", "source", "svg", "png") + ): + raise ValueError("each figure must declare label, source, svg, and png strings") + label = entry["label"] + if not re.fullmatch(r"fig:[A-Za-z0-9_-]+", label) or label in declared: + raise ValueError(f"invalid or duplicate figure label: {label}") + if not re.fullmatch(r"[0-9][0-9]_[^/\\]+\.md", entry["source"]): + raise ValueError(f"invalid figure source: {entry['source']}") + for suffix in ("svg", "png"): + name = entry[suffix] + if Path(name).name != name or not name.endswith(f".{suffix}") or name in filenames: + raise ValueError(f"invalid or duplicate figure filename: {name}") + filenames.add(name) + if Path(entry["svg"]).stem != Path(entry["png"]).stem: + raise ValueError(f"figure companions have different stems: {label}") + declared[label] = entry + except (OSError, ValueError) as exc: + return [f"figure registry invalid: {exc}"] + + errors: list[str] = [] + referenced: set[str] = set() + for source in sorted(manuscript.glob("[0-9][0-9]_*.md")): + text = source.read_text(encoding="utf-8") + for label in re.findall(r"@(fig:[A-Za-z0-9_-]+)", text): + if label not in declared: + errors.append(f"figure registry has no entry for reference {label} in {source.name}") + for image in re.finditer(r"!\[[^\]]*\]\(figures/([^)]+)\)(?:\{([^}]*)\})?", text): + name, attributes = image.groups() + label_match = re.search(r"#(fig:[A-Za-z0-9_-]+)", attributes or "") + label = label_match.group(1) if label_match else "" + entry = declared.get(label) + if entry is None: + errors.append(f"figure registry has no entry for {name} ({label or 'unlabelled'})") + continue + referenced.add(label) + if source.name != entry["source"] or name not in (entry["svg"], entry["png"]): + errors.append(f"figure registry reference mismatch: {label} in {source.name}: {name}") + for label in sorted(declared.keys() - referenced): + errors.append(f"figure registry orphan declaration: {label}") + if check_files: + directory = figure_dir if figure_dir is not None else manuscript / "figures" + actual = ( + { + path.name + for path in directory.iterdir() + if path.is_file() and path.suffix in {".svg", ".png"} + } + if directory.is_dir() + else set() + ) + for name in sorted(filenames - actual): + errors.append(f"figure registry missing asset: {name}") + for name in sorted(actual - filenames): + errors.append(f"figure registry orphan asset: {name}") + return errors + + def validate() -> list[str]: errors: list[str] = [] sources = sorted(MANUSCRIPT.glob("[0-9][0-9]_*.md")) @@ -51,9 +124,7 @@ def validate() -> list[str]: missing_citations = sorted(citations - _bibliography_keys()) if missing_citations: errors.append("missing bibliography keys: " + ", ".join(missing_citations)) - for figure in re.findall(r"\]\((figures/[^)]+)\)", all_text): - if not (MANUSCRIPT / figure).is_file(): - errors.append(f"missing referenced figure: {figure}") + errors.extend(figure_registry_errors(MANUSCRIPT)) tokens = set(re.findall(r"\{\{([A-Za-z_][A-Za-z0-9_]*)\}\}", all_text)) try: undeclared = sorted(tokens - _declared_variables()) diff --git a/tests/test_publication.py b/tests/test_publication.py index 197aaac..6cd0c26 100644 --- a/tests/test_publication.py +++ b/tests/test_publication.py @@ -1,5 +1,6 @@ """Tests for reproducible publication-asset handling.""" +import json import re from pathlib import Path @@ -54,6 +55,47 @@ def test_manuscript_validator_reports_negative_controls(monkeypatch, tmp_path): assert any("LaTeX" in error for error in errors) +@pytest.mark.parametrize("mode", ["--no-figures", "--check"]) +@pytest.mark.parametrize( + ("fault", "message"), + [ + ("reference", "figure registry reference mismatch"), + ("missing", "figure registry missing asset"), + ("orphan", "figure registry orphan asset"), + ], +) +def test_figure_registry_mismatches_fail_validation_and_build( + monkeypatch, tmp_path, capsys, mode, fault, message +): + manuscript = builder.MANUSCRIPT_DIR + for source in [ + *manuscript.glob("[0-9][0-9]_*.md"), + manuscript / "references.bib", + manuscript / "config.yaml", + ]: + (tmp_path / source.name).write_text(source.read_text(encoding="utf-8"), encoding="utf-8") + registry = json.loads((manuscript / "figure-registry.json").read_text(encoding="utf-8")) + figures = tmp_path / "figures" + figures.mkdir() + for entry in registry["figures"]: + for suffix in ("svg", "png"): + (figures / entry[suffix]).write_bytes(b"") + if fault == "reference": + registry["figures"][0]["source"] = "02_formal_model.md" + elif fault == "missing": + (figures / registry["figures"][0]["png"]).unlink() + else: + (figures / "orphan.svg").write_bytes(b"") + (tmp_path / "figure-registry.json").write_text(json.dumps(registry), encoding="utf-8") + monkeypatch.setattr(validator, "MANUSCRIPT", tmp_path) + monkeypatch.setattr(builder, "MANUSCRIPT_DIR", tmp_path) + monkeypatch.setattr(builder, "FIGURE_DIR", figures) + + assert any(message in error for error in validator.validate()) + assert builder.main([mode]) == 1 + assert message in capsys.readouterr().err + + def test_docs_link_checker_reports_broken_links_and_anchors(tmp_path): (tmp_path / "a.md").write_text( "# Heading\n\n" diff --git a/tests/test_release_gate.py b/tests/test_release_gate.py new file mode 100644 index 0000000..7cdabc5 --- /dev/null +++ b/tests/test_release_gate.py @@ -0,0 +1,37 @@ +"""Consumer-visible release gate failure reporting.""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] + + +def test_gnn_arm_without_runtime_cannot_write_a_passing_receipt(tmp_path): + environment = dict(os.environ, UV_OFFLINE="1", UV_NO_SYNC="1") + for name in ("GNN_ROOT", "GNN_RUNTIME_PYTHON", "GNN_PYTHON"): + environment.pop(name, None) + receipt = tmp_path / "gate.json" + result = subprocess.run( + [ + sys.executable, + str(ROOT / "scripts" / "release_check.py"), + "--arm", "gnn", "--timings", str(receipt), "--json", + ], + cwd=ROOT, + env=environment, + stdin=subprocess.DEVNULL, + capture_output=True, + text=True, + check=False, + ) + assert result.returncode != 0, result.stdout + result.stderr + assert "gnn validation not run" in result.stderr + payload = json.loads(receipt.read_text(encoding="utf-8")) + assert payload["arms"]["gnn"]["status"] == "failed" + summary = next(line for line in result.stdout.splitlines() if line.startswith('{"arms":')) + assert json.loads(summary)["ok"] is False