Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/_generated/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@ checked by a gate arm; do not edit these files by hand.

| File | Producer | Check |
| --- | --- | --- |
| [`gate-metrics.md`](gate-metrics.md) | `scripts/gate_metrics.py` from [`gate-timings.json`](gate-timings.json) | `scripts/release_check.py --arm docs` |
| [`gate-metrics.md`](gate-metrics.md) | `scripts/gate_metrics.py` from [`gate-timings.json`](gate-timings.json) and [`gate-baseline.json`](gate-baseline.json) | `scripts/release_check.py --arm docs` |
| [`manuscript-variables.json`](manuscript-variables.json) | `scripts/manuscript_variables.py` | `scripts/release_check.py --arm manuscript` |

`gate-timings.json` is a recorded receipt from one `--timings` run of the gate;
it is evidence for the measured numbers, not a claim about another host.
`gate-baseline.json` records the pre-change arm commands and their measured wall
times. Both are evidence for the measured numbers, not claims about another
host.
77 changes: 77 additions & 0 deletions docs/_generated/gate-baseline.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
{
"schema": "activeblockference-gate-baseline/v1",
"measured": "2026-11-12",
"host": "macOS-27.0.1-arm64-arm-64bit",
"python": "3.12.16",
"conditions": "One manual run per command before the composable gate landed, on the same host. Other estate jobs drove the load average to about 100 and all caches were warm, so these are historical evidence, not a controlled benchmark.",
"arms": {
"lint": {
"seconds": 0.34,
"comparable": true,
"commands": [
{"command": "uv run ruff check blockference tests GRTs scripts", "seconds": 0.34}
]
},
"types": {
"seconds": 10.35,
"comparable": false,
"note": "The baseline mypy run targeted Python 3.12; the arm now targets the requires-python floor (3.11).",
"commands": [
{"command": "uv run mypy blockference", "seconds": 10.35}
]
},
"test-full": {
"seconds": 88.77,
"comparable": false,
"note": "The baseline was `uv run pytest` without `--cov` or `--cov-fail-under=90`; the arm runs coverage. Recorded result: 84 passed, 1 skipped.",
"commands": [
{"command": "uv run pytest", "seconds": 88.77}
]
},
"docs": {
"seconds": 0.22,
"comparable": false,
"note": "The baseline ran only the link/anchor checker; the arm also verifies the generated metrics document.",
"commands": [
{"command": "uv run python scripts/check_docs_links.py", "seconds": 0.22}
]
},
"manuscript": {
"seconds": 18.81,
"comparable": false,
"note": "The baseline ran validation, build, and the build check; the arm also verifies the generated variables file.",
"commands": [
{"command": "uv run python scripts/validate_manuscript.py", "seconds": 3.85},
{"command": "uv run python scripts/build_manuscript.py", "seconds": 7.24},
{"command": "uv run python scripts/build_manuscript.py --check", "seconds": 7.72}
]
},
"gnn": {
"seconds": 14.8,
"comparable": true,
"commands": [
{"command": "uv run python scripts/export_gnn_spec.py --check", "seconds": 3.08},
{"command": "uv run python scripts/validate_gnn_spec.py --json", "seconds": 11.72}
]
},
"smoke": {
"seconds": 15.42,
"comparable": true,
"commands": [
{"command": "uv run blockference pipeline --config configs/smoke.yml --output-root <temporary>/output --run-name gate_smoke", "seconds": 7.57},
{"command": "uv run blockference validation --run-dir <temporary>/output/gate_smoke", "seconds": 3.66},
{"command": "uv run blockference run --config configs/smoke.yml --output <temporary>/run.csv", "seconds": 4.19}
]
},
"package": {
"seconds": 17.67,
"comparable": true,
"commands": [
{"command": "uv build", "seconds": 4.98},
{"command": "uv venv --python <project>/.venv/bin/python <temporary>/venv", "seconds": 0.04},
{"command": "uv pip install --python <temporary>/venv/bin/python dist/*.whl", "seconds": 1.01},
{"command": "<temporary>/venv/bin/python -c \"from blockference import GridWorld; ...\"", "seconds": 11.64}
]
}
}
}
49 changes: 31 additions & 18 deletions docs/_generated/gate-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,43 @@

<!-- Generated by scripts/gate_metrics.py; edit the script, not this file. -->

Measured numbers from one recorded run of the canonical gate on `macOS-27.0.1-arm64-arm-64bit` with Python 3.12.16 at `2026-11-12T13:09:06Z`.
Measured numbers from one recorded run of the canonical gate on `macOS-27.0.1-arm64-arm-64bit` with Python 3.12.16 at `2026-11-12T13:20:11Z`.

Refresh with:

```bash
uv run python scripts/release_check.py --arm all --arm test \
uv run python scripts/release_check.py --arm all --arm test --arm render \
--timings docs/_generated/gate-timings.json
uv run python scripts/gate_metrics.py
```

## Arm wall times

| Arm | Wall seconds | Status | First command |
| --- | --- | --- | --- |
| `lint` | 0.07 | passed | `uv run ruff check blockference tests GRTs scripts` |
| `types` | 0.35 | passed | `uv run mypy blockference` |
| `test` | 12.88 | passed | `uv run pytest -m not slow` |
| `test-full` | 58.06 | passed | `uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90` |
| `docs` | 11.02 | passed | `uv run python scripts/check_docs_links.py` |
| `manuscript` | 7.84 | passed | `uv run python scripts/manuscript_variables.py --check` |
| `gnn` | 5.61 | passed | `uv run python scripts/export_gnn_spec.py --check` |
| `smoke` | 5.23 | passed | `uv run blockference pipeline --config configs/smoke.yml --output-root /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-avw6d67s/output --run-name gate_smoke` |
| `package` | 12.91 | passed | `uv build` |
| `render` | not recorded | — | — |
## Before and after

`n/a` means the arm or selection did not exist before the composable gate.

| Arm | Before (s) | After (s) | After status | After command |
| --- | --- | --- | --- | --- |
| `lint` | 0.34 | 0.05 | passed | `uv run ruff check blockference tests GRTs scripts` |
| `types` | 10.35 | 0.28 | passed | `uv run mypy blockference` |
| `test` | n/a | 15.18 | passed | `uv run pytest -m not slow` |
| `test-full` | 88.77 | 77.03 | passed | `uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90` |
| `docs` | 0.22 | 10.25 | passed | `uv run python scripts/check_docs_links.py` |
| `manuscript` | 18.81 | 7.73 | passed | `uv run python scripts/manuscript_variables.py --check` |
| `gnn` | 14.8 | 5.74 | passed | `uv run python scripts/export_gnn_spec.py --check` |
| `smoke` | 15.42 | 6.1 | passed | `uv run blockference pipeline --config configs/smoke.yml --output-root /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-1k9b70t9/output --run-name gate_smoke` |
| `package` | 17.67 | 12.87 | passed | `uv build` |
| `render` | n/a | 11.65 | passed | `uv run python scripts/build_manuscript.py --render pdf` |

Baseline workloads that are not equivalent to their arms:

- `types`: The baseline mypy run targeted Python 3.12; the arm now targets the requires-python floor (3.11).
- `test-full`: The baseline was `uv run pytest` without `--cov` or `--cov-fail-under=90`; the arm runs coverage. Recorded result: 84 passed, 1 skipped.
- `docs`: The baseline ran only the link/anchor checker; the arm also verifies the generated metrics document.
- `manuscript`: The baseline ran validation, build, and the build check; the arm also verifies the generated variables file.

Baseline conditions: One manual run per command before the composable gate landed, on the same host. Other estate jobs drove the load average to about 100 and all caches were warm, so these are historical evidence, not a controlled benchmark.

The baseline and current receipts are historical evidence. Timings include host load and warm caches; they are not a controlled benchmark.

## Test selection

Expand All @@ -39,6 +52,6 @@ Recorded run: 85 passed, 0 skipped.

`test-full` recorded 90.94% line coverage of `blockference` (floor 90%).

## Receipt
## Receipts

The source of these numbers is [`gate-timings.json`](gate-timings.json), written by `scripts/release_check.py --timings`. An arm absent from that receipt is reported as not recorded, never as passing.
The current numbers come from [`gate-timings.json`](gate-timings.json), written by `scripts/release_check.py --timings`; the before numbers come from [`gate-baseline.json`](gate-baseline.json), recorded once before the composable gate landed. An arm absent from a receipt is reported as not recorded, never as passing.
39 changes: 23 additions & 16 deletions docs/_generated/gate-timings.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,22 @@
"uv run python scripts/check_docs_links.py",
"uv run python scripts/gate_metrics.py --check"
],
"seconds": 11.02,
"seconds": 10.25,
"status": "passed"
},
"gnn": {
"commands": [
"uv run python scripts/export_gnn_spec.py --check",
"uv run python scripts/validate_gnn_spec.py --json"
],
"seconds": 5.61,
"seconds": 5.74,
"status": "passed"
},
"lint": {
"commands": [
"uv run ruff check blockference tests GRTs scripts"
],
"seconds": 0.07,
"seconds": 0.05,
"status": "passed"
},
"manuscript": {
Expand All @@ -30,51 +30,58 @@
"uv run python scripts/build_manuscript.py",
"uv run python scripts/build_manuscript.py --check"
],
"seconds": 7.84,
"seconds": 7.73,
"status": "passed"
},
"package": {
"commands": [
"uv build",
"uv venv --python /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/.venv/bin/python /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-fn41c71n/venv",
"uv pip install --python /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-fn41c71n/venv/bin/python /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/dist/active_blockference-1.0.0-py3-none-any.whl",
"/var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-fn41c71n/venv/bin/python -c from blockference import GridWorld; assert GridWorld(2, {'agent': (0, 0)}).current_state == {'agent': (0, 0)}"
"uv venv --python /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/.venv/bin/python /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-f6ywi5t7/venv",
"uv pip install --python /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-f6ywi5t7/venv/bin/python /Users/friedman/Documents/Git/gitvault/Username/ActiveInferenceInstitute/Public/ActiveBlockference/dist/active_blockference-1.0.0-py3-none-any.whl",
"/var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-package-f6ywi5t7/venv/bin/python -c from blockference import GridWorld; assert GridWorld(2, {'agent': (0, 0)}).current_state == {'agent': (0, 0)}"
],
"seconds": 12.91,
"seconds": 12.87,
"status": "passed"
},
"render": {
"commands": [
"uv run python scripts/build_manuscript.py --render pdf"
],
"seconds": 11.65,
"status": "passed"
},
"smoke": {
"commands": [
"uv run blockference pipeline --config configs/smoke.yml --output-root /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-avw6d67s/output --run-name gate_smoke",
"uv run blockference validation --run-dir /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-avw6d67s/output/gate_smoke",
"uv run blockference run --config configs/smoke.yml --output /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-avw6d67s/run.csv"
"uv run blockference pipeline --config configs/smoke.yml --output-root /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-1k9b70t9/output --run-name gate_smoke",
"uv run blockference validation --run-dir /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-1k9b70t9/output/gate_smoke",
"uv run blockference run --config configs/smoke.yml --output /var/folders/qs/rxkntx5d4hlgz0c7x8v15gym0000gn/T/activeblockference-gate-1k9b70t9/run.csv"
],
"seconds": 5.23,
"seconds": 6.1,
"status": "passed"
},
"test": {
"commands": [
"uv run pytest -m not slow"
],
"seconds": 12.88,
"seconds": 15.18,
"status": "passed"
},
"test-full": {
"commands": [
"uv run pytest --cov=blockference --cov-report=term --cov-report=xml --cov-fail-under=90"
],
"seconds": 58.06,
"seconds": 77.03,
"status": "passed"
},
"types": {
"commands": [
"uv run mypy blockference"
],
"seconds": 0.35,
"seconds": 0.28,
"status": "passed"
}
},
"generated": "2026-11-12T13:09:06Z",
"generated": "2026-11-12T13:20:11Z",
"host": "macOS-27.0.1-arm64-arm-64bit",
"python": "3.12.16",
"schema": "activeblockference-gate/v1",
Expand Down
4 changes: 3 additions & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ smoke runs, and the wheel import check.
mypy targets the `requires-python` floor (`python_version = "3.11"` in
`pyproject.toml`), so one run covers the minimum supported stubs. Notebook
execution, real-pipeline integration, CLI smoke, and figure-byte checks carry
the `slow` marker and are excluded from `--arm test`.
the `slow` marker and are excluded from `--arm test`. The recorded before/after
arm times, including the render arm, are in
[`_generated/gate-metrics.md`](_generated/gate-metrics.md).

The pinned GNN parser conformance check (`scripts/validate_gnn_spec.py`) is
part of the `gnn` arm only when `GNN_ROOT` or `GNN_RUNTIME_PYTHON` is set,
Expand Down
6 changes: 4 additions & 2 deletions docs/gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@ uv run python scripts/release_check.py --list # arms and their commands
```

Measured wall times and test counts live in the generated
[`_generated/gate-metrics.md`](_generated/gate-metrics.md). Do not hand-edit
those numbers: refresh them with the commands the generated page records.
[`_generated/gate-metrics.md`](_generated/gate-metrics.md), which shows the
recorded before/after arm times and labels baseline workloads that are not
equivalent to their arms. Do not hand-edit those numbers: refresh them with the
commands the generated page records.

## Arms

Expand Down
Loading
Loading