diff --git a/.aii/README.md b/.aii/README.md index b41140b..aae520a 100644 --- a/.aii/README.md +++ b/.aii/README.md @@ -41,7 +41,7 @@ Institute repositories without touching this project's own content. The schema i | Task | Command | Working directory | Description | | --- | --- | --- | --- | | `inventory` | `git ls-files` | `.` | List tracked files. | -| `test` | `python .github/scripts/check_archive.py` | `.` | Run the dependency-free archive gate (links, syntax, notebooks). | +| `test` | `python .github/scripts/gate.py fast` | `.` | Run the default dependency-free gate arm (links, syntax, notebooks, gate tests). | ## Links diff --git a/.aii/config.yaml b/.aii/config.yaml index 8cdd78b..8a487c5 100644 --- a/.aii/config.yaml +++ b/.aii/config.yaml @@ -92,8 +92,8 @@ tasks: cmd: git ls-files description: List tracked files. test: - cmd: python .github/scripts/check_archive.py - description: Run the dependency-free archive gate (links, syntax, notebooks). + cmd: python .github/scripts/gate.py fast + description: Run the dependency-free gate arm (links, syntax, notebooks, tests). docs: - README.md - AGENTS.md diff --git a/.aii/docs/INTEGRATION.md b/.aii/docs/INTEGRATION.md index 971357c..6bb80c9 100644 --- a/.aii/docs/INTEGRATION.md +++ b/.aii/docs/INTEGRATION.md @@ -58,7 +58,7 @@ | Task | Command | Working directory | Description | | --- | --- | --- | --- | | `inventory` | `git ls-files` | `.` | List tracked files. | -| `test` | `python .github/scripts/check_archive.py` | `.` | Run the dependency-free archive gate (links, syntax, notebooks). | +| `test` | `python .github/scripts/gate.py fast` | `.` | Run the default dependency-free gate arm (links, syntax, notebooks, gate tests). | ## Validation diff --git a/.github/GNN.md b/.github/GNN.md index f5ea2f5..a8897e1 100644 --- a/.github/GNN.md +++ b/.github/GNN.md @@ -1,12 +1,27 @@ # GNN conformance -`gnn conformance (pinned)` is defined in `.github/workflows/ci.yml` and reads -`.github/gnn-pin.json`, which pins -`ActiveInferenceInstitute/Generalized_Notation_Notation` at `v5.0.3` -(`8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1`). - -The job verifies the pin and installed parser, parses every `*.gnn.md` sketch -in the repository, and type-checks them. When the repository has no sketch yet, -it parses the pinned upstream `input/gnn_files/basics/static_perception.md` -exemplar; this is a pipeline and pin conformance baseline, not a claim that the -repository already contains a model. +The gate command is `.github/scripts/gate.py gnn` (add `--install` to fetch the +pinned parser first). It is the same command the CI job +`gnn-conformance` — displayed as **`gnn conformance (pinned)`** — runs; +see [`docs/RUN_GUIDE.md`](../docs/RUN_GUIDE.md) for every arm. + +The conformance arm reads `.github/gnn-pin.json` (`gnn-importer-pin/v1`), which +pins `ActiveInferenceInstitute/Generalized_Notation_Notation` at tag `v5.0.3`, +revision `8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1` (recorded `2026-11-12`). + +What it validates, in order: + +1. The pin's schema, repository identity, tag shape, 40-hex revision, and + recorded date. +2. The installed `gnn.__version__` against the tag without its `v`. +3. Every `*.gnn.md` sketch in the repository: full parse, syntax validation, and + type checking of the collected models. When the repository has no sketch it + downloads the pinned upstream `input/gnn_files/basics/static_perception.md` + exemplar at the pin's revision and validates that instead, and the JSON + receipt says `"source": "pinned-exemplar"`. + +The arm needs network access and a Python version the pinned release supports +(`>=3.11,<3.15`); CI uses 3.12. The local recipe, including a throwaway virtual +environment, is in [`RUN_GUIDE.md`](../docs/RUN_GUIDE.md#running-the-gnn-arm-locally). +A green result is a pipeline and pin conformance baseline, not a claim that this +repository contains a model. diff --git a/.github/README.md b/.github/README.md index fc8bf3c..681619d 100644 --- a/.github/README.md +++ b/.github/README.md @@ -1,15 +1,29 @@ # `.github/` -- [`workflows/ci.yml`](workflows/ci.yml) — CI: the dependency-free archive gate - and the pinned GNN conformance job. -- [`scripts/check_archive.py`](scripts/check_archive.py) — the repository's - archive gate: relative Markdown links, tracked-Python syntax, notebook JSON - and committed outputs, and tracked Jupyter checkpoints. +- [`workflows/ci.yml`](workflows/ci.yml) — CI: two jobs, `docs and scripts` + (`python .github/scripts/gate.py fast`) and `gnn conformance (pinned)` + (`python .github/scripts/gate.py gnn --install`). The workflow holds no check + body of its own. +- [`scripts/gate.py`](scripts/gate.py) — the repository gate entry point: one + command per arm (`links`, `python`, `notebooks`, `checkpoints`, `archive`, + `tests`, `gnn`, `fast`, `full`, `timings`). Run + `python .github/scripts/gate.py --list`. +- [`scripts/archive_checks.py`](scripts/archive_checks.py) — the dependency-free + archive checks (relative Markdown links, tracked-Python compilation, notebook + JSON and committed outputs, tracked Jupyter checkpoints). +- [`scripts/gnn_conformance.py`](scripts/gnn_conformance.py) — pin validation, + pinned-parser install, and sketch/exemplar conformance. +- [`scripts/tests/test_gate.py`](scripts/tests/test_gate.py) — the gate's own + tests, run by the `tests` arm. - [`dependabot.yml`](dependabot.yml) — dependency update configuration. -- [`gnn-pin.json`](gnn-pin.json) — the pinned Generalized Notation Notation - revision (v5.0.3, commit `8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1`) used by - the conformance job. -- [`GNN.md`](GNN.md) — what the pinned GNN conformance job checks. +- [`gnn-pin.json`](gnn-pin.json) — the `gnn-importer-pin/v1` pin used by the + conformance arm: repository, tag `v5.0.3`, revision + `8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1`, recorded `2026-11-12`. +- [`GNN.md`](GNN.md) — what the pinned GNN conformance arm parses. + +Arms, job commands, and measured wall times are documented in +[`../docs/RUN_GUIDE.md`](../docs/RUN_GUIDE.md) and the generated +[`../docs/gate-timings.md`](../docs/gate-timings.md). Repository change rules and conventions are in [`../AGENTS.md`](../AGENTS.md); the documentation hub is [`../docs/index.md`](../docs/index.md). diff --git a/.github/scripts/archive_checks.py b/.github/scripts/archive_checks.py new file mode 100755 index 0000000..79ed36b --- /dev/null +++ b/.github/scripts/archive_checks.py @@ -0,0 +1,149 @@ +#!/usr/bin/env python3 +"""Dependency-free archive checks for the Knowledge-Engineering research archive. + +Four checks, each of which can fail: + +1. ``links``: every relative Markdown link or image target resolves to a path + inside the repository. +2. ``python``: every tracked Python file compiles under the running Python. +3. ``notebooks``: every tracked notebook is valid JSON and carries no committed + cell outputs, matching the repository rule that notebook outputs stay out of + git. +4. ``checkpoints``: no Jupyter checkpoint file is tracked. + +External URLs are not fetched, so the verdict does not depend on third-party +availability. This module holds the checks; `gate.py` is the command-line entry +point that runs them individually or together. +""" + +from __future__ import annotations + +import json +import re +from pathlib import Path +from subprocess import run as run_process +from urllib.parse import unquote + +INLINE_LINK = re.compile(r"!?\[[^\]]*\]\(([^)]+)\)") +REFERENCE_LINK = re.compile(r"^\[[^\]]+\]:\s*(\S+)") +EXTERNAL_TARGET = re.compile(r"^[a-zA-Z][a-zA-Z0-9+.-]*:") +CHECKPOINT = re.compile(r"(^|/)\.ipynb_checkpoints/") + + +def tracked_files(root: Path) -> list[Path]: + """Return every tracked path, resolved against ``root``.""" + result = run_process( + ["git", "-C", str(root), "ls-files", "-z"], + check=True, + capture_output=True, + ) + return [root / name for name in result.stdout.decode("utf-8").split("\0") if name] + + +def link_target(raw: str) -> str: + """Extract the destination from a Markdown link target, dropping any title.""" + target = raw.strip() + if target.startswith("<") and ">" in target: + return target[1 : target.index(">")] + parts = target.split() + return parts[0] if parts else "" + + +def _relative(path: Path, root: Path) -> str: + try: + return path.relative_to(root).as_posix() + except ValueError: + return str(path) + + +def check_links(files: list[Path], root: Path) -> list[str]: + """Flag relative link or image targets that do not resolve inside ``root``.""" + errors: list[str] = [] + resolved_root = root.resolve() + for path in files: + if path.suffix.lower() not in {".md", ".markdown"}: + continue + text = path.read_text(encoding="utf-8") + targets = INLINE_LINK.findall(text) + REFERENCE_LINK.findall(text) + for raw in targets: + target = unquote(link_target(raw)) + if not target or target.startswith("#") or EXTERNAL_TARGET.match(target): + continue + base = target.split("#", 1)[0] + if not base: + continue + resolved = (path.parent / base).resolve() + try: + resolved.relative_to(resolved_root) + except ValueError: + errors.append(f"{_relative(path, root)}: link escapes repository: {raw}") + continue + if not resolved.exists(): + errors.append(f"{_relative(path, root)}: missing link target: {raw}") + return errors + + +def check_python(files: list[Path], root: Path) -> list[str]: + """Flag tracked Python files that do not compile.""" + errors: list[str] = [] + for path in files: + if path.suffix != ".py": + continue + source = path.read_text(encoding="utf-8") + try: + compile(source, str(path), "exec") + except SyntaxError as exc: + errors.append(f"{_relative(path, root)}: Python syntax error: {exc}") + return errors + + +def check_notebooks(files: list[Path], root: Path) -> list[str]: + """Flag notebooks that are not valid JSON or carry committed cell outputs.""" + errors: list[str] = [] + for path in files: + if path.suffix != ".ipynb": + continue + try: + data = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError as exc: + errors.append(f"{_relative(path, root)}: invalid notebook JSON: {exc}") + continue + if not isinstance(data, dict) or not isinstance(data.get("cells"), list): + errors.append(f"{_relative(path, root)}: notebook has no cells list") + continue + for index, cell in enumerate(data["cells"]): + if isinstance(cell, dict) and cell.get("outputs"): + errors.append( + f"{_relative(path, root)}: cell {index} carries committed outputs" + ) + return errors + + +def check_checkpoints(files: list[Path], root: Path) -> list[str]: + """Flag tracked Jupyter checkpoint files.""" + return [ + f"{_relative(path, root)}: Jupyter checkpoint file is tracked" + for path in files + if CHECKPOINT.search(path.as_posix()) + ] + + +CHECKS = { + "links": check_links, + "python": check_python, + "notebooks": check_notebooks, + "checkpoints": check_checkpoints, +} + + +def run(names: tuple[str, ...], root: Path) -> tuple[int, list[str]]: + """Run the named checks over one ``git ls-files`` read. + + Returns the number of tracked files inspected and the collected errors, so + callers can report a single consistent summary. + """ + files = tracked_files(root) + errors: list[str] = [] + for name in names: + errors.extend(CHECKS[name](files, root)) + return len(files), errors diff --git a/.github/scripts/check_archive.py b/.github/scripts/check_archive.py deleted file mode 100644 index b3f7ca4..0000000 --- a/.github/scripts/check_archive.py +++ /dev/null @@ -1,139 +0,0 @@ -#!/usr/bin/env python3 -"""Dependency-free checks for the Knowledge-Engineering research archive. - -Four gates, all of which can fail: - -1. Every relative Markdown link or image target resolves to a tracked path - inside the repository. -2. Every tracked Python file compiles under the running Python. -3. Every tracked notebook is valid JSON and carries no committed cell outputs, - matching the repository rule that notebook outputs stay out of git. -4. No Jupyter checkpoint file is tracked. - -External URLs are not fetched, so the verdict does not depend on third-party -availability. -""" - -from __future__ import annotations - -import json -import re -import subprocess -import sys -from pathlib import Path -from urllib.parse import unquote - -ROOT = Path(__file__).resolve().parents[2] -INLINE_LINK = re.compile(r"!?\[[^\]]*\]\(([^)]+)\)") -REFERENCE_LINK = re.compile(r"^\[[^\]]+\]:\s*(\S+)", re.MULTILINE) -SCHEME = re.compile(r"^[a-zA-Z][a-zA-Z0-9+.-]*:") -CHECKPOINT = re.compile(r"(^|/)\.ipynb_checkpoints/") - - -def tracked_files() -> list[Path]: - result = subprocess.run( - ["git", "-C", str(ROOT), "ls-files", "-z"], - check=True, - capture_output=True, - ) - return [ROOT / name for name in result.stdout.decode("utf-8").split("\0") if name] - - -def link_target(raw: str) -> str: - target = raw.strip() - if target.startswith("<") and ">" in target: - return target[1 : target.index(">")] - parts = target.split() - return parts[0] if parts else "" - - -def check_links(files: list[Path]) -> list[str]: - errors: list[str] = [] - resolved_root = ROOT.resolve() - for path in files: - if path.suffix.lower() not in {".md", ".markdown"}: - continue - text = path.read_text(encoding="utf-8") - targets = INLINE_LINK.findall(text) + REFERENCE_LINK.findall(text) - for raw in targets: - target = unquote(link_target(raw)) - if not target or target.startswith("#") or SCHEME.match(target): - continue - base = target.split("#", 1)[0] - if not base: - continue - resolved = (path.parent / base).resolve() - try: - resolved.relative_to(resolved_root) - except ValueError: - errors.append(f"{path.relative_to(ROOT)}: link escapes repository: {raw}") - continue - if not resolved.exists(): - errors.append(f"{path.relative_to(ROOT)}: missing link target: {raw}") - return errors - - -def check_python(files: list[Path]) -> list[str]: - errors: list[str] = [] - for path in files: - if path.suffix != ".py": - continue - source = path.read_text(encoding="utf-8") - try: - compile(source, str(path), "exec") - except SyntaxError as exc: - errors.append(f"{path.relative_to(ROOT)}: Python syntax error: {exc}") - return errors - - -def check_notebooks(files: list[Path]) -> list[str]: - errors: list[str] = [] - for path in files: - if path.suffix != ".ipynb": - continue - try: - data = json.loads(path.read_text(encoding="utf-8")) - except json.JSONDecodeError as exc: - errors.append(f"{path.relative_to(ROOT)}: invalid notebook JSON: {exc}") - continue - if not isinstance(data, dict) or not isinstance(data.get("cells"), list): - errors.append(f"{path.relative_to(ROOT)}: notebook has no cells list") - continue - for index, cell in enumerate(data["cells"]): - if isinstance(cell, dict) and cell.get("outputs"): - errors.append( - f"{path.relative_to(ROOT)}: cell {index} carries committed outputs" - ) - return errors - - -def check_checkpoints(files: list[Path]) -> list[str]: - return [ - f"{path.relative_to(ROOT)}: Jupyter checkpoint file is tracked" - for path in files - if CHECKPOINT.search(path.as_posix()) - ] - - -def main() -> int: - files = tracked_files() - errors = ( - check_links(files) - + check_python(files) - + check_notebooks(files) - + check_checkpoints(files) - ) - if errors: - print(f"{len(errors)} archive check(s) failed:", file=sys.stderr) - for error in errors: - print(f"- {error}", file=sys.stderr) - return 1 - print( - f"checked {len(files)} tracked files: links resolve; Python compiles; " - "notebooks are output-free JSON; no checkpoints" - ) - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/.github/scripts/gate.py b/.github/scripts/gate.py new file mode 100755 index 0000000..0495eaf --- /dev/null +++ b/.github/scripts/gate.py @@ -0,0 +1,286 @@ +#!/usr/bin/env python3 +"""Single entry point for every Knowledge-Engineering gate arm. + +The arms are separately runnable and each one is exactly what CI invokes, so a +local run and a CI run execute the same command: + +=================================== ================================================== +arm what it checks +=================================== ================================================== +``links`` relative Markdown link and image targets resolve +``python`` tracked Python files compile +``notebooks`` tracked notebooks are output-free JSON +``checkpoints`` no Jupyter checkpoint file is tracked +``archive`` the four archive arms over one file listing +``tests`` the gate's own unit tests +``gnn`` pinned GNN parser and sketch/exemplar conformance +``fast`` (default) ``archive`` + ``tests``; what CI's docs job runs +``full`` ``fast`` + ``gnn``; the whole gate +``timings`` measure each arm once, rewrite docs/gate-timings.md +=================================== ================================================== + +``gnn`` is the only arm that needs the network and third-party packages: it +installs the distribution pinned in ``.github/gnn-pin.json`` when ``--install`` +is given. ``full`` combines the offline arms with it. + +There is no separate lint or type arm because the repository has no linter or +type-checker configuration: the archive checks are the only static analysis it +defines, and they are not replaced by an unconfigured tool. +""" + +from __future__ import annotations + +import argparse +import datetime as dt +import platform +import subprocess +import sys +import time +import unittest +from pathlib import Path + +SCRIPTS = Path(__file__).resolve().parent +ROOT = SCRIPTS.parents[1] + +import archive_checks # noqa: E402 (sibling module, resolved from the script directory) +import gnn_conformance # noqa: E402 + +ARCHIVE_ARMS = ("links", "python", "notebooks", "checkpoints") +TIMINGS_DOC = Path("docs") / "gate-timings.md" +TIMINGS_MARKER = ( + "" +) + +DESCRIPTIONS = { + "links": "relative Markdown link and image targets resolve inside the repository", + "python": "tracked Python files compile under the running interpreter", + "notebooks": "tracked notebooks are valid JSON with no committed outputs", + "checkpoints": "no Jupyter checkpoint file is tracked", + "archive": "all four archive checks over one `git ls-files` read", + "tests": "the gate's own dependency-free unit tests", + "gnn": "pinned GNN parser, sketch parsing, and type checking", + "fast": "archive checks plus tests; the default arm and CI's docs job", + "full": "the fast arm plus the pinned GNN conformance arm", + "timings": "measure each offline arm once and rewrite the generated timings page", +} +ARM_ORDER = ( + "links", + "python", + "notebooks", + "checkpoints", + "archive", + "tests", + "gnn", + "fast", + "full", + "timings", +) + + +def command_for(arm: str) -> str: + """Return the documented command line for ``arm``.""" + return f"python .github/scripts/gate.py {arm}" + + +def _report(label: str, inspected: int, errors: list[str], note: str = "") -> int: + if errors: + print(f"{label}: {len(errors)} check(s) failed:", file=sys.stderr) + for error in errors: + print(f"- {error}", file=sys.stderr) + return 1 + summary = f"{label}: ok ({inspected} tracked files" + summary += f"; {note}" if note else "" + print(f"{summary})") + return 0 + + +def arm_archive(names: tuple[str, ...], label: str) -> int: + inspected, errors = archive_checks.run(names, ROOT) + notes = { + "links": "links resolve", + "python": "Python compiles", + "notebooks": "notebooks are output-free JSON", + "checkpoints": "no checkpoints", + } + note = "; ".join(notes[name] for name in names) + return _report(label, inspected, errors, note) + + +def arm_tests() -> int: + suite = unittest.TestLoader().discover( + str(SCRIPTS / "tests"), top_level_dir=str(SCRIPTS) + ) + total = suite.countTestCases() + if not total: + print("tests: no test cases were discovered", file=sys.stderr) + return 1 + result = unittest.TextTestRunner(verbosity=1, stream=sys.stdout).run(suite) + if not result.wasSuccessful(): + return 1 + print(f"tests: ok ({total} tests)") + return 0 + + +def run_arm(arm: str, *, install: bool = False) -> int: + """Run one arm and return its exit status.""" + if arm in ARCHIVE_ARMS: + return arm_archive((arm,), arm) + if arm == "archive": + return arm_archive(ARCHIVE_ARMS, "archive") + if arm == "tests": + return arm_tests() + if arm == "gnn": + return gnn_conformance.run(ROOT, install_first=install) + if arm == "fast": + return 0 if arm_archive(ARCHIVE_ARMS, "archive") == 0 and arm_tests() == 0 else 1 + if arm == "full": + if arm_archive(ARCHIVE_ARMS, "archive") != 0 or arm_tests() != 0: + return 1 + return gnn_conformance.run(ROOT, install_first=install) + raise ValueError(f"unknown arm: {arm}") + + +def _measure(arm: str) -> float: + started = time.perf_counter() + result = subprocess.run( + [sys.executable, str(Path(__file__).resolve()), arm], + cwd=ROOT, + capture_output=True, + text=True, + ) + elapsed = time.perf_counter() - started + if result.returncode != 0: + sys.stdout.write(result.stdout) + sys.stderr.write(result.stderr) + raise SystemExit(f"timings: arm {arm!r} failed; fix it before measuring") + return elapsed + + +def timings_arms() -> tuple[str, ...]: + """The arms the generated timings page measures.""" + return tuple(arm for arm in ARM_ORDER if arm not in {"gnn", "full", "timings"}) + + +def render_timings(rows: list[tuple[str, float]], gnn_seconds: float | None) -> str: + """Render the generated timings page.""" + implementation = platform.python_implementation() + version = platform.python_version() + lines = [ + TIMINGS_MARKER, + "# Gate arm timings", + "", + "Wall time for one run of each arm on this host, measured " + f"{dt.datetime.now(dt.UTC).strftime('%Y-%m-%d')} with " + f"{implementation} {version} on {platform.system()} " + f"{platform.machine()}. These are single local samples, not CI " + "guarantees: GitHub Actions uses different hardware and also pays for " + "checkout and interpreter setup.", + "", + "| Arm | Command | Wall time |", + "| --- | --- | --- |", + ] + for arm, seconds in rows: + lines.append(f"| `{arm}` | `{command_for(arm)}` | {seconds:.2f} s |") + lines += [ + "", + "## Interpreter-specific arms", + "", + "`gnn` is measured separately because it installs third-party packages " + "and reads the network, so its duration depends on the package cache " + "and mirror.", + ] + if gnn_seconds is None: + lines.append( + "It was not measured for this page: the pinned parser was not " + "importable in the measuring interpreter." + ) + else: + lines.append( + f"With the pinned parser already installed it took {gnn_seconds:.2f} s " + "for one conformance check, excluding the install step; the install " + "step's duration depends on pip's cache. Refresh this page from an " + "interpreter that has the parser installed to record it again." + ) + return "\n".join(lines) + "\n" + + +def write_timings(output: Path, *, include_gnn: bool) -> int: + arms = timings_arms() + # The `tests` arm asserts that this page lists every measured arm, and `fast` + # runs it, so a first generation would otherwise measure itself into a + # failure. Write a provisional page, measure, then replace it with the + # measured one; restore the previous bytes if measuring fails. + previous = output.read_text(encoding="utf-8") if output.is_file() else None + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text( + render_timings([(arm, 0.0) for arm in arms], None), encoding="utf-8" + ) + try: + rows = [(arm, _measure(arm)) for arm in arms] + gnn_seconds: float | None = _measure("gnn") if include_gnn else None + document = render_timings(rows, gnn_seconds) + except BaseException: + if previous is None: + output.unlink(missing_ok=True) + else: + output.write_text(previous, encoding="utf-8") + raise + output.write_text(document, encoding="utf-8") + print(document) + print(f"timings: wrote {output}") + return 0 + + +def parsers() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="gate.py", + description="Run one gate arm; every arm is what CI runs.", + ) + parser.add_argument( + "arm", + nargs="?", + default="fast", + choices=ARM_ORDER, + help="arm to run (default: fast)", + ) + parser.add_argument( + "--install", + action="store_true", + help="install the pinned GNN parser before the gnn or full arm", + ) + parser.add_argument( + "--output", + type=Path, + default=ROOT / TIMINGS_DOC, + help="destination of the generated timings page (timings arm only)", + ) + parser.add_argument( + "--include-gnn", + action="store_true", + help="also measure the gnn arm (timings arm only; needs the parser)", + ) + parser.add_argument( + "--list", action="store_true", help="list the arms and their commands" + ) + return parser + + +def main(argv: list[str] | None = None) -> int: + options = parsers().parse_args(argv) + if options.list: + width = max(len(arm) for arm in ARM_ORDER) + for arm in ARM_ORDER: + print(f"{arm:<{width}} {command_for(arm)}") + print(f"{'':<{width}} {DESCRIPTIONS[arm]}") + return 0 + if options.install and options.arm not in {"gnn", "full"}: + print("gate.py: --install is only valid for the gnn and full arms", file=sys.stderr) + return 2 + if options.arm == "timings": + return write_timings(options.output, include_gnn=options.include_gnn) + return run_arm(options.arm, install=options.install) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/.github/scripts/gnn_conformance.py b/.github/scripts/gnn_conformance.py new file mode 100755 index 0000000..05330bc --- /dev/null +++ b/.github/scripts/gnn_conformance.py @@ -0,0 +1,172 @@ +#!/usr/bin/env python3 +"""Pinned Generalized Notation Notation (GNN) conformance for this archive. + +The repository pins one upstream parser revision in ``.github/gnn-pin.json`` +(schema ``gnn-importer-pin/v1``). This module validates that pin, installs the +pinned distribution when asked, and verifies that + +* the installed parser reports the pinned version, +* every ``*.gnn.md`` sketch in the repository parses, passes the syntax + validator, and passes the type checker, or +* when the repository carries no sketch, a pinned upstream exemplar does. + +The GitHub Actions job ``gnn conformance (pinned)`` and the local gate arm +``gnn`` both call `gate.py`, which calls this module, so the job body exists +once. Fetching the parser and the exemplar needs the network; nothing here +touches the committed research artifacts. +""" + +from __future__ import annotations + +import json +import re +import shutil +import subprocess +import sys +import tempfile +import urllib.request +from pathlib import Path + +PIN_PATH = Path(".github") / "gnn-pin.json" +PIN_SCHEMA = "gnn-importer-pin/v1" +EXPECTED_REPOSITORY = "ActiveInferenceInstitute/Generalized_Notation_Notation" +EXEMPLAR_PATH = "input/gnn_files/basics/static_perception.md" +RUNTIME_DEPENDENCIES = ("defusedxml", "psutil", "pyyaml") +TAG = re.compile(r"^v(\d+\.\d+\.\d+)$") +REVISION = re.compile(r"^[0-9a-f]{40}$") +RECORDED = re.compile(r"^\d{4}-\d{2}-\d{2}$") +FETCH_TIMEOUT = 60 + + +def load_pin(root: Path) -> dict: + """Read the pin file, raising `ValueError` when it is not an object.""" + data = json.loads((root / PIN_PATH).read_text(encoding="utf-8")) + if not isinstance(data, dict): + raise ValueError(f"{PIN_PATH.as_posix()}: pin file is not a JSON object") + return data + + +def validate_pin(pin: dict) -> list[str]: + """Return every schema or identity problem with ``pin``.""" + errors: list[str] = [] + if pin.get("schema") != PIN_SCHEMA: + errors.append(f"pin schema is {pin.get('schema')!r}, expected {PIN_SCHEMA!r}") + if pin.get("repository") != EXPECTED_REPOSITORY: + errors.append( + f"pin repository is {pin.get('repository')!r}, " + f"expected {EXPECTED_REPOSITORY!r}" + ) + if not TAG.match(str(pin.get("tag", ""))): + errors.append(f"pin tag {pin.get('tag')!r} is not vMAJOR.MINOR.PATCH") + if not REVISION.match(str(pin.get("revision", ""))): + errors.append(f"pin revision {pin.get('revision')!r} is not a 40-hex commit") + if not RECORDED.match(str(pin.get("recorded", ""))): + errors.append(f"pin recorded {pin.get('recorded')!r} is not YYYY-MM-DD") + return errors + + +def install_spec(pin: dict) -> str: + """Return the pip requirement for the pinned distribution.""" + return ( + "generalized-notation-notation @ " + f"git+https://github.com/{pin['repository']}.git@{pin['revision']}" + ) + + +def install(pin: dict) -> None: + """Install the parser's runtime dependencies and the pinned distribution.""" + pip = [sys.executable, "-m", "pip", "install", "--disable-pip-version-check"] + subprocess.check_call([*pip, *RUNTIME_DEPENDENCIES]) + subprocess.check_call([*pip, "--no-deps", install_spec(pin)]) + + +def _exemplar_bytes(pin: dict) -> bytes: + """Download the pinned upstream exemplar.""" + url = ( + "https://raw.githubusercontent.com/" + f"{pin['repository']}/{pin['revision']}/{EXEMPLAR_PATH}" + ) + with urllib.request.urlopen(url, timeout=FETCH_TIMEOUT) as response: + return response.read() + + +def _check_parses(sketch: Path) -> None: + from gnn.parsers.basic import parse_gnn_formal, validate_gnn_syntax + + parsed = parse_gnn_formal(sketch) + if parsed is None or getattr(parsed, "parse_errors", None): + raise SystemExit(f"{sketch} did not parse cleanly") + valid, errors = validate_gnn_syntax(sketch.read_text(encoding="utf-8")) + if not valid: + raise SystemExit(f"{sketch} syntax errors: {errors!r}") + + +def _check_types(sketches: list[Path]) -> None: + from gnn.type_checker import GNNTypeChecker + + with tempfile.TemporaryDirectory(prefix="gnn-types-") as output_dir: + model_dir = Path(output_dir) / "models" + model_dir.mkdir() + for index, sketch in enumerate(sketches): + shutil.copy2(sketch, model_dir / f"{index:03d}_{sketch.name}") + if not GNNTypeChecker().validate_gnn_files(model_dir, Path(output_dir)): + raise SystemExit("GNN type checker rejected the checked sketches") + + +def verify(root: Path, pin: dict) -> dict: + """Verify the installed parser and the checked sketches; return the receipt.""" + try: + from gnn import __version__ + except ImportError as exc: + raise SystemExit( + "the pinned GNN parser is not installed for this interpreter; " + "run `python .github/scripts/gate.py gnn --install`" + ) from exc + + expected_version = str(pin["tag"]).removeprefix("v") + if __version__ != expected_version: + raise SystemExit(f"installed GNN {__version__!r} does not match {pin['tag']!r}") + + sketches = sorted( + path + for path in root.rglob("*.gnn.md") + if ".git" not in path.relative_to(root).parts + ) + + with tempfile.TemporaryDirectory(prefix="gnn-conformance-") as workdir: + if sketches: + checked, source = sketches, "repository" + else: + exemplar = Path(workdir) / Path(EXEMPLAR_PATH).name + exemplar.write_bytes(_exemplar_bytes(pin)) + checked, source = [exemplar], "pinned-exemplar" + for sketch in checked: + _check_parses(sketch) + _check_types(checked) + + return { + "gnn-version": __version__, + "pin-revision": pin["revision"], + "models": len(checked), + "source": source, + "result": "verified", + } + + +def run(root: Path, *, install_first: bool = False) -> int: + """Validate the pin, optionally install the parser, then verify.""" + try: + pin = load_pin(root) + except (OSError, ValueError) as exc: + print(f"gnn conformance: {exc}", file=sys.stderr) + return 1 + errors = validate_pin(pin) + if errors: + print("gnn conformance: pin does not satisfy the contract:", file=sys.stderr) + for error in errors: + print(f"- {error}", file=sys.stderr) + return 1 + if install_first: + install(pin) + print(json.dumps(verify(root, pin), sort_keys=True)) + return 0 diff --git a/.github/scripts/tests/__init__.py b/.github/scripts/tests/__init__.py new file mode 100644 index 0000000..199fc7f --- /dev/null +++ b/.github/scripts/tests/__init__.py @@ -0,0 +1 @@ +"""Tests for the repository gate; run with `python .github/scripts/gate.py tests`.""" diff --git a/.github/scripts/tests/test_gate.py b/.github/scripts/tests/test_gate.py new file mode 100644 index 0000000..2087bee --- /dev/null +++ b/.github/scripts/tests/test_gate.py @@ -0,0 +1,186 @@ +"""Tests for the repository gate. + +These exercise the checkers against synthetic trees, so a green run means the +gate still catches the failures its documentation names. They run as the +`tests` arm and as part of `fast`. +""" + +from __future__ import annotations + +import json +import sys +import tempfile +import unittest +from pathlib import Path + +SCRIPTS = Path(__file__).resolve().parents[1] +ROOT = SCRIPTS.parents[1] +if str(SCRIPTS) not in sys.path: + sys.path.insert(0, str(SCRIPTS)) + +import archive_checks +import gate +import gnn_conformance + + +class TreeCase(unittest.TestCase): + """Base case providing a throwaway repository root.""" + + def setUp(self) -> None: + self._temp = tempfile.TemporaryDirectory() + self.root = Path(self._temp.name).resolve() + + def tearDown(self) -> None: + self._temp.cleanup() + + def write(self, relative: str, text: str) -> Path: + path = self.root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + return path + + +class LinkTargetTests(unittest.TestCase): + def test_bare_path(self) -> None: + self.assertEqual(archive_checks.link_target("docs/a.md"), "docs/a.md") + + def test_angle_brackets_and_title(self) -> None: + self.assertEqual( + archive_checks.link_target(" \"a title\""), "docs/a b.md" + ) + + def test_title_is_dropped(self) -> None: + self.assertEqual(archive_checks.link_target('a.md "title"'), "a.md") + + +class LinkCheckTests(TreeCase): + def test_resolving_and_ignored_targets(self) -> None: + self.write("docs/target.md", "# target\n") + page = self.write( + "docs/page.md", + "[ok](target.md)\n" + "[fragment](target.md#section)\n" + "[anchor](#section)\n" + "[external](https://example.org/x)\n" + "[mail](mailto:a@example.org)\n", + ) + self.assertEqual(archive_checks.check_links([page], self.root), []) + + def test_missing_target_is_reported(self) -> None: + page = self.write("docs/page.md", "[gone](does-not-exist.md)\n") + errors = archive_checks.check_links([page], self.root) + self.assertEqual(len(errors), 1) + self.assertIn("docs/page.md: missing link target: does-not-exist.md", errors[0]) + + def test_escaping_target_is_reported(self) -> None: + outside = self.root.parent / f"{self.root.name}-outside.md" + outside.write_text("# outside\n", encoding="utf-8") + try: + page = self.write("page.md", "[out](../outside.md)\n") + errors = archive_checks.check_links([page], self.root) + self.assertEqual(len(errors), 1) + self.assertIn("link escapes repository", errors[0]) + finally: + outside.unlink() + + def test_non_markdown_files_are_skipped(self) -> None: + script = self.write("docs/note.txt", "[gone](does-not-exist.md)\n") + self.assertEqual(archive_checks.check_links([script], self.root), []) + + +class PythonCheckTests(TreeCase): + def test_valid_and_invalid_sources(self) -> None: + good = self.write("a.py", "def ok() -> int:\n return 1\n") + bad = self.write("b.py", "def broken(:\n") + errors = archive_checks.check_python([good, bad], self.root) + self.assertEqual(len(errors), 1) + self.assertIn("b.py: Python syntax error", errors[0]) + + +class NotebookCheckTests(TreeCase): + def test_output_free_notebook_passes(self) -> None: + notebook = self.write( + "n.ipynb", + json.dumps({"cells": [{"cell_type": "code", "outputs": []}], "nbformat": 4}), + ) + self.assertEqual(archive_checks.check_notebooks([notebook], self.root), []) + + def test_committed_outputs_are_reported(self) -> None: + notebook = self.write( + "n.ipynb", + json.dumps({"cells": [{"cell_type": "code", "outputs": [{"a": 1}]}]}), + ) + errors = archive_checks.check_notebooks([notebook], self.root) + self.assertEqual(len(errors), 1) + self.assertIn("cell 0 carries committed outputs", errors[0]) + + def test_invalid_json_is_reported(self) -> None: + notebook = self.write("n.ipynb", "{not json") + errors = archive_checks.check_notebooks([notebook], self.root) + self.assertEqual(len(errors), 1) + self.assertIn("invalid notebook JSON", errors[0]) + + +class CheckpointCheckTests(TreeCase): + def test_nested_checkpoint_is_reported(self) -> None: + tracked = [ + self.write("a/.ipynb_checkpoints/n-checkpoint.ipynb", "{}"), + self.write("b/n.ipynb", "{}"), + ] + errors = archive_checks.check_checkpoints(tracked, self.root) + self.assertEqual(len(errors), 1) + self.assertIn("a/.ipynb_checkpoints/n-checkpoint.ipynb", errors[0]) + + +class PinTests(unittest.TestCase): + def test_repository_pin_satisfies_the_contract(self) -> None: + pin = gnn_conformance.load_pin(ROOT) + self.assertEqual(gnn_conformance.validate_pin(pin), []) + self.assertTrue(gnn_conformance.REVISION.match(pin["revision"])) + self.assertIn( + f"git+https://github.com/{pin['repository']}.git@{pin['revision']}", + gnn_conformance.install_spec(pin), + ) + + def test_broken_pins_are_rejected(self) -> None: + pin = gnn_conformance.load_pin(ROOT) + cases = { + "schema": {**pin, "schema": "gnn-importer-pin/v2"}, + "repository": {**pin, "repository": "someone/else"}, + "tag": {**pin, "tag": "5.0.3"}, + "revision": {**pin, "revision": pin["revision"][:12]}, + "recorded": {**pin, "recorded": "12-11-2026"}, + } + for field, broken in cases.items(): + with self.subTest(field=field): + self.assertTrue( + any(field in error for error in gnn_conformance.validate_pin(broken)), + f"{field} mutation was accepted", + ) + + +class CommandContractTests(unittest.TestCase): + """CI must invoke the same commands the documentation names.""" + + def test_workflow_calls_the_documented_arms(self) -> None: + workflow = (ROOT / ".github" / "workflows" / "ci.yml").read_text(encoding="utf-8") + self.assertIn(gate.command_for("fast"), workflow) + self.assertIn(f"{gate.command_for('gnn')} --install", workflow) + + def test_every_arm_has_a_description_and_command(self) -> None: + self.assertEqual(set(gate.ARM_ORDER), set(gate.DESCRIPTIONS)) + for arm in gate.ARM_ORDER: + self.assertTrue(gate.command_for(arm).startswith("python .github/scripts/")) + + def test_timings_page_lists_every_measured_arm(self) -> None: + page = ROOT / gate.TIMINGS_DOC + self.assertTrue(page.is_file(), f"{gate.TIMINGS_DOC} is missing") + text = page.read_text(encoding="utf-8") + self.assertIn(gate.TIMINGS_MARKER, text) + for arm in gate.timings_arms(): + self.assertIn(f"| `{arm}` | `{gate.command_for(arm)}` |", text) + self.assertNotIn(f"| `gnn` | `{gate.command_for('gnn')}` |", text) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 285e4db..e884347 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -13,21 +13,28 @@ concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true +# Both jobs run the repository gate entry point, `.github/scripts/gate.py`, so +# the arm bodies live in that script and not in this workflow. Each job keeps +# its existing name because these job names are the status checks on `main`. +# +# The conformance job reads `.github/gnn-pin.json` (schema +# `gnn-importer-pin/v1`); `docs/RUN_GUIDE.md` documents the arms, job commands +# and pins, and `.github/GNN.md` the conformance semantics. + jobs: docs-and-scripts: - timeout-minutes: 60 name: docs and scripts runs-on: ubuntu-latest + timeout-minutes: 10 steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Set up Python uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0 with: python-version: "3.12" - - name: Check links, Python syntax, notebooks, and checkpoints - run: python .github/scripts/check_archive.py + - name: Gate — links, Python syntax, notebooks, checkpoints, gate tests + run: python .github/scripts/gate.py fast - # GNN pin: .github/gnn-pin.json (v5.0.3, 8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1) gnn-conformance: name: gnn conformance (pinned) runs-on: ubuntu-latest @@ -38,96 +45,5 @@ jobs: uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0 with: python-version: "3.12" - - name: Install pinned GNN parser - run: | - python -m pip install --disable-pip-version-check defusedxml psutil pyyaml - python - <<'PY' - import json - import subprocess - import sys - from pathlib import Path - - pin = json.loads(Path(".github/gnn-pin.json").read_text(encoding="utf-8")) - assert pin["repository"] == "ActiveInferenceInstitute/Generalized_Notation_Notation" - assert pin["tag"] == "v5.0.3" - assert pin["revision"] == "8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1" - spec = ( - "generalized-notation-notation @ " - f"git+https://github.com/{pin['repository']}.git@{pin['revision']}" - ) - subprocess.check_call( - [sys.executable, "-m", "pip", "install", "--disable-pip-version-check", "--no-deps", spec] - ) - PY - - name: Verify pinned parser and repository sketches - run: | - python - <<'PY' - import json - import shutil - import tempfile - import urllib.request - from pathlib import Path - - from gnn import __version__ - from gnn.parsers.basic import parse_gnn_formal, validate_gnn_syntax - from gnn.type_checker import GNNTypeChecker - - pin = json.loads(Path(".github/gnn-pin.json").read_text(encoding="utf-8")) - expected_version = pin["tag"].removeprefix("v") - if __version__ != expected_version: - raise SystemExit(f"installed GNN {__version__!r} does not match {pin['tag']!r}") - - sketches = sorted(Path(".").rglob("*.gnn.md")) - if sketches: - for sketch in sketches: - parsed = parse_gnn_formal(sketch) - if parsed is None or getattr(parsed, "parse_errors", None): - raise SystemExit(f"{sketch} did not parse cleanly") - valid, errors = validate_gnn_syntax(sketch.read_text(encoding="utf-8")) - if not valid: - raise SystemExit(f"{sketch} syntax errors: {errors!r}") - with tempfile.TemporaryDirectory(prefix="repo-gnn-") as output_dir: - model_dir = Path(output_dir) / "models" - model_dir.mkdir() - for index, sketch in enumerate(sketches): - shutil.copy2(sketch, model_dir / f"{index:03d}_{sketch.name}") - if not GNNTypeChecker().validate_gnn_files(model_dir, Path(output_dir)): - raise SystemExit("GNN type checker rejected the repository sketches") - checked = len(sketches) - source = "repository" - else: - exemplar_url = ( - "https://raw.githubusercontent.com/ActiveInferenceInstitute/" - "Generalized_Notation_Notation/" - f"{pin['revision']}/input/gnn_files/basics/static_perception.md" - ) - with tempfile.TemporaryDirectory(prefix="pinned-exemplar-") as output_dir: - exemplar = Path(output_dir) / "static_perception.md" - urllib.request.urlretrieve(exemplar_url, exemplar) - parsed = parse_gnn_formal(exemplar) - if parsed is None or getattr(parsed, "parse_errors", None): - raise SystemExit("pinned exemplar did not parse cleanly") - valid, errors = validate_gnn_syntax(exemplar.read_text(encoding="utf-8")) - if not valid: - raise SystemExit(f"pinned exemplar syntax errors: {errors!r}") - model_dir = Path(output_dir) / "models" - model_dir.mkdir() - shutil.copy2(exemplar, model_dir / exemplar.name) - if not GNNTypeChecker().validate_gnn_files(model_dir, Path(output_dir)): - raise SystemExit("GNN type checker rejected the pinned exemplar") - checked = 1 - source = "pinned-exemplar" - - print( - json.dumps( - { - "gnn-version": __version__, - "pin-revision": pin["revision"], - "models": checked, - "source": source, - "result": "verified", - }, - sort_keys=True, - ) - ) - PY + - name: Gate — install and verify the pinned GNN parser + run: python .github/scripts/gate.py gnn --install diff --git a/AGENTS.md b/AGENTS.md index 31b75ea..236ef91 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,17 +25,23 @@ Knowledge-Engineering is a public research-data and exploratory tooling archive. ## Verification -The repository gate is dependency-free and must pass before publishing: +The repository gate is one entry point with individually runnable arms, and it +must pass before publishing: ```bash -python .github/scripts/check_archive.py +python .github/scripts/gate.py fast ``` -It checks relative Markdown links and anchors, compiles tracked Python files, -rejects notebooks that are not output-free JSON, and rejects tracked Jupyter -checkpoints. For Python edits, also run `python -m py_compile` on changed -scripts. For notebook edits, verify JSON validity and compare code-cell sources -before and after output cleanup. For documentation edits, check relative links -and ensure dates/status claims match repository contents. +`fast` runs the four archive checks over one `git ls-files` read (relative +Markdown links resolve, tracked Python compiles, notebooks are output-free JSON, +no Jupyter checkpoint is tracked) plus the gate's own tests. CI calls exactly +this command, and separately +`python .github/scripts/gate.py gnn --install` for the pinned GNN conformance. +Arms, job keys, the pin, and measured wall times are in +[`docs/RUN_GUIDE.md`](docs/RUN_GUIDE.md). For Python edits, also run +`python -m py_compile` on changed scripts. For notebook edits, verify JSON +validity and compare code-cell sources before and after output cleanup. For +documentation edits, check relative links and ensure dates/status claims match +repository contents. See `CONTRIBUTING.md` for public contribution guidance and `TO-DO.md` for the current audit state. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 11a09a7..d490ac9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -25,6 +25,18 @@ Thank you for improving this research archive. ## Pull requests +Run the repository gate before opening a pull request; CI runs the same +commands: + +```bash +python .github/scripts/gate.py fast +``` + +Add the GNN arm (`python .github/scripts/gate.py gnn --install`) when you change +`.github/gnn-pin.json` or the conformance script. Arms and their measured wall +times are in [docs/RUN_GUIDE.md](docs/RUN_GUIDE.md) and +[docs/gate-timings.md](docs/gate-timings.md). + Describe the files changed, the verification performed, and any remaining limitations. Keep data provenance clear: include source, query date, filters, and transformations for newly generated exports where known. ## License diff --git a/Literature_Analysis/README.md b/Literature_Analysis/README.md index 5721d07..7e34382 100644 --- a/Literature_Analysis/README.md +++ b/Literature_Analysis/README.md @@ -53,9 +53,12 @@ normalize or silently regenerate committed exports during unrelated work. ## Gate -The repository's dependency-free archive gate checks relative links, Python -syntax, notebook JSON and outputs, and Jupyter checkpoints: +The repository gate is one script with separately runnable arms; the default +`fast` arm checks relative links, Python syntax, notebook JSON and outputs, +Jupyter checkpoints, and the gate's own tests: ```bash -python .github/scripts/check_archive.py +python .github/scripts/gate.py fast ``` + +See [docs/RUN_GUIDE.md](../docs/RUN_GUIDE.md) for every arm. diff --git a/README.md b/README.md index f070af1..cb22140 100644 --- a/README.md +++ b/README.md @@ -43,14 +43,16 @@ The notebook uses `pandas` and Biopython's `Bio.Entrez` client. Open `Literature The PDF scraper expects a local PDF corpus at `Literature_Analysis/PDF_Scrape/resc/pdfList/`, which is not included in this repository. Its parameter files and invocation details are documented in the [PDF_Scrape README](Literature_Analysis/PDF_Scrape/README.md). -### Verify the archive +### Verify the repository -The repository's own gate is dependency-free. It fails when a relative Markdown link does not resolve, a tracked Python file does not compile, a notebook is invalid JSON or carries committed cell outputs, or a Jupyter checkpoint file is tracked: +One gate script, runnable arm by arm. The default `fast` arm is dependency-free. It fails when a relative Markdown link does not resolve, a tracked Python file does not compile, a notebook is invalid JSON or carries committed cell outputs, a Jupyter checkpoint file is tracked, or the gate's own tests fail: ```bash -python .github/scripts/check_archive.py +python .github/scripts/gate.py fast ``` +List every arm with `python .github/scripts/gate.py --list`; the network-backed `gnn` conformance arm and the measured wall time for each arm are documented in [docs/RUN_GUIDE.md](docs/RUN_GUIDE.md) and [docs/gate-timings.md](docs/gate-timings.md). + ## Documentation The [documentation hub](docs/) provides a [quickstart](docs/quickstart.md), a [data provenance inventory](docs/provenance.md), and an [architecture overview](docs/architecture.md) alongside the area READMEs above. diff --git a/TO-DO.md b/TO-DO.md index d479004..04179f9 100644 --- a/TO-DO.md +++ b/TO-DO.md @@ -1,6 +1,6 @@ # TO-DO — Knowledge-Engineering documentation pass -Last reviewed: 2026-11-02 (canonical-ontology pointer pass) after 2026-08-02 (docs-deep review; see `REVIEW_LOG_2026-08-02.md`) +Last reviewed: 2026-11-12 (composable gate entry point) after 2026-11-02 (canonical-ontology pointer pass) and 2026-08-02 (docs-deep review; see `REVIEW_LOG_2026-08-02.md`) Sections: - **Minor** — typo, broken link, formatting, small cleanups. @@ -28,6 +28,7 @@ Sections: - [x] **Improve `.aii/config.yaml` description** — `repo.description` was a bare Zenodo URL + DOI; replaced with a human-readable description and registered all top-level and area doc artifacts. ✓ commits `94ff9a7`, `2c2660d` - [x] **Harden PDF_Scrape entrypoint behavior** — `streamingExport.py` prompted at import time and `manager.py` executed at import. Now: lazy export configuration (import-safe), `if __name__ == "__main__"` guard, and failure logs include the exception message. Verified end-to-end in a throwaway venv (257-record run via stdin). ✓ commit `5bb023f` (fix: guard manager entrypoint and make streaming export configuration lazy) - [x] **Point at the canonical ontology home** — `Ontology_Snapshot/` is a frozen December 2022 export, so the README, `docs/provenance.md`, and the area README now name [`Active_Inference_Ontology`](https://github.com/ActiveInferenceInstitute/Active_Inference_Ontology) as the published home: current v5 release (May 25, 2023, 429 terms) with stable identities and typed relations, the archived December 12, 2022 snapshot of the same 411-row export, and the GNN vocabulary bridge for `ActInfOntologyAnnotation` labels. 402 of the 408 terms in the snapshot appear verbatim in the v5 release the canonical repository publishes. The snapshot CSVs were not touched. Verified: the canonical paths exist on that repository's `main` branch, the term overlap was measured from the committed CSVs, and every relative link in the edited files resolves. ✓ commit `fbaefeb` +- [x] **Compose the gate into one entry point** — the archive checks moved into `.github/scripts/archive_checks.py` and the GNN job body moved out of `ci.yml` into `.github/scripts/gnn_conformance.py`; `.github/scripts/gate.py` is now the only command line, with separately runnable arms (`links`, `python`, `notebooks`, `checkpoints`, `archive`, `tests`, `gnn`, `fast`, `full`, `timings`). Both CI jobs run one arm each, so no check body is duplicated in the workflow, and the pin is validated against schema `gnn-importer-pin/v1` in code instead of two inline heredocs. The gate gained tests (`.github/scripts/tests/test_gate.py`) and a generated wall-time page (`docs/gate-timings.md`); `docs/RUN_GUIDE.md` and `docs/documentation-index.md` document them. ✓ 2026-11-12 ## Major diff --git a/docs/README.md b/docs/README.md index 2a642d9..9b13d34 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,14 +2,21 @@ This directory holds the repository-level guides for the archive. Start with [index.md](index.md), then follow the area READMEs linked there. +[documentation-index.md](documentation-index.md) lists every long-lived page in +one flat index. - [index.md](index.md) — hub and reference map. +- [RUN_GUIDE.md](RUN_GUIDE.md) — the gate entry point, its arms, the CI jobs, and + the GNN pin. +- [gate-timings.md](gate-timings.md) — per-arm wall times, generated by the gate. - [quickstart.md](quickstart.md) — browse the archive, run the PubMed notebook, and run PDF keyword extraction. - [provenance.md](provenance.md) — per-dataset inventory: formats, dates, sizes, and caveats. - [architecture.md](architecture.md) — how the literature-analysis components fit together and cross-reference each other. +- [documentation-index.md](documentation-index.md) — flat index of long-lived + documentation. This is a documentation hub over a research-data archive: there is no build step or package to install. The canonical per-area documentation remains the README @@ -21,15 +28,22 @@ license, and [AGENTS.md](../AGENTS.md) for contribution and preservation rules. ## Gate failure mode [`.github/workflows/ci.yml`](../.github/workflows/ci.yml) runs -[`.github/scripts/check_archive.py`](../.github/scripts/check_archive.py) on -every pull request and on pushes to `main`. The gate fails, with exit code 1, -when a relative link does not resolve, a tracked Python file does not compile, a -notebook is not valid JSON, a notebook carries committed outputs, or an -`.ipynb_checkpoints` directory is tracked. The proof is run 41719405985, job -`docs and scripts`: the log line +[`.github/scripts/gate.py`](../.github/scripts/gate.py) on every pull request and +on pushes to `main`. Job `docs and scripts` runs the fast arm, +`python .github/scripts/gate.py fast`, which checks that relative Markdown links +resolve, tracked Python compiles, notebooks are output-free JSON, no +`.ipynb_checkpoints` directory is tracked, and the gate's own tests pass. It +exits 1 and lists every failure when any of those break. + +The proof is run 41719405985, job `docs and scripts`: the log line `- docs/README.md: missing link target: does-not-exist.md` is followed by -`##[error]Process completed with exit code 1.` Reproduce it locally with -`python .github/scripts/check_archive.py`. The workflow also runs the pinned GNN -conformance baseline (`gnn conformance (pinned)`, see `.github/GNN.md`): it -verifies the parser pin and, as this archive carries no `.gnn.md` model file, -validates the pinned upstream exemplar. +`##[error]Process completed with exit code 1.`. Reproduce it locally with +`python .github/scripts/gate.py links`. + +The workflow also runs the pinned GNN conformance baseline, job +`gnn conformance (pinned)`, as +`python .github/scripts/gate.py gnn --install`; it verifies the parser pin and, +as this archive carries no `.gnn.md` model file, validates the pinned upstream +exemplar. [RUN_GUIDE.md](RUN_GUIDE.md) lists every arm, and `.github/GNN.md` +records what the conformance arm parses. Wall times for each arm are in the +generated [gate-timings.md](gate-timings.md). diff --git a/docs/RUN_GUIDE.md b/docs/RUN_GUIDE.md new file mode 100644 index 0000000..8d06a04 --- /dev/null +++ b/docs/RUN_GUIDE.md @@ -0,0 +1,132 @@ +# Gate and CI run guide + +One entry point, `.github/scripts/gate.py`, owns every repository check. The +workflow references it and nothing else, so a local run and a CI run execute the +same command. Everything below is copied from the script's `--list` and from +`.github/workflows/ci.yml`; if they disagree, the code is right and this page is +stale. + +```console +$ python .github/scripts/gate.py --list +``` + +## Arms + +| Arm | Command | What it checks | +| --- | --- | --- | +| `links` | `python .github/scripts/gate.py links` | Relative Markdown link and image targets resolve inside the repository. | +| `python` | `python .github/scripts/gate.py python` | Tracked Python files compile under the running interpreter. | +| `notebooks` | `python .github/scripts/gate.py notebooks` | Tracked notebooks are valid JSON with no committed cell outputs. | +| `checkpoints` | `python .github/scripts/gate.py checkpoints` | No Jupyter checkpoint file is tracked. | +| `archive` | `python .github/scripts/gate.py archive` | The four archive checks over one `git ls-files` read. | +| `tests` | `python .github/scripts/gate.py tests` | The gate's own dependency-free unit tests. | +| `gnn` | `python .github/scripts/gate.py gnn --install` | The pinned GNN parser, sketch parsing, and type checking. | +| `fast` (default) | `python .github/scripts/gate.py fast` | `archive` + `tests`; CI's docs job runs exactly this. | +| `full` | `python .github/scripts/gate.py full` | `fast` + `gnn`; the whole gate. | +| `timings` | `python .github/scripts/gate.py timings` | Measure each offline arm once and rewrite the timings page. | + +`links`, `python`, `notebooks`, `checkpoints`, and `archive` share one +implementation in `.github/scripts/archive_checks.py`; running `archive` reads +`git ls-files` once instead of four times. `gnn` lives in +`.github/scripts/gnn_conformance.py` and is the only arm that needs the network. + +## Where the arms are measured + +[`gate-timings.md`](gate-timings.md) is generated; never edit it by hand. It +records one local sample per offline arm on the host that ran + +```bash +python .github/scripts/gate.py timings --include-gnn +``` + +`--include-gnn` adds the `gnn` measurement and needs the pinned parser already +importable in the measuring interpreter. `gnn` and `full` never appear in the +table, because both need the pinned parser and the network; `--include-gnn` +records `gnn` alone in the page's separate section, and `full` adds nothing a +reader cannot get by adding the fast table to that number. The `tests` arm fails +when the page stops listing every measured arm with its current command, so a +new arm cannot land without refreshing it. + +## CI + +[`.github/workflows/ci.yml`](../.github/workflows/ci.yml) has two jobs. Neither +job contains a check body of its own. + +| Job key | Display name | Command | +| --- | --- | --- | +| `docs-and-scripts` | `docs and scripts` | `python .github/scripts/gate.py fast` | +| `gnn-conformance` | `gnn conformance (pinned)` | `python .github/scripts/gate.py gnn --install` | + +Both jobs check out the repository and install CPython 3.12. The job keys and +display names are unchanged from the previous workflow, because the failure +evidence recorded in [`docs/README.md`](README.md#gate-failure-mode) and +`.github/GNN.md` cites `docs and scripts` and `gnn conformance (pinned)`; `main` +carries no branch protection, so no required status check constrains them. The +two jobs stay separate so a pinned-parser install failure cannot mask the fast +archive verdict, and so a GNN outage cannot fail the fast lane. + +The interpreter pin and the dependency installs need no project environment: +this repository has no `pyproject.toml`, `requirements.txt`, or lockfile, so +each job uses `actions/setup-python` pinned by commit. The Active Inference +Institute composite action +`ActiveInferenceInstitute/.github/.github/actions/setup-python-uv@7159e67315330aac8e414aebae0ad1e999a41448` +exists and is the organisation's pinned way to install uv plus CPython, but this +workflow does not use it: there is no uv project to sync, so it would add a +toolchain the gate never calls. Adopt it (with `python-version: "3.12"`) if an +arm gains a locked dependency set. + +## The GNN pin + +| Fact | Value | +| --- | --- | +| Pin file | `.github/gnn-pin.json` | +| Schema | `gnn-importer-pin/v1` | +| Repository | `ActiveInferenceInstitute/Generalized_Notation_Notation` | +| Tag | `v5.0.3` | +| Revision | `8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1` | +| Recorded | `2026-11-12` | + +`.github/GNN.md` records what the conformance arm parses. The arm validates the +schema and repository identity, installs +`generalized-notation-notation @ git+https://github.com/.git@` +with `--no-deps` plus the runtime dependencies `defusedxml`, `psutil`, and +`pyyaml`, then checks that the installed `gnn.__version__` equals the tag +without its `v`. Parsing then covers every `*.gnn.md` sketch in the repository +(full parse, syntax validation, type checking). This archive carries none, so +the arm validates the pinned upstream exemplar +`input/gnn_files/basics/static_perception.md` at the pinned revision and prints +a JSON receipt naming `"source": "pinned-exemplar"`. + +### Running the GNN arm locally + +The pinned release declares Python `>=3.11,<3.15`; pip refuses to install it on +a newer interpreter (`3.15.2 not in '<3.15,>=3.11'`). CI uses 3.12. Use a +throwaway environment at a supported version so the arm does not modify your +default interpreter: + +```bash +python3.12 -m venv ~/.cache/gnn-venv +~/.cache/gnn-venv/bin/python .github/scripts/gate.py gnn --install +``` + +Both steps need network access to PyPI and GitHub. + +## Why no lint or type arm + +The repository has no lint, type-check or packaging configuration: its Python is +six exploratory scripts under `Literature_Analysis/PDF_Scrape/` plus this gate's +own modules and tests, and it ships no package. The archive checks are the only +static analysis it defines, so `links`/`python`/`notebooks`/`checkpoints` are its +lint surface. Adding Ruff or mypy without a project to configure them for would +report on files whose execution depends on an uncommitted PDF corpus, not on this +repository's contract. If the scraper becomes an installable package, add a +`pyproject.toml` first and then a `lint` arm that CI calls by the same name. + +## What the gate does not prove + +- External URLs are never fetched; a dead external link passes `links`. +- `notebooks` checks JSON structure and output emptiness, not that a notebook + runs; the PubMed notebook needs `pandas`, `biopython`, and network Entrez + access, none of which this host or CI provides here. +- `gnn` proves parser and pin conformance. With no committed sketch it says + nothing about a model of this repository's own. diff --git a/docs/documentation-index.md b/docs/documentation-index.md new file mode 100644 index 0000000..bc668c9 --- /dev/null +++ b/docs/documentation-index.md @@ -0,0 +1,51 @@ +# Documentation index + +Flat index of this repository's long-lived documentation. `README.md` in each +directory is that directory's user-facing guide; this page lists them in one +place. Generated pages are marked as such and are refreshed by a script. + +## Entry points + +- [README.md](../README.md) — repository overview, contents table, citation, license. +- [AGENTS.md](../AGENTS.md) — agent-facing layout, change rules, verification. +- [CONTRIBUTING.md](../CONTRIBUTING.md) — contribution, provenance, and data-preservation rules. +- [CITATION.cff](../CITATION.cff) — machine-readable citation metadata (Zenodo DOI). + +## Documentation hub + +- [docs/README.md](README.md) — hub: guides, gate, and the recorded gate failure mode. +- [docs/index.md](index.md) — the same map rendered as guide and reference tables. +- [docs/RUN_GUIDE.md](RUN_GUIDE.md) — gate entry point, arms, CI jobs, GNN pin. +- [docs/gate-timings.md](gate-timings.md) — **generated** per-arm wall times. + +## Guides + +- [docs/quickstart.md](quickstart.md) — browse the archive, run the PubMed notebook, run the PDF extractor. +- [docs/provenance.md](provenance.md) — per-dataset inventory: formats, dates, sizes, caveats. +- [docs/architecture.md](architecture.md) — how the literature-analysis components relate. + +## Area guides + +- [Literature_Analysis/README.md](../Literature_Analysis/README.md) — folder-level guide to the four literature-analysis subareas. +- [Literature_Analysis/PDF_Scrape/README.md](../Literature_Analysis/PDF_Scrape/README.md) — keyword counting inputs, invocation, dependencies. +- [Literature_Analysis/Pubmed_to_Coda/README.md](../Literature_Analysis/Pubmed_to_Coda/README.md) — PubMed notebook, inputs, outputs, provenance. +- [Literature_Analysis/ResearchRabbit/README.md](../Literature_Analysis/ResearchRabbit/README.md) — ResearchRabbit export inventory. +- [Literature_Analysis/CitationSearch_PublishOrPerish/README.md](../Literature_Analysis/CitationSearch_PublishOrPerish/README.md) — Publish-or-Perish export inventory. +- [Ontology_Snapshot/README.md](../Ontology_Snapshot/README.md) — ontology term and definition snapshots. + +## Continuous integration + +- [.github/README.md](../.github/README.md) — workflow, gate scripts, Dependabot, pin file. +- [.github/GNN.md](../.github/GNN.md) — what the pinned GNN conformance arm validates. +- [.github/gnn-pin.json](../.github/gnn-pin.json) — the `gnn-importer-pin/v1` pin. + +## InstituteOS sidecar + +- [.aii/README.md](../.aii/README.md) — sidecar metadata overview. +- [.aii/docs/INTEGRATION.md](../.aii/docs/INTEGRATION.md) — integration notes and portable tasks. +- [.aii/docs/README.md](../.aii/docs/README.md) — sidecar documentation index. + +## Audit trail + +- [TO-DO.md](../TO-DO.md) — scoped findings and their completion state. +- [REVIEW_LOG_2026-08-02.md](../REVIEW_LOG_2026-08-02.md) — dated review and verification record. diff --git a/docs/gate-timings.md b/docs/gate-timings.md new file mode 100644 index 0000000..a359275 --- /dev/null +++ b/docs/gate-timings.md @@ -0,0 +1,19 @@ + +# Gate arm timings + +Wall time for one run of each arm on this host, measured 2026-11-12 with CPython 3.12.16 on Darwin arm64. These are single local samples, not CI guarantees: GitHub Actions uses different hardware and also pays for checkout and interpreter setup. + +| Arm | Command | Wall time | +| --- | --- | --- | +| `links` | `python .github/scripts/gate.py links` | 0.25 s | +| `python` | `python .github/scripts/gate.py python` | 0.21 s | +| `notebooks` | `python .github/scripts/gate.py notebooks` | 0.20 s | +| `checkpoints` | `python .github/scripts/gate.py checkpoints` | 0.14 s | +| `archive` | `python .github/scripts/gate.py archive` | 0.18 s | +| `tests` | `python .github/scripts/gate.py tests` | 0.38 s | +| `fast` | `python .github/scripts/gate.py fast` | 0.27 s | + +## Interpreter-specific arms + +`gnn` is measured separately because it installs third-party packages and reads the network, so its duration depends on the package cache and mirror. +With the pinned parser already installed it took 0.95 s for one conformance check, excluding the install step; the install step's duration depends on pip's cache. Refresh this page from an interpreter that has the parser installed to record it again. diff --git a/docs/index.md b/docs/index.md index 9a64c85..04b3b74 100644 --- a/docs/index.md +++ b/docs/index.md @@ -11,6 +11,9 @@ This is a documentation hub over a research-data archive — there is no build s | [Quickstart](quickstart.md) | How to browse the archive, run the PubMed notebook, and run the PDF keyword extractor. | | [Data provenance](provenance.md) | Every dataset in the repository: what it is, its format, size, date, and caveats. | | [Architecture](architecture.md) | How the components fit together: pipelines, module flow, and cross-references between areas. | +| [Run guide](RUN_GUIDE.md) | The single gate entry point, its arms, the CI job keys and commands, and the GNN pin. | +| [Gate arm timings](gate-timings.md) | Per-arm wall times, generated by `python .github/scripts/gate.py timings`. | +| [Documentation index](documentation-index.md) | Flat index of every long-lived document in the repository. | ## Reference