Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .aii/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Institute repositories without touching this project's own content. The schema i
| Task | Command | Working directory | Description |
| --- | --- | --- | --- |
| `inventory` | `git ls-files` | `.` | List tracked files. |
| `test` | `python .github/scripts/check_archive.py` | `.` | Run the dependency-free archive gate (links, syntax, notebooks). |
| `test` | `python .github/scripts/gate.py fast` | `.` | Run the default dependency-free gate arm (links, syntax, notebooks, gate tests). |

## Links

Expand Down
4 changes: 2 additions & 2 deletions .aii/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,8 @@ tasks:
cmd: git ls-files
description: List tracked files.
test:
cmd: python .github/scripts/check_archive.py
description: Run the dependency-free archive gate (links, syntax, notebooks).
cmd: python .github/scripts/gate.py fast
description: Run the dependency-free gate arm (links, syntax, notebooks, tests).
docs:
- README.md
- AGENTS.md
Expand Down
2 changes: 1 addition & 1 deletion .aii/docs/INTEGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
| Task | Command | Working directory | Description |
| --- | --- | --- | --- |
| `inventory` | `git ls-files` | `.` | List tracked files. |
| `test` | `python .github/scripts/check_archive.py` | `.` | Run the dependency-free archive gate (links, syntax, notebooks). |
| `test` | `python .github/scripts/gate.py fast` | `.` | Run the default dependency-free gate arm (links, syntax, notebooks, gate tests). |

## Validation

Expand Down
35 changes: 25 additions & 10 deletions .github/GNN.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,27 @@
# GNN conformance

`gnn conformance (pinned)` is defined in `.github/workflows/ci.yml` and reads
`.github/gnn-pin.json`, which pins
`ActiveInferenceInstitute/Generalized_Notation_Notation` at `v5.0.3`
(`8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1`).

The job verifies the pin and installed parser, parses every `*.gnn.md` sketch
in the repository, and type-checks them. When the repository has no sketch yet,
it parses the pinned upstream `input/gnn_files/basics/static_perception.md`
exemplar; this is a pipeline and pin conformance baseline, not a claim that the
repository already contains a model.
The gate command is `.github/scripts/gate.py gnn` (add `--install` to fetch the
pinned parser first). It is the same command the CI job
`gnn-conformance` — displayed as **`gnn conformance (pinned)`** — runs;
see [`docs/RUN_GUIDE.md`](../docs/RUN_GUIDE.md) for every arm.

The conformance arm reads `.github/gnn-pin.json` (`gnn-importer-pin/v1`), which
pins `ActiveInferenceInstitute/Generalized_Notation_Notation` at tag `v5.0.3`,
revision `8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1` (recorded `2026-11-12`).

What it validates, in order:

1. The pin's schema, repository identity, tag shape, 40-hex revision, and
recorded date.
2. The installed `gnn.__version__` against the tag without its `v`.
3. Every `*.gnn.md` sketch in the repository: full parse, syntax validation, and
type checking of the collected models. When the repository has no sketch it
downloads the pinned upstream `input/gnn_files/basics/static_perception.md`
exemplar at the pin's revision and validates that instead, and the JSON
receipt says `"source": "pinned-exemplar"`.

The arm needs network access and a Python version the pinned release supports
(`>=3.11,<3.15`); CI uses 3.12. The local recipe, including a throwaway virtual
environment, is in [`RUN_GUIDE.md`](../docs/RUN_GUIDE.md#running-the-gnn-arm-locally).
A green result is a pipeline and pin conformance baseline, not a claim that this
repository contains a model.
32 changes: 23 additions & 9 deletions .github/README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,29 @@
# `.github/`

- [`workflows/ci.yml`](workflows/ci.yml) — CI: the dependency-free archive gate
and the pinned GNN conformance job.
- [`scripts/check_archive.py`](scripts/check_archive.py) — the repository's
archive gate: relative Markdown links, tracked-Python syntax, notebook JSON
and committed outputs, and tracked Jupyter checkpoints.
- [`workflows/ci.yml`](workflows/ci.yml) — CI: two jobs, `docs and scripts`
(`python .github/scripts/gate.py fast`) and `gnn conformance (pinned)`
(`python .github/scripts/gate.py gnn --install`). The workflow holds no check
body of its own.
- [`scripts/gate.py`](scripts/gate.py) — the repository gate entry point: one
command per arm (`links`, `python`, `notebooks`, `checkpoints`, `archive`,
`tests`, `gnn`, `fast`, `full`, `timings`). Run
`python .github/scripts/gate.py --list`.
- [`scripts/archive_checks.py`](scripts/archive_checks.py) — the dependency-free
archive checks (relative Markdown links, tracked-Python compilation, notebook
JSON and committed outputs, tracked Jupyter checkpoints).
- [`scripts/gnn_conformance.py`](scripts/gnn_conformance.py) — pin validation,
pinned-parser install, and sketch/exemplar conformance.
- [`scripts/tests/test_gate.py`](scripts/tests/test_gate.py) — the gate's own
tests, run by the `tests` arm.
- [`dependabot.yml`](dependabot.yml) — dependency update configuration.
- [`gnn-pin.json`](gnn-pin.json) — the pinned Generalized Notation Notation
revision (v5.0.3, commit `8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1`) used by
the conformance job.
- [`GNN.md`](GNN.md) — what the pinned GNN conformance job checks.
- [`gnn-pin.json`](gnn-pin.json) — the `gnn-importer-pin/v1` pin used by the
conformance arm: repository, tag `v5.0.3`, revision
`8f293d9ba810e6eda5c09842d9c7ed9ffa5f92c1`, recorded `2026-11-12`.
- [`GNN.md`](GNN.md) — what the pinned GNN conformance arm parses.

Arms, job commands, and measured wall times are documented in
[`../docs/RUN_GUIDE.md`](../docs/RUN_GUIDE.md) and the generated
[`../docs/gate-timings.md`](../docs/gate-timings.md).

Repository change rules and conventions are in [`../AGENTS.md`](../AGENTS.md);
the documentation hub is [`../docs/index.md`](../docs/index.md).
149 changes: 149 additions & 0 deletions .github/scripts/archive_checks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
#!/usr/bin/env python3
"""Dependency-free archive checks for the Knowledge-Engineering research archive.

Four checks, each of which can fail:

1. ``links``: every relative Markdown link or image target resolves to a path
inside the repository.
2. ``python``: every tracked Python file compiles under the running Python.
3. ``notebooks``: every tracked notebook is valid JSON and carries no committed
cell outputs, matching the repository rule that notebook outputs stay out of
git.
4. ``checkpoints``: no Jupyter checkpoint file is tracked.

External URLs are not fetched, so the verdict does not depend on third-party
availability. This module holds the checks; `gate.py` is the command-line entry
point that runs them individually or together.
"""

from __future__ import annotations

import json
import re
from pathlib import Path
from subprocess import run as run_process
from urllib.parse import unquote

INLINE_LINK = re.compile(r"!?\[[^\]]*\]\(([^)]+)\)")
REFERENCE_LINK = re.compile(r"^\[[^\]]+\]:\s*(\S+)")
EXTERNAL_TARGET = re.compile(r"^[a-zA-Z][a-zA-Z0-9+.-]*:")
CHECKPOINT = re.compile(r"(^|/)\.ipynb_checkpoints/")


def tracked_files(root: Path) -> list[Path]:
"""Return every tracked path, resolved against ``root``."""
result = run_process(
["git", "-C", str(root), "ls-files", "-z"],
check=True,
capture_output=True,
)
return [root / name for name in result.stdout.decode("utf-8").split("\0") if name]


def link_target(raw: str) -> str:
"""Extract the destination from a Markdown link target, dropping any title."""
target = raw.strip()
if target.startswith("<") and ">" in target:
return target[1 : target.index(">")]
parts = target.split()
return parts[0] if parts else ""


def _relative(path: Path, root: Path) -> str:
try:
return path.relative_to(root).as_posix()
except ValueError:
return str(path)


def check_links(files: list[Path], root: Path) -> list[str]:
"""Flag relative link or image targets that do not resolve inside ``root``."""
errors: list[str] = []
resolved_root = root.resolve()
for path in files:
if path.suffix.lower() not in {".md", ".markdown"}:
continue
text = path.read_text(encoding="utf-8")
targets = INLINE_LINK.findall(text) + REFERENCE_LINK.findall(text)
for raw in targets:
target = unquote(link_target(raw))
if not target or target.startswith("#") or EXTERNAL_TARGET.match(target):
continue
base = target.split("#", 1)[0]
if not base:
continue
resolved = (path.parent / base).resolve()
try:
resolved.relative_to(resolved_root)
except ValueError:
errors.append(f"{_relative(path, root)}: link escapes repository: {raw}")
continue
if not resolved.exists():
errors.append(f"{_relative(path, root)}: missing link target: {raw}")
return errors


def check_python(files: list[Path], root: Path) -> list[str]:
"""Flag tracked Python files that do not compile."""
errors: list[str] = []
for path in files:
if path.suffix != ".py":
continue
source = path.read_text(encoding="utf-8")
try:
compile(source, str(path), "exec")
except SyntaxError as exc:
errors.append(f"{_relative(path, root)}: Python syntax error: {exc}")
return errors


def check_notebooks(files: list[Path], root: Path) -> list[str]:
"""Flag notebooks that are not valid JSON or carry committed cell outputs."""
errors: list[str] = []
for path in files:
if path.suffix != ".ipynb":
continue
try:
data = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
errors.append(f"{_relative(path, root)}: invalid notebook JSON: {exc}")
continue
if not isinstance(data, dict) or not isinstance(data.get("cells"), list):
errors.append(f"{_relative(path, root)}: notebook has no cells list")
continue
for index, cell in enumerate(data["cells"]):
if isinstance(cell, dict) and cell.get("outputs"):
errors.append(
f"{_relative(path, root)}: cell {index} carries committed outputs"
)
return errors


def check_checkpoints(files: list[Path], root: Path) -> list[str]:
"""Flag tracked Jupyter checkpoint files."""
return [
f"{_relative(path, root)}: Jupyter checkpoint file is tracked"
for path in files
if CHECKPOINT.search(path.as_posix())
]


CHECKS = {
"links": check_links,
"python": check_python,
"notebooks": check_notebooks,
"checkpoints": check_checkpoints,
}


def run(names: tuple[str, ...], root: Path) -> tuple[int, list[str]]:
"""Run the named checks over one ``git ls-files`` read.

Returns the number of tracked files inspected and the collected errors, so
callers can report a single consistent summary.
"""
files = tracked_files(root)
errors: list[str] = []
for name in names:
errors.extend(CHECKS[name](files, root))
return len(files), errors
139 changes: 0 additions & 139 deletions .github/scripts/check_archive.py

This file was deleted.

Loading
Loading