diff --git a/.claude/commands/ac_refresh_colab_contexts.md b/.claude/commands/ac_refresh_colab_contexts.md new file mode 120000 index 0000000..e0120da --- /dev/null +++ b/.claude/commands/ac_refresh_colab_contexts.md @@ -0,0 +1 @@ +../../skills/ac_refresh_colab_contexts.md \ No newline at end of file diff --git a/.claude/skills/ac_refresh_colab_contexts.md b/.claude/skills/ac_refresh_colab_contexts.md new file mode 120000 index 0000000..e0120da --- /dev/null +++ b/.claude/skills/ac_refresh_colab_contexts.md @@ -0,0 +1 @@ +../../skills/ac_refresh_colab_contexts.md \ No newline at end of file diff --git a/.codex/skills/autocti-assistant-ac-refresh-colab-contexts/SKILL.md b/.codex/skills/autocti-assistant-ac-refresh-colab-contexts/SKILL.md new file mode 100644 index 0000000..bad033d --- /dev/null +++ b/.codex/skills/autocti-assistant-ac-refresh-colab-contexts/SKILL.md @@ -0,0 +1,7 @@ +--- +name: autocti-assistant-ac-refresh-colab-contexts +description: "Maintainer upkeep for the link between this assistant's skills and the Colab notebooks of autocti_workspace. Runs the deterministic coverage check (`python -m autoassistant.colab_contexts check` plus PyAutoHands' `autohands gemini_context report`), then AI-scans what it flags — notebooks reached only by `default`, mapping entries that match nothing, missing or renamed skills, entries near the size cap — against the workspace script headers and the skill descriptions, proposes minimal `colab/contexts.yaml` edits, re-checks, re-stamps the workspace notebooks (the release does not regenerate them), and opens the matching PRs. Use after a skill is added, renamed or substantially resized, after the workspace gains or moves folders, and before a release. Not for user CTI questions, and never for editing the skills or the workspace scripts themselves." +--- + + +Read and follow the canonical skill: [SKILL](../../../skills/ac_refresh_colab_contexts.md). diff --git a/.github/workflows/colab-contexts.yml b/.github/workflows/colab-contexts.yml new file mode 100644 index 0000000..3b9ecb6 --- /dev/null +++ b/.github/workflows/colab-contexts.yml @@ -0,0 +1,36 @@ +# Colab-contexts check — colab/contexts.yaml stays loadable by PyAutoHands. +# +# PyAutoHands' `autohands gemini_context apply` stamps every autocti_workspace notebook +# with a Colab Gemini instruction assembled from colab/contexts.yaml (preface + core +# rules + the mapped skills), and fails if a mapped skill is missing or an instruction +# exceeds the mapping's cap (`cap:`, default 20,000). This job runs the same checks on +# every PR, so renaming a skill or growing one past the cap fails here, beside the +# change, rather than when the notebooks are next stamped. See +# autoassistant/colab_contexts.py. +name: colab-contexts + +on: + pull_request: + workflow_dispatch: + +permissions: + contents: read + +jobs: + colab-contexts: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install PyYAML + pytest + run: python -m pip install PyYAML pytest + + - name: Every mapped skill exists and every entry fits the cap + run: python -m autoassistant.colab_contexts check + + - name: Mapping tests + run: python -m pytest -q autoassistant/tests/test_colab_contexts.py diff --git a/Makefile b/Makefile index d6d70f0..e67abae 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: validate-literature-citations audit test +.PHONY: validate-literature-citations audit test colab-contexts validate-literature-citations: python -m autoassistant.literature validate-citations @@ -10,3 +10,7 @@ audit: # Assistant tooling test suite (slow: the gate tests import autocti per case). test: python -m pytest autoassistant/tests -q + +# colab/contexts.yaml: mapped skills exist and every entry fits the Gemini cap (`cap:` in the file). +colab-contexts: + python -m autoassistant.colab_contexts check diff --git a/autoassistant/colab_contexts.py b/autoassistant/colab_contexts.py new file mode 100644 index 0000000..faeefc4 --- /dev/null +++ b/autoassistant/colab_contexts.py @@ -0,0 +1,190 @@ +"""Check `colab/contexts.yaml` and stamp this repository's own Colab notebooks with it. + +`colab/contexts.yaml` maps workspace notebook paths to skills. PyAutoHands +(`autohands/build_util.py`, `inject_gemini_instructions`, run on autocti_workspace by +`autohands gemini_context apply`) assembles, for each notebook, the preface + core +rules + mapped skills and writes them into the notebook's metadata as its Colab Gemini instruction (`metadata.colab.aiContexts` + +`selectedAiContext`). Colab's Gemini cannot read this repository; the instruction is how +the skills reach it. + +This module is the assistant's side of that contract, self-contained (PyYAML + stdlib) +because the assistant does not depend on PyAutoHands: + +- ``check`` — every mapped skill exists, none is on the file's ``never_map`` list, and + every entry assembles to at most the mapping's cap (`cap:`, default 20,000). CI runs it on every PR + (``.github/workflows/colab-contexts.yml``), so a renamed skill or an over-long entry + fails here before it can fail a stamping run. +- ``apply`` — stamps one notebook, for notebooks this repository ships itself, keeping the file's JSON layout so the diff touches metadata only. + +The assembly and the instruction id mirror PyAutoHands' ``assemble_gemini_context`` and +``gemini_context_id``; change both or neither. + +Usage: + + python -m autoassistant.colab_contexts check + python -m autoassistant.colab_contexts apply path/to/notebook.ipynb \\ + --name autocti_workspace/notebooks/dataset_1d/modeling/start_here.ipynb +""" + +from __future__ import annotations + +import argparse +import fnmatch +import itertools +import json +import sys +import uuid +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +CONTEXTS_FILE = Path("colab") / "contexts.yaml" +CAP = 20_000 +# A mapping may set its own `cap:` (its skills may be longer) up to this ceiling, the +# size verified in Colab. Mirrors PyAutoHands' GEMINI_CONTEXT_CAP / GEMINI_CONTEXT_CAP_MAX. +CAP_MAX = 50_000 +_ID_URL = "https://github.com/PyAutoLabs/colab-gemini/{project}" + +# ``check`` sizes each entry with this stand-in notebook name, not the entry's glob: the +# rules text embeds the real path, and a glob can be shorter than the paths it matches. +# 240 characters is longer than any workspace notebook path (the longest today is ~110), +# so an entry that passes here cannot fail the release's per-notebook cap on path length. +_PROBE_NAME = "repository/" + "p" * (240 - len("repository/") - len(".ipynb")) + ".ipynb" + + +def load(root: Path = ROOT) -> dict: + import yaml + + return yaml.safe_load((Path(root) / CONTEXTS_FILE).read_text(encoding="utf-8")) + + +def context_id(project: str) -> str: + return str(uuid.uuid5(uuid.NAMESPACE_URL, _ID_URL.format(project=project))) + + +def match(contexts: dict, notebook_name: str) -> tuple[str, list[str]]: + """First matching entry wins (most specific first); else ``default``.""" + for entry in contexts["contexts"]: + if fnmatch.fnmatchcase(notebook_name, entry["match"]): + return entry["match"], list(entry["skills"]) + return "default", list(contexts["default"]["skills"]) + + +def assemble(contexts: dict, notebook_name: str, root: Path = ROOT) -> tuple[str, str]: + """``(entry, text)`` for ``notebook_name`` ("/").""" + label, skills = match(contexts, notebook_name) + repo, _, relpath = notebook_name.partition("/") + sections = [ + contexts["preface"].strip(), + contexts["rules"].replace("{notebook}", relpath).replace("{repo}", repo).strip(), + "# Reference material: assistant skills", + ] + for skill in skills: + text = (Path(root) / "skills" / f"{skill}.md").read_text(encoding="utf-8").strip() + sections.append(f"## Skill: {skill}\n\n{text}") + return label, "\n\n".join(sections) + "\n" + + +def cap_of(contexts: dict) -> int: + """The cap in force: the mapping's `cap:`, else ``CAP``. Raises outside (0, CAP_MAX].""" + cap = contexts.get("cap", CAP) + if isinstance(cap, bool) or not isinstance(cap, int) or not 0 < cap <= CAP_MAX: + raise ValueError(f"cap {cap!r} must be a positive integer no larger than {CAP_MAX}") + return cap + + +def check(root: Path = ROOT) -> list[str]: + """Problems with the mapping; empty when it is sound.""" + contexts = load(root) + problems = [] + for key in ("name", "project", "preface", "rules", "default", "contexts"): + if key not in contexts: + problems.append(f"missing top-level '{key}'") + if problems: + return problems + try: + cap = cap_of(contexts) + except ValueError as exc: + return [str(exc)] + never_map = set(contexts.get("never_map") or []) + entries = [("default", contexts["default"].get("skills") or [])] + [ + (e.get("match"), e.get("skills") or []) for e in contexts["contexts"] + ] + seen = set() + for label, skills in entries: + if not label or not skills: + problems.append(f"entry {label!r} needs both 'match' and 'skills'") + continue + if label in seen: + problems.append(f"entry {label!r} appears twice (the second can never match)") + seen.add(label) + missing = [s for s in skills if not (Path(root) / "skills" / f"{s}.md").is_file()] + if missing: + problems.append(f"entry {label!r} maps missing skill(s): {', '.join(missing)}") + continue + banned = sorted(never_map.intersection(skills)) + if banned: + problems.append(f"entry {label!r} maps never_map skill(s): {', '.join(banned)}") + size = len(_assemble_skills(contexts, _PROBE_NAME, skills, root)) + if size > cap: + problems.append(f"entry {label!r} assembles to {size} characters, over the {cap} cap") + return problems + + +def _assemble_skills(contexts: dict, name: str, skills: list[str], root: Path) -> str: + probe = dict(contexts, contexts=[{"match": "*", "skills": skills}]) + return assemble(probe, name, root)[1] + + +def dump_like(original_text: str, notebook: dict) -> str: + """Serialise ``notebook`` in the JSON layout ``original_text`` uses (metadata-only diffs).""" + original = json.loads(original_text) + newline = "\n" if original_text.endswith("\n") else "" + for indent, ensure_ascii, sort_keys in itertools.product((1, 2, 4), (False, True), (False, True)): + kwargs = dict(indent=indent, ensure_ascii=ensure_ascii, sort_keys=sort_keys) + if json.dumps(original, **kwargs) + newline == original_text: + return json.dumps(notebook, **kwargs) + newline + return json.dumps(notebook, indent=1, ensure_ascii=False, sort_keys=True) + "\n" + + +def apply(notebook_path: Path, notebook_name: str, root: Path = ROOT) -> str: + """Write the instruction for ``notebook_name`` into ``notebook_path``; returns the entry.""" + contexts = load(root) + label, text = assemble(contexts, notebook_name, root) + cap = cap_of(contexts) + if len(text) > cap: + raise ValueError(f"{notebook_name}: entry {label!r} assembles to {len(text)} characters, over {cap}") + original_text = Path(notebook_path).read_text(encoding="utf-8") + notebook = json.loads(original_text) + cid = context_id(contexts["project"]) + colab = notebook.setdefault("metadata", {}).setdefault("colab", {}) + colab.setdefault("aiContexts", {})[cid] = {"id": cid, "name": contexts["name"], "context": text} + colab["selectedAiContext"] = cid + new_text = dump_like(original_text, notebook) + if new_text != original_text: + Path(notebook_path).write_text(new_text, encoding="utf-8") + return label + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(prog="python -m autoassistant.colab_contexts") + sub = parser.add_subparsers(dest="command", required=True) + sub.add_parser("check", help="Validate colab/contexts.yaml") + p_apply = sub.add_parser("apply", help="Stamp one notebook") + p_apply.add_argument("notebook") + p_apply.add_argument("--name", required=True, help='"/"') + args = parser.parse_args(argv) + + if args.command == "check": + problems = check() + for problem in problems: + print(f"colab/contexts.yaml: {problem}", file=sys.stderr) + if not problems: + print(f"colab/contexts.yaml: OK ({len(load()['contexts'])} entries + default, cap {cap_of(load())})") + return 1 if problems else 0 + + print(f"{args.notebook}: {apply(Path(args.notebook), args.name)}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/autoassistant/tests/test_colab_contexts.py b/autoassistant/tests/test_colab_contexts.py new file mode 100644 index 0000000..214f6fa --- /dev/null +++ b/autoassistant/tests/test_colab_contexts.py @@ -0,0 +1,103 @@ +"""Pin `colab/contexts.yaml`: sound mapping, PyAutoHands-identical ids, metadata-only stamping. + +Stdlib + PyYAML, fast: nothing is executed. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from autoassistant import colab_contexts + +NOTEBOOK_NAME = "autocti_workspace/notebooks/dataset_1d/modeling/start_here.ipynb" + + +def test_mapping_is_sound(): + assert colab_contexts.check() == [] + + +def test_project_and_name(): + contexts = colab_contexts.load() + assert contexts["project"] == "autocti" + assert contexts["default"]["skills"] + + +def test_context_id_matches_pyautohands(): + # PyAutoHands' build_util.gemini_context_id("autocti") — the id every stamped + # autocti_workspace notebook carries. Literal, so a drift on either side fails here. + assert colab_contexts.context_id("autocti") == "a3d7c160-b952-56fb-b83a-321b7b858913" + + +def test_check_flags_never_map_and_over_cap(tmp_path): + root = tmp_path / "assistant" + (root / "colab").mkdir(parents=True) + (root / "skills").mkdir() + (root / "skills" / "small.md").write_text("small") + (root / "skills" / "maint.md").write_text("maint") + (root / "skills" / "huge.md").write_text("x" * colab_contexts.CAP) + (root / "colab" / "contexts.yaml").write_text( + "name: T\nproject: autocti\npreface: P\nrules: R {notebook} {repo}\n" + "never_map: [maint]\ndefault: {skills: [small]}\ncontexts:\n" + " - {match: a/*, skills: [maint]}\n - {match: b/*, skills: [huge]}\n" + " - {match: c/*, skills: [missing]}\n" + ) + problems = "\n".join(colab_contexts.check(root)) + assert "never_map" in problems and "over the" in problems and "missing" in problems + + +def test_apply_changes_metadata_only(tmp_path): + notebook = { + "cells": [{"cell_type": "markdown", "id": "cell-0", "metadata": {}, "source": ["Hi é"]}], + "metadata": {"colab": {"provenance": []}, "kernelspec": {"name": "python3"}}, + "nbformat": 4, + "nbformat_minor": 5, + } + path = tmp_path / "x.ipynb" + path.write_text(json.dumps(notebook, indent=1, ensure_ascii=False, sort_keys=True) + "\n") + colab_contexts.apply(path, NOTEBOOK_NAME) + first = path.read_text() + colab_contexts.apply(path, NOTEBOOK_NAME) # idempotent + assert path.read_text() == first + stamped = json.loads(first) + assert stamped["cells"] == notebook["cells"] + assert stamped["metadata"]["kernelspec"] == {"name": "python3"} + colab = stamped["metadata"]["colab"] + assert colab["provenance"] == [] + assert colab["selectedAiContext"] == "a3d7c160-b952-56fb-b83a-321b7b858913" + (entry,) = colab["aiContexts"].values() + assert NOTEBOOK_NAME.split("/", 1)[1] in entry["context"] + assert first.endswith("}\n") and "Hi é" in first + + +def _mini(tmp_path, cap_line=""): + root = tmp_path / "assistant" + (root / "colab").mkdir(parents=True) + (root / "skills").mkdir() + (root / "skills" / "mid.md").write_text("y" * 25_000) + (root / "colab" / "contexts.yaml").write_text( + f"{cap_line}\nname: T\nproject: p\npreface: P\nrules: R {{notebook}} {{repo}}\n" + "default: {skills: [mid]}\ncontexts: []\n" + ) + return root + + +def test_cap_defaults_to_20k(tmp_path): + root = _mini(tmp_path) + assert colab_contexts.cap_of(colab_contexts.load(root)) == colab_contexts.CAP == 20_000 + assert "over the 20000 cap" in "\n".join(colab_contexts.check(root)) + + +def test_custom_cap_honoured(tmp_path): + root = _mini(tmp_path, "cap: 30000") + assert colab_contexts.check(root) == [] + + +@pytest.mark.parametrize("cap", ["50001", "0", "-1", "lots", "true"]) +def test_cap_out_of_range_rejected(tmp_path, cap): + root = _mini(tmp_path, f"cap: {cap}") + assert "no larger than 50000" in "\n".join(colab_contexts.check(root)) + with pytest.raises(ValueError, match="no larger than 50000"): + colab_contexts.cap_of(colab_contexts.load(root)) diff --git a/colab/contexts.yaml b/colab/contexts.yaml new file mode 100644 index 0000000..2718cec --- /dev/null +++ b/colab/contexts.yaml @@ -0,0 +1,138 @@ +# Colab Gemini instructions — which assistant skills travel with which notebook. +# +# Colab's Gemini sidebar cannot read this repository, but it honours per-notebook +# instructions stored in the notebook's metadata (`metadata.colab.aiContexts` + +# `selectedAiContext`). The release does not generate autocti_workspace notebooks, so +# PyAutoHands' standalone injector writes one such instruction into each of them: +# +# autohands gemini_context apply /notebooks \ +# --project autocti --assistant-root +# +# Each instruction is assembled from this file: +# +# preface + rules (with {notebook} and {repo} filled in) + each mapped skill's text +# +# The skills stay here (single source); notebooks carry a snapshot only. +# +# Matching +# Each notebook is named "/", e.g. +# "autocti_workspace/notebooks/dataset_1d/modeling/start_here.ipynb". +# `match` is an fnmatch glob over that name (`*` also crosses `/`). +# The FIRST entry in `contexts` whose glob matches wins, so entries are written +# most specific first. A notebook no entry matches gets `default`. +# +# Limits +# Every assembled context must stay within `cap` (default 20,000 characters; this +# file sets none). Stamping fails, naming the entry, rather than truncating; +# `python -m autoassistant.colab_contexts check` (CI: .github/workflows/colab-contexts.yml) +# checks every entry here first. Skills listed under `never_map` (maintainer and +# agent-infrastructure skills that only make sense with the repository checked out) +# may not be mapped. +# +# Upkeep: skills/ac_refresh_colab_contexts.md — when a skill is added or renamed, or +# the workspace gains a folder, update the entries here, rerun the check and +# `autohands gemini_context report`, and re-apply to the workspace notebooks. + +name: PyAutoCTI assistant + +# The project key autocti_workspace notebooks are stamped with. It sets the +# instruction's id, so it matches the id PyAutoHands gives autocti_workspace. +project: autocti + +preface: | + These instructions were prepared by the PyAutoCTI assistant + (https://github.com/PyAutoLabs/autocti_assistant). The reference material at the + end was written for a coding agent working inside a checkout of that repository. + Here in Google Colab there is no checkout: ignore any step about repository files, + hooks, `scripts/` or `wiki/` paths, saving scripts, or terminal tools, and use the + PyAutoCTI knowledge in that material to help with this notebook. + +rules: | + You are helping someone work through a PyAutoCTI notebook ({notebook} in {repo}). + Explain what it does and help them adapt it to their own questions. + + Core rules: + 1. Stay with this notebook's API. Only use PyAutoCTI calls that appear in the + notebook or in the reference material below. Never invent class names, functions + or arguments. If you are unsure whether something exists, say so and suggest + checking with `dir()` or the PyAutoCTI docs (https://pyautocti.readthedocs.io). + 2. Real data → inspect before fitting. Before composing or running any model-fit on + real CCD data, ask the user to plot the dataset and its FPR / EPER regions, then + ask one question about artefacts that would bias the fit (cosmic rays, hot or bad + pixels, anything unexpected in the trails) and whether the FPR or the EPER should + be masked. Simulated data is exempt. This check is never waived. + 3. Never claim a fit or a cell ran, or what it returned. Only the user's own outputs + tell you that; ask them to share what they see. + 4. When the answer depends on something you do not know (their data, their goal, what + a cell printed), ask one focused question rather than guessing. + 5. For multi-step projects (calibrating a new instrument or CCD, many datasets, a + paper's analysis), point them to the PyAutoCTI assistant + https://github.com/PyAutoLabs/autocti_assistant, which runs in Claude Code or + Codex with its full skills and safety checks. + +never_map: + - _bootstrap_skill + - _style + - ac_audit_skill_apis + - ac_refresh_colab_contexts + - ac_setup_environment + - contribute-upstream + - feedback + - start-new-project + +default: + skills: [ac_workspace_navigation, ac_fit_cti_model] + +contexts: + + # --- overview tour --------------------------------------------------------------- + - match: autocti_workspace/notebooks/overview/overview_1_* + skills: [ac_simulate_dataset_1d, ac_plot] + - match: autocti_workspace/notebooks/overview/overview_2_* + skills: [ac_simulate_imaging_ci, ac_plot] + - match: autocti_workspace/notebooks/overview/overview_3_* + skills: [ac_simulate_dataset_1d, ac_plot] + - match: autocti_workspace/notebooks/overview/overview_4_* + skills: [ac_simulate_imaging_ci, ac_mask_and_extract] + - match: autocti_workspace/notebooks/overview/overview_5_* + skills: [ac_fit_cti_model, ac_plot] + - match: autocti_workspace/notebooks/overview/overview_6_* + skills: [ac_fit_cti_model, ac_load_results] + + # --- plot: plotting guides per dataset (ccd/, fpa/, diagnostics/, plotters/) ------- + - match: autocti_workspace/notebooks/plot/*/database* + skills: [ac_plot, ac_load_results] + - match: autocti_workspace/notebooks/plot/* + skills: [ac_plot] + + # --- dataset_1d advanced ----------------------------------------------------------- + - match: autocti_workspace/notebooks/dataset_1d/advanced/temporal/fit* + skills: [ac_fit_cti_model, ac_load_results] + - match: autocti_workspace/notebooks/dataset_1d/advanced/temporal/plot* + skills: [ac_load_results, ac_plot] + - match: autocti_workspace/notebooks/dataset_1d/advanced/temporal/* + skills: [ac_load_results] + - match: autocti_workspace/notebooks/*/advanced/database/* + skills: [ac_load_results] + + # --- imaging_ci advanced ----------------------------------------------------------- + - match: autocti_workspace/notebooks/imaging_ci/advanced/chaining/* + skills: [ac_fit_cti_model, ac_workspace_navigation] + + # --- per-geometry pipeline stages -------------------------------------------------- + - match: autocti_workspace/notebooks/dataset_1d/simulators/* + skills: [ac_simulate_dataset_1d] + - match: autocti_workspace/notebooks/imaging_ci/simulators/* + skills: [ac_simulate_imaging_ci] + - match: autocti_workspace/notebooks/imaging_ci/data_preparation/* + skills: [ac_mask_and_extract, ac_simulate_imaging_ci] + - match: autocti_workspace/notebooks/*/modeling/features/visualize_full* + skills: [ac_fit_cti_model, ac_plot] + - match: autocti_workspace/notebooks/*/modeling/* + skills: [ac_fit_cti_model, ac_mask_and_extract] + - match: autocti_workspace/notebooks/*/correction/* + skills: [ac_correct_cti] + - match: autocti_workspace/notebooks/*/results/* + skills: [ac_load_results, ac_plot] + - match: autocti_workspace/notebooks/*/extract.ipynb + skills: [ac_mask_and_extract] diff --git a/modes/maintainer.md b/modes/maintainer.md index be01583..446cb46 100644 --- a/modes/maintainer.md +++ b/modes/maintainer.md @@ -31,6 +31,7 @@ Use the existing skills, not new docs: - Authoring or evolving a skill → [`skills/_bootstrap_skill.md`](../skills/_bootstrap_skill.md). - Regenerating `wiki/core/` against pinned sources → `ac_update_wiki`. - API gate / version baseline → [`skills/ac_audit_skill_apis.md`](../skills/ac_audit_skill_apis.md). +- Colab Gemini skill mapping (`colab/contexts.yaml`) → [`skills/ac_refresh_colab_contexts.md`](../skills/ac_refresh_colab_contexts.md). ## Release-time wiki-currency check (two triggers, one check) @@ -95,8 +96,10 @@ scripts in `scripts/` (`*_cosmos_web_ring.py`, tied to a named lens; only `scripts/`'s own AGENTS/CLAUDE/README docs are generic). **Mixed** (structure generic, values domain-specific): `llms.txt` read-order, -`config/`, `benchmarks/README.md` (protocol generic, benchmark table domain), the -maintainer smoke tests below. +`config/`, `benchmarks/README.md` (protocol generic, benchmark table domain), +`colab/contexts.yaml` (preface and rules shape generic, skill mapping and CTI rules +domain; a newborn remaps it to its own skills and workspace), the maintainer smoke +tests below. **Per-clone data** (never copied to a newborn — each clone accumulates its own): `benchmarks/runs/` and the regenerated `benchmarks/RESULTS.md`. A newborn starts with diff --git a/skills/README.md b/skills/README.md index 71db980..070c1bf 100644 --- a/skills/README.md +++ b/skills/README.md @@ -69,6 +69,10 @@ grown in use. the symbol audit, the public-surface gate, the idiom deny-list, provenance and citation paths. Read it before regenerating the API baseline — it is the difference between a stale baseline and a genuinely stale doc. +- [`ac_refresh_colab_contexts.md`](./ac_refresh_colab_contexts.md) — maintainer + upkeep for `colab/contexts.yaml`, the skill-to-notebook mapping behind the Colab + Gemini instructions: run the check and the PyAutoHands coverage report, fix + default-only notebooks, dead entries and near-cap entries, and open a PR. ### The CTI calibration pipeline diff --git a/skills/ac_refresh_colab_contexts.md b/skills/ac_refresh_colab_contexts.md new file mode 100644 index 0000000..275e216 --- /dev/null +++ b/skills/ac_refresh_colab_contexts.md @@ -0,0 +1,265 @@ +--- +name: ac_refresh_colab_contexts +description: Maintainer upkeep for the link between this assistant's skills and the Colab notebooks of autocti_workspace. Runs the deterministic coverage check (`python -m autoassistant.colab_contexts check` plus PyAutoHands' `autohands gemini_context report`), then AI-scans what it flags — notebooks reached only by `default`, mapping entries that match nothing, missing or renamed skills, entries near the size cap — against the workspace script headers and the skill descriptions, proposes minimal `colab/contexts.yaml` edits, re-checks, re-stamps the workspace notebooks (the release does not regenerate them), and opens the matching PRs. Use after a skill is added, renamed or substantially resized, after the workspace gains or moves folders, and before a release. Not for user CTI questions, and never for editing the skills or the workspace scripts themselves. +--- + +# Refreshing the Colab Gemini skill map + +Every notebook in `autocti_workspace` carries a Colab Gemini instruction in its +metadata, built from this repository's [`colab/contexts.yaml`](../colab/contexts.yaml): +the preface, the core rules, then the text of the one or two skills whose `match` glob +first fits the notebook's path. Colab's Gemini cannot read this repository, so that +mapping is the only way the skills reach it. + +**CTI differs from the other assistants here.** The PyAutoHands release does not +regenerate `autocti_workspace`'s notebooks, so it never re-stamps them either. The +instruction a notebook carries is whatever was last applied with +`autohands gemini_context apply`. A mapping change therefore reaches users only when +you re-stamp the workspace notebooks and that change merges (pass 3). + +The mapping goes stale quietly. A renamed skill breaks the next stamp. A new workspace +folder falls through to `default` and its users get generic help. A skill that grows +pushes an entry over the size cap. Nothing in a skill edit or a workspace PR flags +these, so this skill is the routine refresh: a deterministic report says *where* the +mapping is thin, and you read the notebooks and skills to decide *what* it should say. + +The file's own header explains the matching rules (first match wins, `*` also crosses +`/`, `never_map`, the cap); [`autoassistant/colab_contexts.py`](../autoassistant/colab_contexts.py) +is the assistant-side check. + +## When to run it + +- After a skill is **added, renamed or removed**, or its text grows or shrinks + substantially (sizes count against the cap). +- After `autocti_workspace` **gains, renames or moves a folder** of scripts, or its + notebooks are regenerated (a regenerated notebook loses its stamp). +- **Before a release**, so the workspace notebooks carry a current map. + +## Ask + +Confirm with the maintainer before starting: **report only**, or **edit, re-stamp and +open PRs** in this session. + +## Orient — locate the checkouts + +You need this repository (the current directory), a PyAutoHands checkout for the +report and the stamp, and an `autocti_workspace` checkout. Prefer an existing checkout; +clone into the gitignored `sources/` folder only when none is found: + +```bash +source activate.sh + +find_checkout() { # $1 = repository name; prints the first checkout found + for d in "sources/$1" "../$1" "../../organs/$1"; do + [ -d "$d/.git" ] || [ -f "$d/.git" ] && { echo "$d"; return; } + done + git clone --quiet --depth 1 "https://github.com/PyAutoLabs/$1" "sources/$1" && echo "sources/$1" +} +HANDS=$(find_checkout PyAutoHands) +WS_MAIN=$(find_checkout autocti_workspace) + +refresh_checkout() { # fast-forward only a clean checkout on main; never touch anything else + if [ -z "$(git -C "$1" status --porcelain)" ] && [ "$(git -C "$1" branch --show-current)" = main ]; then + git -C "$1" pull --quiet --ff-only || echo "WARNING: could not fast-forward $1" + else + echo "WARNING: $1 has local changes or is not on main; using it as it is" + fi +} +for d in "$HANDS" "$WS_MAIN"; do refresh_checkout "$d"; done +"$HANDS/bin/autohands" gemini_context --help >/dev/null 2>&1 \ + || echo "STOP: Hands checkout predates #324 — update it" +``` + +`../` is a sibling clone; `../../organs/PyAutoHands` is where PyAutoHands sits in +the PyAutoLabs development layout. `refresh_checkout` brings every located checkout up +to `main` so you map what users will get — but only a clean checkout already on `main`; +one with local changes or on another branch is used as it is, with a warning (say so in +the PR). If the last line prints `STOP`, the PyAutoHands checkout is older than the +`gemini_context` command: update it before going on. Never edit, commit to, push or +switch branches in the checkouts located here — they are read-only. The pass 3 +re-stamp happens in a separate checkout made for that change. + +The report reads the workspace's committed `notebooks/` tree. A folder that exists +only under `scripts/` (its notebooks not generated yet) is invisible to it; pass 1's +hand check finds those. + +## Branch — pass 1: the deterministic report + +Run the assistant-side check, then the PyAutoHands coverage report: + +```bash +python -m autoassistant.colab_contexts check + +"$HANDS/bin/autohands" gemini_context report --workspace "$WS_MAIN" \ + --project autocti --assistant-root . --json "${TMPDIR:-/tmp}/colab_report_autocti.json" +``` + +`check` fails on a mapped skill that does not exist, a `never_map` skill that is mapped, +or an entry over the cap. The report prints a short summary and writes JSON with: + +- `default_notebooks` / `default_only_folders` — notebooks no entry matches. +- `unused_entries` — entries for this workspace that match no notebook. +- `missing_skills`, `never_map_violations` — mapped skills that are gone or banned. +- `sizes` (`max`, `median`, `over_cap`, `largest`) and `entries` (per-entry notebook + count and `max_chars`) — how close each entry sits to `cap`. + +Without `--json` the JSON follows the summary on stdout. A report exits 1 on a missing +skill, a `never_map` violation or an over-cap notebook. Save the text summary; it goes +into the PR body. + +Two gaps the report cannot see, so check them by hand: + +```bash +python - "$WS_MAIN" autocti_workspace <<'EOF' +import sys +from pathlib import Path +from autoassistant.colab_contexts import load, match +ws, repo = Path(sys.argv[1]), sys.argv[2] +ctx = load() +# Scripts whose notebooks would reach only `default`. +for py in sorted((ws / "scripts").rglob("*.py")): + if py.name == "__init__.py": + continue + rel = py.relative_to(ws / "scripts").with_suffix(".ipynb").as_posix() + if match(ctx, f"{repo}/notebooks/{rel}")[0] == "default": + print("default only (script):", rel) +# Skills neither mapped anywhere nor listed in never_map. +used = set(ctx["default"]["skills"]) | {s for e in ctx["contexts"] for s in e["skills"]} +for p in sorted(Path("skills").glob("*.md")): + if p.stem != "README" and p.stem not in used | set(ctx.get("never_map") or []): + print("unmapped skill:", p.stem) +EOF +``` + +Ignore helper modules that never become notebooks; a script folder whose notebooks do +not exist yet is a real gap. + +## Branch — pass 2: the AI scan + +Work only on what pass 1 flagged. For each item, read before you propose: + +- **The notebook's purpose.** The docstring at the top of the matching + `scripts/.py` in the workspace (and the folder's `README.rst` or `README.md` + if it has one) says what the notebook teaches. +- **The candidate skills.** The `description` line in each skill's frontmatter + (`grep -m1 '^description:' skills/ac_*.md`) says what the skill covers and what it + is not for. + +Then propose the smallest edit that fixes each item: + +- **Notebooks reached only by `default`.** Add one entry with the narrowest glob that + covers the folder (`.../folder/*`) — per notebook only when one notebook needs + different skills from its siblings. Give it one or two skills, the closest to the + notebook's task first. Place it **above** every broader entry that would otherwise + match, since the first match wins. If `default` really is the best fit, leave it and + say so in the PR. +- **Entries that match nothing.** Find where the folder went (`git -C "$WS_MAIN" log + --diff-filter=R --summary -- scripts/` or a search of the tree). Retarget the glob if + it moved; delete the entry if the folder is gone. Keep an entry whose folder exists + under `scripts/` but has no notebooks yet — they will exist once generated. +- **Missing skills.** A renamed skill → swap in the new name. A deleted skill → pick + the replacement from the skill descriptions, or drop it if the other skill suffices. +- **New skills nobody maps.** Map a user-facing skill where its workspace folder is; add + a maintainer or agent-infrastructure skill (one that needs this repository checked + out) to `never_map`. A skill with no workspace notebook to serve (one paired with a + separate pipeline, say) can stay unmapped — note it in the PR rather than forcing it + onto a folder. +- **Entries over the cap** must be fixed: drop the second skill or swap in a shorter + one. **Entries near it** (`max_chars` above about 90% of `cap`) only need a line in + the PR; a single-skill entry near the cap means that skill wants trimming, which is a + separate change. Never raise `cap` yourself: it is a human decision (default 20,000, + ceiling 50,000) — propose it in the PR instead. + +Constraints on every edit: most specific entries first; one or two skills per entry; +never map a `never_map` skill; every entry within `cap`; touch only the entries the +report flagged, and keep the file's section comments in place. + +## Branch — pass 3: apply, re-check and re-stamp + +Edit `colab/contexts.yaml`, then repeat pass 1 until it is clean: + +```bash +python -m autoassistant.colab_contexts check +python -m pytest -q autoassistant/tests/test_colab_contexts.py +"$HANDS/bin/autohands" gemini_context report --workspace "$WS_MAIN" --project autocti \ + --assistant-root . --json "${TMPDIR:-/tmp}/colab_report_autocti.json" +``` + +The report should exit 0, with no `default only` lines you did not choose on purpose. + +Then re-stamp the workspace notebooks, because no release will do it. That is a change +to `autocti_workspace`, so it runs in a checkout made for this change — never in +`$WS_MAIN`, which the skill only located (in the PyAutoLabs layout it is the canonical +checkout). In the PyAutoLabs development setup, use the `autocti_workspace` worktree +that `start_dev` / `start_workspace` creates for the task: + +```bash +WS_EDIT= +``` + +Elsewhere, make a throwaway clone on a dated branch: + +```bash +WS_EDIT="sources/autocti_workspace_restamp_$(date +%Y%m%d)" +git clone --quiet --depth 1 https://github.com/PyAutoLabs/autocti_workspace "$WS_EDIT" +git -C "$WS_EDIT" switch -c "colab-contexts-refresh-$(date +%Y%m%d)" +``` + +Then stamp it and look at the diff: + +```bash +"$HANDS/bin/autohands" gemini_context apply "$WS_EDIT/notebooks" \ + --project autocti --assistant-root . +git -C "$WS_EDIT" diff --stat +``` + +`apply` prints how many notebooks it stamped; `Stamped 0 notebooks` means a wrong path. +The diff must touch notebook metadata only — no cell source or output. If anything else +changed, stop and find out why before going on. The workspace README's Colab Gemini +note documents the same command for whoever regenerates the notebooks. + +## Combine — open the PRs + +Show the maintainer both diffs. With their go-ahead, commit each on a branch (never +`main`), push the branches and open two PRs through the normal maintainer workflow (in +the PyAutoLabs development setup, `start_dev` → `ship_workspace`): + +1. **autocti_assistant** — the `colab/contexts.yaml` change. The PR body carries the + before and after report summaries, pasted as printed; one line per edit (the entry, + the skills, and why); and anything left for a human: a proposed cap change, a + `default` kept on purpose, a skill that needs trimming. +2. **autocti_workspace** — the metadata-only re-stamp, linking the assistant PR. Merge + it after the assistant PR, so the stamped text matches the mapping on `main`. + +Merging stays with a human. + +The same report is upkeep evidence elsewhere: PyAutoBroca reads the +`autohands gemini_context report` output to record how fresh the skill↔workspace link +is and to flag "refresh needed". It never runs this scan or edits the mapping, so keep +the report itself the record — paste its output rather than re-describing it. + +## When NOT to invoke this skill + +- For a user's CTI question, or while helping someone with a notebook. +- To edit a skill's content or a workspace script — those are their own changes; run + this skill afterwards. +- Inside unrelated feature work. A mapping diff should be its own reviewable PR. + +## Agent procedural checklist + +1. Confirm report-only or edit, re-stamp and PR. +2. `source activate.sh`; locate (or clone into `sources/`) PyAutoHands and + `autocti_workspace`; bring the workspace up to `main`. +3. Run `python -m autoassistant.colab_contexts check` and `autohands gemini_context + report --project autocti`; save the summary. +4. Run the script-tree and unmapped-skill scan. +5. For each flagged item, read the script header and the skill descriptions, then + propose the minimal `colab/contexts.yaml` edit (most specific first, one or two + skills, `never_map` and `cap` respected). +6. Apply; re-run `check`, the colab tests and the report until clean. +7. In the task's workspace worktree (or a throwaway dated-branch clone) — never the + located `$WS_MAIN` — re-stamp with `autohands gemini_context apply + /notebooks --project autocti --assistant-root .`; confirm the diff is + metadata only. +8. With the maintainer's go-ahead, commit both on branches, push them and open the two + PRs (assistant first, workspace re-stamp linked). Never push to `main`.