From 8d1e17f1cca563d290329eed0c78ab282ee20c4f Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 07:39:22 +0000 Subject: [PATCH 1/2] Trim the experimental Colab notebook to a Gemini sandbox MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Keep only the two setup cells — install the pinned assistant stack, print the Gemini sidebar context — and drop the guided tour (data inspection, model and likelihood, the small fit, the results download). A short closing cell says the notebook is now the reader's sandbox for working in PyAutoFit with Gemini. Edited in scripts/experimental_colab.py and regenerated with `python -m autoassistant.to_notebook`; README's Colab section follows. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01FS8ubAQEXFKyv4GxDaSpra --- README.md | 11 +- notebooks/experimental_colab.ipynb | 179 ++++------------------------- scripts/experimental_colab.py | 133 +++------------------ 3 files changed, 45 insertions(+), 278 deletions(-) diff --git a/README.md b/README.md index 984f24a..84ff2c6 100644 --- a/README.md +++ b/README.md @@ -470,15 +470,16 @@ come away understanding the workflow, not just the commands. [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PyAutoLabs/autofit_assistant/blob/main/notebooks/experimental_colab.ipynb) -Try the assistant's bundled Gaussian fit in a fresh CPU runtime, with context to -copy into Colab's Gemini sidebar. The notebook installs PyAutoFit, introduces the -model and likelihood, runs a small fit, and lets you download the results. +Two cells in a fresh CPU runtime: one installs the assistant's PyAutoFit stack, +one prints the context to copy into Colab's Gemini sidebar. After that the +notebook is your sandbox — write your own cells and work through them with Gemini. This is experimental: cloning the assistant does not automatically load its instructions into Gemini. Follow the notebook's explicit context step. Gemini availability depends on your Google account and Colab settings. Save a notebook -copy and download results before the runtime resets; this is a small browser -experiment, with larger inference projects continuing in the full assistant. +copy and download anything you want to keep before the runtime resets; this is a +small browser experiment, with larger inference projects continuing in the full +assistant. ## License diff --git a/notebooks/experimental_colab.ipynb b/notebooks/experimental_colab.ipynb index a5f6399..3136975 100644 --- a/notebooks/experimental_colab.ipynb +++ b/notebooks/experimental_colab.ipynb @@ -6,25 +6,22 @@ "source": [ "# Experimental: PyAutoFit assistant in Colab\n", "\n", - "Try a small inference problem alongside Colab's Gemini sidebar. Use a **fresh CPU\n", - "runtime**, then run the cells in order. Installation can take a few minutes.\n", + "Install the assistant's PyAutoFit stack in a **fresh CPU runtime** and give Colab's\n", + "Gemini sidebar its context, then experiment in your own cells. Run the two cells\n", + "below in order; installation can take a few minutes.\n", "\n", "This is a notebook experiment, not the full CLI assistant or an HPC service.\n", "Cloning the repository does **not** make Gemini read its instructions. Step 2\n", "provides context to copy into the sidebar explicitly. Gemini availability depends\n", - "on your Google account and Colab settings; the fit also works without Gemini.\n", + "on your Google account and Colab settings.\n", "\n", "Save a copy of this notebook in Drive to keep your edits. Runtime files disappear\n", - "when Colab resets: download your results at the end.\n", + "when Colab resets: download anything you want to keep before disconnecting.\n", "\n", "__Contents__\n", "\n", "1. Setup\n", - "2. Gemini context\n", - "3. Data inspection\n", - "4. Model and likelihood\n", - "5. Fit\n", - "6. Download and continue" + "2. Gemini context" ], "id": "cell-0" }, @@ -82,8 +79,8 @@ "## 2. Give Gemini the assistant context\n", "\n", "Run this cell, then **copy its printed text into the Gemini sidebar**. Ask Gemini\n", - "to acknowledge the supplied context before discussing the next cell. The context\n", - "comes from this pinned assistant checkout; no API key or Gemini SDK is needed.\n", + "to acknowledge the supplied context before you start work. The context comes from\n", + "this pinned assistant checkout; no API key or Gemini SDK is needed.\n", "\n", "Only the text you supply is guaranteed to be part of that conversation. If Gemini\n", "needs another skill, open that file in Colab's Files panel and paste the relevant\n", @@ -103,14 +100,14 @@ " \"skills/af_wrap_likelihood.md\",\n", "]\n", "bootstrap = (\n", - " \"Help me work through this experimental PyAutoFit notebook in teacher mode. \"\n", + " \"Help me work on PyAutoFit in this experimental Colab notebook in teacher mode. \"\n", " \"Use only notebook-relevant guidance from the reference material below and \"\n", " \"the code I share. Skip CLI onboarding, filesystem checks, project-memory \"\n", " \"creation and maintainer workflows. Ask one question at \"\n", " \"a time; give hints before solutions. Do not claim to have read runtime files \"\n", - " \"or run code unless you actually have. Preserve the supplied likelihood's \"\n", - " \"numerics. Before adapting this example to real data, help me inspect the \"\n", - " \"data and discuss artefacts, outliers and selection effects. Explain proposed \"\n", + " \"or run code unless you actually have. Preserve the numerics of any \"\n", + " \"likelihood I supply. Before fitting real data, help me inspect it and discuss \"\n", + " \"artefacts, outliers and selection effects. Explain proposed \"\n", " \"changes before I run them. If an API is uncertain, ask me to inspect it.\\n\"\n", ")\n", "context = bootstrap + \"\\nAssistant reference: \" + ASSISTANT_REF + \"\\n\"\n", @@ -126,154 +123,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 3. Inspect the bundled simulated data\n", + "## That is the whole notebook\n", "\n", - "The assistant's Gaussian is **peak-normalised**: normalization is its height,\n", - "not its integrated area. The simulation used centre 50, height 25 and width 10.\n", - "Its likelihood assumes independent Gaussian measurement errors and omits the\n", - "constant noise-normalisation term. Evidence values therefore use that convention.\n", + "This is now your assistant sandbox. Add your own cells below and use Gemini in\n", + "the sidebar to do things in PyAutoFit — compose a model, wrap a likelihood, run\n", + "a search, read a result. Paste the cell or error you are working on into your\n", + "message: the sidebar cannot reliably see the notebook.\n", "\n", - "Ask Gemini: *What features of this plot should constrain the three parameters?*\n", - "The model and likelihood below are imported unchanged from the assistant's tour." + "The assistant's own worked example (data, model, likelihood) is in\n", + "`scripts/start_here/` in the checkout. For larger inference projects, continue in\n", + "a local clone of [autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant)." ], "id": "cell-4" - }, - { - "cell_type": "code", - "metadata": {}, - "source": [ - "import json\n", - "import numpy as np\n", - "import matplotlib.pyplot as plt\n", - "sys.path.insert(0, str(repo / \"scripts\" / \"start_here\"))\n", - "from gaussian import Gaussian\n", - "from analysis import Analysis\n", - "\n", - "data = np.asarray(json.loads(Path(\"dataset/gaussian_x1/data.json\").read_text()))\n", - "noise_map = np.asarray(json.loads(Path(\"dataset/gaussian_x1/noise_map.json\").read_text()))\n", - "x = np.arange(data.size)\n", - "plt.errorbar(x, data, yerr=noise_map, fmt=\".k\")\n", - "plt.xlabel(\"x\")\n", - "plt.ylabel(\"Profile amplitude\")\n", - "plt.show()" - ], - "execution_count": null, - "outputs": [], - "id": "cell-5" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 4. Compose the model and check the likelihood\n", - "\n", - "Ask Gemini: *Explain these priors. What would happen if I fixed the width?*\n", - "The check below compares the imported likelihood with its written equation." - ], - "id": "cell-6" - }, - { - "cell_type": "code", - "metadata": {}, - "source": [ - "model = af.Model(\n", - " Gaussian,\n", - " centre=af.UniformPrior(lower_limit=0.0, upper_limit=100.0),\n", - " normalization=af.UniformPrior(lower_limit=0.0, upper_limit=100.0),\n", - " sigma=af.UniformPrior(lower_limit=0.1, upper_limit=30.0),\n", - ")\n", - "analysis = Analysis(data=data, noise_map=noise_map)\n", - "truth = Gaussian(centre=50.0, normalization=25.0, sigma=10.0)\n", - "expected = -0.5 * np.sum(((data - truth.model_data_from(x)) / noise_map) ** 2)\n", - "assert np.isclose(analysis.log_likelihood_function(truth), expected)\n", - "print(model.info)\n", - "print(\"Log likelihood at simulation inputs:\", expected)" - ], - "execution_count": null, - "outputs": [], - "id": "cell-7" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 5. Run a small fit\n", - "\n", - "Run this cell when ready. It performs actual nested sampling with 50 live points,\n", - "chosen for experimentation, not precision results. Runtime varies. The same\n", - "completed run reloads if you execute this cell again. Change `name` for a new\n", - "experiment, especially after changing the data or likelihood.\n", - "\n", - "Ask Gemini: *Before I run this, what do you expect the recovered parameters to be?*" - ], - "id": "cell-8" - }, - { - "cell_type": "code", - "metadata": {}, - "source": [ - "search = af.DynestyStatic(\n", - " path_prefix=\"experimental_colab\", name=\"gaussian_x1\", nlive=50,\n", - " number_of_cores=1,\n", - ")\n", - "result = search.fit(model=model, analysis=analysis)\n", - "print(result.info)\n", - "best = result.max_log_likelihood_instance\n", - "plt.errorbar(x, data, yerr=noise_map, fmt=\".k\", label=\"data\")\n", - "plt.plot(x, best.model_data_from(x), label=\"maximum likelihood fit\")\n", - "plt.xlabel(\"x\")\n", - "plt.ylabel(\"Profile amplitude\")\n", - "plt.legend()\n", - "plt.show()" - ], - "execution_count": null, - "outputs": [], - "id": "cell-9" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 6. Keep the results and continue\n", - "\n", - "Ask Gemini: *Help me explain the parameter uncertainties and one limitation of\n", - "this fit. Then suggest one change I can predict and test.* Share `result.info`\n", - "with your question. The fit and residual figures are also under\n", - "`output/experimental_colab/` in the Files panel.\n", - "\n", - "Run this last cell to download the fit outputs plus the model, likelihood, data,\n", - "configuration and context used here. Save your notebook copy separately: the\n", - "archive does not contain your edited notebook or Gemini conversation. Download\n", - "before disconnecting. For larger inference projects, continue in a local clone\n", - "of [autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant), carrying\n", - "your scientific question, priors, likelihood and results with you." - ], - "id": "cell-10" - }, - { - "cell_type": "code", - "metadata": {}, - "source": [ - "import zipfile\n", - "from google.colab import files\n", - "\n", - "archive = repo.parent / \"autofit_colab_results.zip\"\n", - "with zipfile.ZipFile(archive, \"w\", zipfile.ZIP_DEFLATED) as bundle:\n", - " for directory in [\"output/experimental_colab\", \"scripts/start_here\",\n", - " \"dataset/gaussian_x1\", \"config\"]:\n", - " for path in Path(directory).rglob(\"*\"):\n", - " if path.is_file() and \"__pycache__\" not in path.parts:\n", - " bundle.write(path, str(path))\n", - " bundle.writestr(\"gemini_context.txt\", context)\n", - " bundle.writestr(\"environment.txt\", subprocess.check_output(\n", - " [sys.executable, \"-m\", \"pip\", \"freeze\"], text=True))\n", - " bundle.writestr(\"assistant_ref.txt\", ASSISTANT_REF + \"\\n\" +\n", - " subprocess.check_output([\"git\", \"rev-parse\", \"HEAD\"], text=True))\n", - "files.download(str(archive))" - ], - "execution_count": null, - "outputs": [], - "id": "cell-11" } ], "metadata": { diff --git a/scripts/experimental_colab.py b/scripts/experimental_colab.py index 6c764b1..d83ebe5 100644 --- a/scripts/experimental_colab.py +++ b/scripts/experimental_colab.py @@ -1,25 +1,22 @@ """ # Experimental: PyAutoFit assistant in Colab -Try a small inference problem alongside Colab's Gemini sidebar. Use a **fresh CPU -runtime**, then run the cells in order. Installation can take a few minutes. +Install the assistant's PyAutoFit stack in a **fresh CPU runtime** and give Colab's +Gemini sidebar its context, then experiment in your own cells. Run the two cells +below in order; installation can take a few minutes. This is a notebook experiment, not the full CLI assistant or an HPC service. Cloning the repository does **not** make Gemini read its instructions. Step 2 provides context to copy into the sidebar explicitly. Gemini availability depends -on your Google account and Colab settings; the fit also works without Gemini. +on your Google account and Colab settings. Save a copy of this notebook in Drive to keep your edits. Runtime files disappear -when Colab resets: download your results at the end. +when Colab resets: download anything you want to keep before disconnecting. __Contents__ 1. Setup 2. Gemini context -3. Data inspection -4. Model and likelihood -5. Fit -6. Download and continue """ import os @@ -65,8 +62,8 @@ ## 2. Give Gemini the assistant context Run this cell, then **copy its printed text into the Gemini sidebar**. Ask Gemini -to acknowledge the supplied context before discussing the next cell. The context -comes from this pinned assistant checkout; no API key or Gemini SDK is needed. +to acknowledge the supplied context before you start work. The context comes from +this pinned assistant checkout; no API key or Gemini SDK is needed. Only the text you supply is guaranteed to be part of that conversation. If Gemini needs another skill, open that file in Colab's Files panel and paste the relevant @@ -81,14 +78,14 @@ "skills/af_wrap_likelihood.md", ] bootstrap = ( - "Help me work through this experimental PyAutoFit notebook in teacher mode. " + "Help me work on PyAutoFit in this experimental Colab notebook in teacher mode. " "Use only notebook-relevant guidance from the reference material below and " "the code I share. Skip CLI onboarding, filesystem checks, project-memory " "creation and maintainer workflows. Ask one question at " "a time; give hints before solutions. Do not claim to have read runtime files " - "or run code unless you actually have. Preserve the supplied likelihood's " - "numerics. Before adapting this example to real data, help me inspect the " - "data and discuss artefacts, outliers and selection effects. Explain proposed " + "or run code unless you actually have. Preserve the numerics of any " + "likelihood I supply. Before fitting real data, help me inspect it and discuss " + "artefacts, outliers and selection effects. Explain proposed " "changes before I run them. If an API is uncertain, ask me to inspect it.\n" ) context = bootstrap + "\nAssistant reference: " + ASSISTANT_REF + "\n" @@ -97,106 +94,14 @@ print(context) """ -## 3. Inspect the bundled simulated data +## That is the whole notebook -The assistant's Gaussian is **peak-normalised**: normalization is its height, -not its integrated area. The simulation used centre 50, height 25 and width 10. -Its likelihood assumes independent Gaussian measurement errors and omits the -constant noise-normalisation term. Evidence values therefore use that convention. +This is now your assistant sandbox. Add your own cells below and use Gemini in +the sidebar to do things in PyAutoFit — compose a model, wrap a likelihood, run +a search, read a result. Paste the cell or error you are working on into your +message: the sidebar cannot reliably see the notebook. -Ask Gemini: *What features of this plot should constrain the three parameters?* -The model and likelihood below are imported unchanged from the assistant's tour. +The assistant's own worked example (data, model, likelihood) is in +`scripts/start_here/` in the checkout. For larger inference projects, continue in +a local clone of [autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant). """ - -import json -import numpy as np -import matplotlib.pyplot as plt -sys.path.insert(0, str(repo / "scripts" / "start_here")) -from gaussian import Gaussian -from analysis import Analysis - -data = np.asarray(json.loads(Path("dataset/gaussian_x1/data.json").read_text())) -noise_map = np.asarray(json.loads(Path("dataset/gaussian_x1/noise_map.json").read_text())) -x = np.arange(data.size) -plt.errorbar(x, data, yerr=noise_map, fmt=".k") -plt.xlabel("x") -plt.ylabel("Profile amplitude") -plt.show() - -""" -## 4. Compose the model and check the likelihood - -Ask Gemini: *Explain these priors. What would happen if I fixed the width?* -The check below compares the imported likelihood with its written equation. -""" - -model = af.Model( - Gaussian, - centre=af.UniformPrior(lower_limit=0.0, upper_limit=100.0), - normalization=af.UniformPrior(lower_limit=0.0, upper_limit=100.0), - sigma=af.UniformPrior(lower_limit=0.1, upper_limit=30.0), -) -analysis = Analysis(data=data, noise_map=noise_map) -truth = Gaussian(centre=50.0, normalization=25.0, sigma=10.0) -expected = -0.5 * np.sum(((data - truth.model_data_from(x)) / noise_map) ** 2) -assert np.isclose(analysis.log_likelihood_function(truth), expected) -print(model.info) -print("Log likelihood at simulation inputs:", expected) - -""" -## 5. Run a small fit - -Run this cell when ready. It performs actual nested sampling with 50 live points, -chosen for experimentation, not precision results. Runtime varies. The same -completed run reloads if you execute this cell again. Change `name` for a new -experiment, especially after changing the data or likelihood. - -Ask Gemini: *Before I run this, what do you expect the recovered parameters to be?* -""" - -search = af.DynestyStatic( - path_prefix="experimental_colab", name="gaussian_x1", nlive=50, - number_of_cores=1, -) -result = search.fit(model=model, analysis=analysis) -print(result.info) -best = result.max_log_likelihood_instance -plt.errorbar(x, data, yerr=noise_map, fmt=".k", label="data") -plt.plot(x, best.model_data_from(x), label="maximum likelihood fit") -plt.xlabel("x") -plt.ylabel("Profile amplitude") -plt.legend() -plt.show() - -""" -## 6. Keep the results and continue - -Ask Gemini: *Help me explain the parameter uncertainties and one limitation of -this fit. Then suggest one change I can predict and test.* Share `result.info` -with your question. The fit and residual figures are also under -`output/experimental_colab/` in the Files panel. - -Run this last cell to download the fit outputs plus the model, likelihood, data, -configuration and context used here. Save your notebook copy separately: the -archive does not contain your edited notebook or Gemini conversation. Download -before disconnecting. For larger inference projects, continue in a local clone -of [autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant), carrying -your scientific question, priors, likelihood and results with you. -""" - -import zipfile -from google.colab import files - -archive = repo.parent / "autofit_colab_results.zip" -with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as bundle: - for directory in ["output/experimental_colab", "scripts/start_here", - "dataset/gaussian_x1", "config"]: - for path in Path(directory).rglob("*"): - if path.is_file() and "__pycache__" not in path.parts: - bundle.write(path, str(path)) - bundle.writestr("gemini_context.txt", context) - bundle.writestr("environment.txt", subprocess.check_output( - [sys.executable, "-m", "pip", "freeze"], text=True)) - bundle.writestr("assistant_ref.txt", ASSISTANT_REF + "\n" + - subprocess.check_output(["git", "rev-parse", "HEAD"], text=True)) -files.download(str(archive)) From 85f65704098ac0caccff33055f21ed30796fca6b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 07:58:48 +0000 Subject: [PATCH 2/2] Widen and retarget the Colab notebook's Gemini context MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The sidebar only ever sees what the notebook prints, so pick that payload for a sandbox rather than the deleted guided tour: - add BYOL mode and a "3. Your sandbox" section built around it — paste your own likelihood, numerics untouched, nothing fitted before your go-ahead; - add the skills a sandbox session reaches for next: configure_search, run_search, plot_fit, debug_fit_failure; - send an excerpt of AGENTS.md (preamble, safety invariants, three-layer model, domain adaptation) instead of the whole file — the omitted sections direct an agent that can read files and run commands, which the sidebar cannot; - add show() so any other assistant file can be printed and pasted when it is needed, rather than fixing everything in the one paste. Context goes from ~32 KB to ~58 KB (~14k tokens), all of it applicable. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01FS8ubAQEXFKyv4GxDaSpra --- README.md | 7 +- notebooks/experimental_colab.ipynb | 106 ++++++++++++++++++++++++----- scripts/experimental_colab.py | 97 +++++++++++++++++++++----- 3 files changed, 173 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 84ff2c6..8cc70ef 100644 --- a/README.md +++ b/README.md @@ -471,8 +471,11 @@ come away understanding the workflow, not just the commands. [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PyAutoLabs/autofit_assistant/blob/main/notebooks/experimental_colab.ipynb) Two cells in a fresh CPU runtime: one installs the assistant's PyAutoFit stack, -one prints the context to copy into Colab's Gemini sidebar. After that the -notebook is your sandbox — write your own cells and work through them with Gemini. +one prints the context to copy into Colab's Gemini sidebar — the safety +invariants, teacher and BYOL modes, and the skills for composing a model, +wrapping a likelihood, running a search, plotting a fit and debugging one that +fails. After that the notebook is your sandbox: write your own cells, paste your +own likelihood, and print any other assistant file on demand with `show(...)`. This is experimental: cloning the assistant does not automatically load its instructions into Gemini. Follow the notebook's explicit context step. Gemini diff --git a/notebooks/experimental_colab.ipynb b/notebooks/experimental_colab.ipynb index 3136975..e990f2f 100644 --- a/notebooks/experimental_colab.ipynb +++ b/notebooks/experimental_colab.ipynb @@ -7,8 +7,8 @@ "# Experimental: PyAutoFit assistant in Colab\n", "\n", "Install the assistant's PyAutoFit stack in a **fresh CPU runtime** and give Colab's\n", - "Gemini sidebar its context, then experiment in your own cells. Run the two cells\n", - "below in order; installation can take a few minutes.\n", + "Gemini sidebar its context, then experiment in your own cells. Run the first two\n", + "cells in order; installation can take a few minutes.\n", "\n", "This is a notebook experiment, not the full CLI assistant or an HPC service.\n", "Cloning the repository does **not** make Gemini read its instructions. Step 2\n", @@ -21,7 +21,8 @@ "__Contents__\n", "\n", "1. Setup\n", - "2. Gemini context" + "2. Gemini context\n", + "3. Your sandbox" ], "id": "cell-0" }, @@ -82,10 +83,16 @@ "to acknowledge the supplied context before you start work. The context comes from\n", "this pinned assistant checkout; no API key or Gemini SDK is needed.\n", "\n", - "Only the text you supply is guaranteed to be part of that conversation. If Gemini\n", - "needs another skill, open that file in Colab's Files panel and paste the relevant\n", - "section. Sidebar access to notebook content can vary: include the cell or error\n", - "in your message when needed." + "What it sends: the parts of `AGENTS.md` a sidebar session can actually act on,\n", + "teacher and BYOL modes, and the skills for composing a model, wrapping a\n", + "likelihood, configuring and running a search, plotting a fit and debugging one\n", + "that fails. The rest of the assistant is left out deliberately — much of it\n", + "directs an agent that can read files and run commands, which the sidebar cannot.\n", + "Section 3 prints any other file on demand.\n", + "\n", + "Only the text you supply is guaranteed to be part of that conversation. Sidebar\n", + "access to notebook content can vary: include the cell or error in your message\n", + "when needed." ], "id": "cell-2" }, @@ -93,12 +100,44 @@ "cell_type": "code", "metadata": {}, "source": [ + "# The sidebar has no filesystem, so only what is pasted is in context. These\n", + "# AGENTS.md sections carry the invariants and the map; the omitted ones describe\n", + "# session start-up, commit cadence and maintainer workflows that cannot run here.\n", + "agents_sections = [\n", + " \"Safety invariants\",\n", + " \"The three-layer model\",\n", + " \"Domain adaptation\",\n", + "]\n", "context_paths = [\n", - " \"AGENTS.md\",\n", " \"modes/teacher.md\",\n", + " \"modes/byol.md\",\n", " \"skills/af_compose_model.md\",\n", " \"skills/af_wrap_likelihood.md\",\n", + " \"skills/af_configure_search.md\",\n", + " \"skills/af_run_search.md\",\n", + " \"skills/af_plot_fit.md\",\n", + " \"skills/af_debug_fit_failure.md\",\n", "]\n", + "\n", + "\n", + "def agents_excerpt(wanted):\n", + " \"\"\"Return the named `## ` sections of AGENTS.md, plus its preamble, in file order.\"\"\"\n", + " blocks = {\"__preamble__\": []}\n", + " heading = \"__preamble__\"\n", + " for line in Path(\"AGENTS.md\").read_text().splitlines():\n", + " if line.startswith(\"## \"):\n", + " heading = line[3:].strip()\n", + " blocks[heading] = []\n", + " blocks[heading].append(line)\n", + " excerpt = [\"\\n\".join(blocks[\"__preamble__\"]).strip()]\n", + " for prefix in wanted:\n", + " matched = [h for h in blocks if h.startswith(prefix)]\n", + " if len(matched) != 1:\n", + " raise RuntimeError(f\"AGENTS.md section {prefix!r} matched {matched}\")\n", + " excerpt.append(\"\\n\".join(blocks[matched[0]]).strip())\n", + " return \"\\n\\n\".join(excerpt)\n", + "\n", + "\n", "bootstrap = (\n", " \"Help me work on PyAutoFit in this experimental Colab notebook in teacher mode. \"\n", " \"Use only notebook-relevant guidance from the reference material below and \"\n", @@ -111,9 +150,11 @@ " \"changes before I run them. If an API is uncertain, ask me to inspect it.\\n\"\n", ")\n", "context = bootstrap + \"\\nAssistant reference: \" + ASSISTANT_REF + \"\\n\"\n", + "context += \"\\n--- AGENTS.md (excerpt) ---\\n\" + agents_excerpt(agents_sections)\n", "for relative in context_paths:\n", - " context += \"\\n--- \" + relative + \" ---\\n\" + Path(relative).read_text()\n", - "print(context)" + " context += \"\\n\\n--- \" + relative + \" ---\\n\" + Path(relative).read_text()\n", + "print(context)\n", + "print(\"\\n[context: %d characters]\" % len(context))" ], "execution_count": null, "outputs": [], @@ -123,18 +164,47 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## That is the whole notebook\n", + "## 3. Your sandbox\n", + "\n", + "This is now your assistant sandbox. Add your own cells and use Gemini in the\n", + "sidebar to do things in PyAutoFit — compose a model, wrap a likelihood, run a\n", + "search, read a result. Paste the cell or the error you are working on into your\n", + "message; the sidebar cannot reliably see the notebook.\n", "\n", - "This is now your assistant sandbox. Add your own cells below and use Gemini in\n", - "the sidebar to do things in PyAutoFit — compose a model, wrap a likelihood, run\n", - "a search, read a result. Paste the cell or error you are working on into your\n", - "message: the sidebar cannot reliably see the notebook.\n", + "__Bring your own likelihood.__ The context above includes the assistant's BYOL\n", + "mode — overview, model, wrap and validate, choose a search, your go-ahead, then\n", + "run and inspect. Paste your own likelihood function into the sidebar and have\n", + "Gemini set PyAutoFit up around it. Two rules it is told to keep: your numerics are\n", + "never re-parametrised or \"fixed\", and nothing is fitted before you say go. Paste\n", + "the code itself, since the sidebar reads no files — and before fitting real data,\n", + "plot it and say what artefacts, outliers or selection effects you know of.\n", "\n", - "The assistant's own worked example (data, model, likelihood) is in\n", - "`scripts/start_here/` in the checkout. For larger inference projects, continue in\n", - "a local clone of [autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant)." + "The assistant's own worked example (data, model, likelihood, and the six steps\n", + "the guided tour walks) is in `scripts/start_here/` in the checkout. Run the cell\n", + "below to print any other assistant file when Gemini needs it, then paste the\n", + "output. For larger inference projects, continue in a local clone of\n", + "[autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant)." ], "id": "cell-4" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def show(relative):\n", + " \"\"\"Print any file from the assistant checkout, to copy into the Gemini sidebar.\"\"\"\n", + " print(Path(relative).read_text())\n", + "\n", + "\n", + "# e.g. show(\"skills/README.md\") — one line per skill, to pick the next one\n", + "# show(\"skills/af_simulate_dataset.md\") — any skill in full\n", + "# show(\"wiki/core/index.md\") — the curated PyAutoFit reference\n", + "# show(\"scripts/start_here/analysis.py\") — the bundled Analysis class\n", + "show(\"skills/README.md\")" + ], + "execution_count": null, + "outputs": [], + "id": "cell-5" } ], "metadata": { diff --git a/scripts/experimental_colab.py b/scripts/experimental_colab.py index d83ebe5..6357b82 100644 --- a/scripts/experimental_colab.py +++ b/scripts/experimental_colab.py @@ -2,8 +2,8 @@ # Experimental: PyAutoFit assistant in Colab Install the assistant's PyAutoFit stack in a **fresh CPU runtime** and give Colab's -Gemini sidebar its context, then experiment in your own cells. Run the two cells -below in order; installation can take a few minutes. +Gemini sidebar its context, then experiment in your own cells. Run the first two +cells in order; installation can take a few minutes. This is a notebook experiment, not the full CLI assistant or an HPC service. Cloning the repository does **not** make Gemini read its instructions. Step 2 @@ -17,6 +17,7 @@ 1. Setup 2. Gemini context +3. Your sandbox """ import os @@ -65,18 +66,56 @@ to acknowledge the supplied context before you start work. The context comes from this pinned assistant checkout; no API key or Gemini SDK is needed. -Only the text you supply is guaranteed to be part of that conversation. If Gemini -needs another skill, open that file in Colab's Files panel and paste the relevant -section. Sidebar access to notebook content can vary: include the cell or error -in your message when needed. +What it sends: the parts of `AGENTS.md` a sidebar session can actually act on, +teacher and BYOL modes, and the skills for composing a model, wrapping a +likelihood, configuring and running a search, plotting a fit and debugging one +that fails. The rest of the assistant is left out deliberately — much of it +directs an agent that can read files and run commands, which the sidebar cannot. +Section 3 prints any other file on demand. + +Only the text you supply is guaranteed to be part of that conversation. Sidebar +access to notebook content can vary: include the cell or error in your message +when needed. """ +# The sidebar has no filesystem, so only what is pasted is in context. These +# AGENTS.md sections carry the invariants and the map; the omitted ones describe +# session start-up, commit cadence and maintainer workflows that cannot run here. +agents_sections = [ + "Safety invariants", + "The three-layer model", + "Domain adaptation", +] context_paths = [ - "AGENTS.md", "modes/teacher.md", + "modes/byol.md", "skills/af_compose_model.md", "skills/af_wrap_likelihood.md", + "skills/af_configure_search.md", + "skills/af_run_search.md", + "skills/af_plot_fit.md", + "skills/af_debug_fit_failure.md", ] + + +def agents_excerpt(wanted): + """Return the named `## ` sections of AGENTS.md, plus its preamble, in file order.""" + blocks = {"__preamble__": []} + heading = "__preamble__" + for line in Path("AGENTS.md").read_text().splitlines(): + if line.startswith("## "): + heading = line[3:].strip() + blocks[heading] = [] + blocks[heading].append(line) + excerpt = ["\n".join(blocks["__preamble__"]).strip()] + for prefix in wanted: + matched = [h for h in blocks if h.startswith(prefix)] + if len(matched) != 1: + raise RuntimeError(f"AGENTS.md section {prefix!r} matched {matched}") + excerpt.append("\n".join(blocks[matched[0]]).strip()) + return "\n\n".join(excerpt) + + bootstrap = ( "Help me work on PyAutoFit in this experimental Colab notebook in teacher mode. " "Use only notebook-relevant guidance from the reference material below and " @@ -89,19 +128,43 @@ "changes before I run them. If an API is uncertain, ask me to inspect it.\n" ) context = bootstrap + "\nAssistant reference: " + ASSISTANT_REF + "\n" +context += "\n--- AGENTS.md (excerpt) ---\n" + agents_excerpt(agents_sections) for relative in context_paths: - context += "\n--- " + relative + " ---\n" + Path(relative).read_text() + context += "\n\n--- " + relative + " ---\n" + Path(relative).read_text() print(context) +print("\n[context: %d characters]" % len(context)) """ -## That is the whole notebook +## 3. Your sandbox + +This is now your assistant sandbox. Add your own cells and use Gemini in the +sidebar to do things in PyAutoFit — compose a model, wrap a likelihood, run a +search, read a result. Paste the cell or the error you are working on into your +message; the sidebar cannot reliably see the notebook. + +__Bring your own likelihood.__ The context above includes the assistant's BYOL +mode — overview, model, wrap and validate, choose a search, your go-ahead, then +run and inspect. Paste your own likelihood function into the sidebar and have +Gemini set PyAutoFit up around it. Two rules it is told to keep: your numerics are +never re-parametrised or "fixed", and nothing is fitted before you say go. Paste +the code itself, since the sidebar reads no files — and before fitting real data, +plot it and say what artefacts, outliers or selection effects you know of. + +The assistant's own worked example (data, model, likelihood, and the six steps +the guided tour walks) is in `scripts/start_here/` in the checkout. Run the cell +below to print any other assistant file when Gemini needs it, then paste the +output. For larger inference projects, continue in a local clone of +[autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant). +""" -This is now your assistant sandbox. Add your own cells below and use Gemini in -the sidebar to do things in PyAutoFit — compose a model, wrap a likelihood, run -a search, read a result. Paste the cell or error you are working on into your -message: the sidebar cannot reliably see the notebook. -The assistant's own worked example (data, model, likelihood) is in -`scripts/start_here/` in the checkout. For larger inference projects, continue in -a local clone of [autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant). -""" +def show(relative): + """Print any file from the assistant checkout, to copy into the Gemini sidebar.""" + print(Path(relative).read_text()) + + +# e.g. show("skills/README.md") — one line per skill, to pick the next one +# show("skills/af_simulate_dataset.md") — any skill in full +# show("wiki/core/index.md") — the curated PyAutoFit reference +# show("scripts/start_here/analysis.py") — the bundled Analysis class +show("skills/README.md")