Skip to content

Make the experimental Colab notebook a Gemini sandbox #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jammy2211 merged 2 commits into from
Sep 15, 2026
Merged

Make the experimental Colab notebook a Gemini sandbox #46

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8d1e17f
Trim the experimental Colab notebook to a Gemini sandbox
claude Sep 15, 2026
85f6570
Widen and retarget the Colab notebook's Gemini context
claude Sep 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -470,15 +470,19 @@ come away understanding the workflow, not just the commands.

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PyAutoLabs/autofit_assistant/blob/main/notebooks/experimental_colab.ipynb)

Try the assistant's bundled Gaussian fit in a fresh CPU runtime, with context to
copy into Colab's Gemini sidebar. The notebook installs PyAutoFit, introduces the
model and likelihood, runs a small fit, and lets you download the results.
Two cells in a fresh CPU runtime: one installs the assistant's PyAutoFit stack,
one prints the context to copy into Colab's Gemini sidebar — the safety
invariants, teacher and BYOL modes, and the skills for composing a model,
wrapping a likelihood, running a search, plotting a fit and debugging one that
fails. After that the notebook is your sandbox: write your own cells, paste your
own likelihood, and print any other assistant file on demand with `show(...)`.

This is experimental: cloning the assistant does not automatically load its
instructions into Gemini. Follow the notebook's explicit context step. Gemini
availability depends on your Google account and Colab settings. Save a notebook
copy and download results before the runtime resets; this is a small browser
experiment, with larger inference projects continuing in the full assistant.
copy and download anything you want to keep before the runtime resets; this is a
small browser experiment, with larger inference projects continuing in the full
assistant.

## License

Expand Down
247 changes: 89 additions & 158 deletions notebooks/experimental_colab.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,23 @@
"source": [
"# Experimental: PyAutoFit assistant in Colab\n",
"\n",
"Try a small inference problem alongside Colab's Gemini sidebar. Use a **fresh CPU\n",
"runtime**, then run the cells in order. Installation can take a few minutes.\n",
"Install the assistant's PyAutoFit stack in a **fresh CPU runtime** and give Colab's\n",
"Gemini sidebar its context, then experiment in your own cells. Run the first two\n",
"cells in order; installation can take a few minutes.\n",
"\n",
"This is a notebook experiment, not the full CLI assistant or an HPC service.\n",
"Cloning the repository does **not** make Gemini read its instructions. Step 2\n",
"provides context to copy into the sidebar explicitly. Gemini availability depends\n",
"on your Google account and Colab settings; the fit also works without Gemini.\n",
"on your Google account and Colab settings.\n",
"\n",
"Save a copy of this notebook in Drive to keep your edits. Runtime files disappear\n",
"when Colab resets: download your results at the end.\n",
"when Colab resets: download anything you want to keep before disconnecting.\n",
"\n",
"__Contents__\n",
"\n",
"1. Setup\n",
"2. Gemini context\n",
"3. Data inspection\n",
"4. Model and likelihood\n",
"5. Fit\n",
"6. Download and continue"
"3. Your sandbox"
],
"id": "cell-0"
},
Expand Down Expand Up @@ -82,41 +80,81 @@
"## 2. Give Gemini the assistant context\n",
"\n",
"Run this cell, then **copy its printed text into the Gemini sidebar**. Ask Gemini\n",
"to acknowledge the supplied context before discussing the next cell. The context\n",
"comes from this pinned assistant checkout; no API key or Gemini SDK is needed.\n",
"\n",
"Only the text you supply is guaranteed to be part of that conversation. If Gemini\n",
"needs another skill, open that file in Colab's Files panel and paste the relevant\n",
"section. Sidebar access to notebook content can vary: include the cell or error\n",
"in your message when needed."
"to acknowledge the supplied context before you start work. The context comes from\n",
"this pinned assistant checkout; no API key or Gemini SDK is needed.\n",
"\n",
"What it sends: the parts of `AGENTS.md` a sidebar session can actually act on,\n",
"teacher and BYOL modes, and the skills for composing a model, wrapping a\n",
"likelihood, configuring and running a search, plotting a fit and debugging one\n",
"that fails. The rest of the assistant is left out deliberately — much of it\n",
"directs an agent that can read files and run commands, which the sidebar cannot.\n",
"Section 3 prints any other file on demand.\n",
"\n",
"Only the text you supply is guaranteed to be part of that conversation. Sidebar\n",
"access to notebook content can vary: include the cell or error in your message\n",
"when needed."
],
"id": "cell-2"
},
{
"cell_type": "code",
"metadata": {},
"source": [
"# The sidebar has no filesystem, so only what is pasted is in context. These\n",
"# AGENTS.md sections carry the invariants and the map; the omitted ones describe\n",
"# session start-up, commit cadence and maintainer workflows that cannot run here.\n",
"agents_sections = [\n",
" \"Safety invariants\",\n",
" \"The three-layer model\",\n",
" \"Domain adaptation\",\n",
"]\n",
"context_paths = [\n",
" \"AGENTS.md\",\n",
" \"modes/teacher.md\",\n",
" \"modes/byol.md\",\n",
" \"skills/af_compose_model.md\",\n",
" \"skills/af_wrap_likelihood.md\",\n",
" \"skills/af_configure_search.md\",\n",
" \"skills/af_run_search.md\",\n",
" \"skills/af_plot_fit.md\",\n",
" \"skills/af_debug_fit_failure.md\",\n",
"]\n",
"\n",
"\n",
"def agents_excerpt(wanted):\n",
" \"\"\"Return the named `## ` sections of AGENTS.md, plus its preamble, in file order.\"\"\"\n",
" blocks = {\"__preamble__\": []}\n",
" heading = \"__preamble__\"\n",
" for line in Path(\"AGENTS.md\").read_text().splitlines():\n",
" if line.startswith(\"## \"):\n",
" heading = line[3:].strip()\n",
" blocks[heading] = []\n",
" blocks[heading].append(line)\n",
" excerpt = [\"\\n\".join(blocks[\"__preamble__\"]).strip()]\n",
" for prefix in wanted:\n",
" matched = [h for h in blocks if h.startswith(prefix)]\n",
" if len(matched) != 1:\n",
" raise RuntimeError(f\"AGENTS.md section {prefix!r} matched {matched}\")\n",
" excerpt.append(\"\\n\".join(blocks[matched[0]]).strip())\n",
" return \"\\n\\n\".join(excerpt)\n",
"\n",
"\n",
"bootstrap = (\n",
" \"Help me work through this experimental PyAutoFit notebook in teacher mode. \"\n",
" \"Help me work on PyAutoFit in this experimental Colab notebook in teacher mode. \"\n",
" \"Use only notebook-relevant guidance from the reference material below and \"\n",
" \"the code I share. Skip CLI onboarding, filesystem checks, project-memory \"\n",
" \"creation and maintainer workflows. Ask one question at \"\n",
" \"a time; give hints before solutions. Do not claim to have read runtime files \"\n",
" \"or run code unless you actually have. Preserve the supplied likelihood's \"\n",
" \"numerics. Before adapting this example to real data, help me inspect the \"\n",
" \"data and discuss artefacts, outliers and selection effects. Explain proposed \"\n",
" \"or run code unless you actually have. Preserve the numerics of any \"\n",
" \"likelihood I supply. Before fitting real data, help me inspect it and discuss \"\n",
" \"artefacts, outliers and selection effects. Explain proposed \"\n",
" \"changes before I run them. If an API is uncertain, ask me to inspect it.\\n\"\n",
")\n",
"context = bootstrap + \"\\nAssistant reference: \" + ASSISTANT_REF + \"\\n\"\n",
"context += \"\\n--- AGENTS.md (excerpt) ---\\n\" + agents_excerpt(agents_sections)\n",
"for relative in context_paths:\n",
" context += \"\\n--- \" + relative + \" ---\\n\" + Path(relative).read_text()\n",
"print(context)"
" context += \"\\n\\n--- \" + relative + \" ---\\n\" + Path(relative).read_text()\n",
"print(context)\n",
"print(\"\\n[context: %d characters]\" % len(context))"
],
"execution_count": null,
"outputs": [],
Expand All @@ -126,154 +164,47 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Inspect the bundled simulated data\n",
"\n",
"The assistant's Gaussian is **peak-normalised**: normalization is its height,\n",
"not its integrated area. The simulation used centre 50, height 25 and width 10.\n",
"Its likelihood assumes independent Gaussian measurement errors and omits the\n",
"constant noise-normalisation term. Evidence values therefore use that convention.\n",
"\n",
"Ask Gemini: *What features of this plot should constrain the three parameters?*\n",
"The model and likelihood below are imported unchanged from the assistant's tour."
"## 3. Your sandbox\n",
"\n",
"This is now your assistant sandbox. Add your own cells and use Gemini in the\n",
"sidebar to do things in PyAutoFit — compose a model, wrap a likelihood, run a\n",
"search, read a result. Paste the cell or the error you are working on into your\n",
"message; the sidebar cannot reliably see the notebook.\n",
"\n",
"__Bring your own likelihood.__ The context above includes the assistant's BYOL\n",
"mode — overview, model, wrap and validate, choose a search, your go-ahead, then\n",
"run and inspect. Paste your own likelihood function into the sidebar and have\n",
"Gemini set PyAutoFit up around it. Two rules it is told to keep: your numerics are\n",
"never re-parametrised or \"fixed\", and nothing is fitted before you say go. Paste\n",
"the code itself, since the sidebar reads no files — and before fitting real data,\n",
"plot it and say what artefacts, outliers or selection effects you know of.\n",
"\n",
"The assistant's own worked example (data, model, likelihood, and the six steps\n",
"the guided tour walks) is in `scripts/start_here/` in the checkout. Run the cell\n",
"below to print any other assistant file when Gemini needs it, then paste the\n",
"output. For larger inference projects, continue in a local clone of\n",
"[autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant)."
],
"id": "cell-4"
},
{
"cell_type": "code",
"metadata": {},
"source": [
"import json\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"sys.path.insert(0, str(repo / \"scripts\" / \"start_here\"))\n",
"from gaussian import Gaussian\n",
"from analysis import Analysis\n",
"def show(relative):\n",
" \"\"\"Print any file from the assistant checkout, to copy into the Gemini sidebar.\"\"\"\n",
" print(Path(relative).read_text())\n",
"\n",
"data = np.asarray(json.loads(Path(\"dataset/gaussian_x1/data.json\").read_text()))\n",
"noise_map = np.asarray(json.loads(Path(\"dataset/gaussian_x1/noise_map.json\").read_text()))\n",
"x = np.arange(data.size)\n",
"plt.errorbar(x, data, yerr=noise_map, fmt=\".k\")\n",
"plt.xlabel(\"x\")\n",
"plt.ylabel(\"Profile amplitude\")\n",
"plt.show()"
],
"execution_count": null,
"outputs": [],
"id": "cell-5"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Compose the model and check the likelihood\n",
"\n",
"Ask Gemini: *Explain these priors. What would happen if I fixed the width?*\n",
"The check below compares the imported likelihood with its written equation."
],
"id": "cell-6"
},
{
"cell_type": "code",
"metadata": {},
"source": [
"model = af.Model(\n",
" Gaussian,\n",
" centre=af.UniformPrior(lower_limit=0.0, upper_limit=100.0),\n",
" normalization=af.UniformPrior(lower_limit=0.0, upper_limit=100.0),\n",
" sigma=af.UniformPrior(lower_limit=0.1, upper_limit=30.0),\n",
")\n",
"analysis = Analysis(data=data, noise_map=noise_map)\n",
"truth = Gaussian(centre=50.0, normalization=25.0, sigma=10.0)\n",
"expected = -0.5 * np.sum(((data - truth.model_data_from(x)) / noise_map) ** 2)\n",
"assert np.isclose(analysis.log_likelihood_function(truth), expected)\n",
"print(model.info)\n",
"print(\"Log likelihood at simulation inputs:\", expected)"
"# e.g. show(\"skills/README.md\") — one line per skill, to pick the next one\n",
"# show(\"skills/af_simulate_dataset.md\") — any skill in full\n",
"# show(\"wiki/core/index.md\") — the curated PyAutoFit reference\n",
"# show(\"scripts/start_here/analysis.py\") — the bundled Analysis class\n",
"show(\"skills/README.md\")"
],
"execution_count": null,
"outputs": [],
"id": "cell-7"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Run a small fit\n",
"\n",
"Run this cell when ready. It performs actual nested sampling with 50 live points,\n",
"chosen for experimentation, not precision results. Runtime varies. The same\n",
"completed run reloads if you execute this cell again. Change `name` for a new\n",
"experiment, especially after changing the data or likelihood.\n",
"\n",
"Ask Gemini: *Before I run this, what do you expect the recovered parameters to be?*"
],
"id": "cell-8"
},
{
"cell_type": "code",
"metadata": {},
"source": [
"search = af.DynestyStatic(\n",
" path_prefix=\"experimental_colab\", name=\"gaussian_x1\", nlive=50,\n",
" number_of_cores=1,\n",
")\n",
"result = search.fit(model=model, analysis=analysis)\n",
"print(result.info)\n",
"best = result.max_log_likelihood_instance\n",
"plt.errorbar(x, data, yerr=noise_map, fmt=\".k\", label=\"data\")\n",
"plt.plot(x, best.model_data_from(x), label=\"maximum likelihood fit\")\n",
"plt.xlabel(\"x\")\n",
"plt.ylabel(\"Profile amplitude\")\n",
"plt.legend()\n",
"plt.show()"
],
"execution_count": null,
"outputs": [],
"id": "cell-9"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. Keep the results and continue\n",
"\n",
"Ask Gemini: *Help me explain the parameter uncertainties and one limitation of\n",
"this fit. Then suggest one change I can predict and test.* Share `result.info`\n",
"with your question. The fit and residual figures are also under\n",
"`output/experimental_colab/` in the Files panel.\n",
"\n",
"Run this last cell to download the fit outputs plus the model, likelihood, data,\n",
"configuration and context used here. Save your notebook copy separately: the\n",
"archive does not contain your edited notebook or Gemini conversation. Download\n",
"before disconnecting. For larger inference projects, continue in a local clone\n",
"of [autofit_assistant](https://github.com/PyAutoLabs/autofit_assistant), carrying\n",
"your scientific question, priors, likelihood and results with you."
],
"id": "cell-10"
},
{
"cell_type": "code",
"metadata": {},
"source": [
"import zipfile\n",
"from google.colab import files\n",
"\n",
"archive = repo.parent / \"autofit_colab_results.zip\"\n",
"with zipfile.ZipFile(archive, \"w\", zipfile.ZIP_DEFLATED) as bundle:\n",
" for directory in [\"output/experimental_colab\", \"scripts/start_here\",\n",
" \"dataset/gaussian_x1\", \"config\"]:\n",
" for path in Path(directory).rglob(\"*\"):\n",
" if path.is_file() and \"__pycache__\" not in path.parts:\n",
" bundle.write(path, str(path))\n",
" bundle.writestr(\"gemini_context.txt\", context)\n",
" bundle.writestr(\"environment.txt\", subprocess.check_output(\n",
" [sys.executable, \"-m\", \"pip\", \"freeze\"], text=True))\n",
" bundle.writestr(\"assistant_ref.txt\", ASSISTANT_REF + \"\\n\" +\n",
" subprocess.check_output([\"git\", \"rev-parse\", \"HEAD\"], text=True))\n",
"files.download(str(archive))"
],
"execution_count": null,
"outputs": [],
"id": "cell-11"
"id": "cell-5"
}
],
"metadata": {
Expand Down
Loading
Loading