Repository navigation
site-build: the GPU leg; programming joins the base environment - #87
Conversation
Step 3 of bringing the sites into one environment for every site (QuantEcon/project-monorepo#77), by the owner's order of 2026-11-22: intro, then advanced, then the GPU sites, programming first. A site on the g4dn.2xlarge class now builds on a RunsOn machine with one NVIDIA T4, from QuantEcon's GPU machine image (by the AMI ID the GPU series' own runs-on.yml gives), with no container, and in the base environment alone. In the shape the owner chose on 2026-11-22, the one the environment scan of 2026-11-19 proved: setup-miniconda creates _base/environment.yml (Python 3.13 and anaconda=2026.07, the full image's base), conda env update applies _base/environment-update.yml, and jax[cuda13] at the list's jax version is added, with a check that jax finds the GPU. tools/site-matrix gives each entry its leg and RunsOn machine spec, builds RunsOn's label from --run-id, and refuses a GPU site outside the base environment. The job's container is empty for a GPU site. Its cache key hashes the machine spec and _base/environment.yml in place of the image's marker file. Stage and unstage run in a login shell, where the GPU machine has the environment's python. _base/environment-update.yml is untouched, so intro's and advanced's caches keep their keys. The READMEs name programming and the GPU leg. Three stale lines from the reviews of #85 and #86 are corrected on the way: the count of sites still to move in sites/README.md, the root README's "dp alone", and the workflow's "(dp publishes HTML alone)". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WT8bMacV9Lmy1VLUc6XJoL
Validation: programming built from the pool on the GPU leg, against programming's own buildsBuilds compared, all of programming's
The GPU leg's environment. setup-miniconda created Noise baseline. programming's two own builds differ on 13 of 31 pages: random draws, memory addresses and object identities, timings and download progress. This pull request's build differs from the first on 15 pages: the same 13, plus two where every difference is explained.
On the 13 noise pages, every line that differs is a random value, an address, a timing or install output, as between programming's own two builds. The status page differs only in its times. One file only programming's build has: Warnings. The build log has no Sphinx warning; its only warning lines are Node.js deprecation notices from GitHub's own actions. The cell outputs carry the same warnings and tracebacks as programming's own build, page by page. These are the lectures' own demonstrations: names lists the built-in names, the warning classes among them, and debugging, numpy, oop_intro, python_advanced_features and python_essentials raise errors on purpose. The other legs. intro and dp restored their execution caches under unchanged keys, with every page taken from the cache. So the string form of |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block approval of the GPU build and shared-environment changes.
No open issues 🎉
What changed in this PR
Adds a GPU build leg to the shared site-build workflow and moves programming into the common base environment, following intro and advanced.
Changes:
- Adds RunsOn GPU runner support, Conda environment setup, and CUDA-enabled JAX with a GPU check.
- Enables programming’s strict HTML build and GPU-specific execution cache keys while preserving CPU cache keys.
- Extends matrix tests and updates documentation for the four enabled sites.
| File | Description |
|---|---|
tools/tests/test_site_matrix.py |
Tests GPU matrix entries, invalid configurations, and workflow run-ID forwarding. |
tools/site-matrix |
Adds GPU runner settings and run-ID validation. |
sites/README.md |
Documents GPU builds, environment selection, and caching. |
sites/programming/site.yml |
Selects the shared base environment for programming. |
README.md |
Updates the build overview to include all four enabled sites. |
.github/workflows/site-build.yml |
Adds GPU setup, CUDA verification, cache handling, and programming builds. |
_base/README.md |
Explains the GPU base environment and shared package list. |
_base/environment.yml |
Defines the GPU base with Python 3.13 and Anaconda 2026.07. |
Review effort: Balanced (adaptive)
The organiser's review of #87; the owner chose "Fix the channel, then merge". _base/environment.yml named the channel `default`, copied from programming's environment.yml. That is the anaconda.org user channel "default", empty today, which conda would search first; the Anaconda channels are `defaults`. The GPU leg's cache key hashes this file, so programming's key changes. Wording: the base environment's container and root-user notes say they are the container leg's; extra_packages go into "the environment jb runs in"; sites/README's environment row names the GPU leg; _base/README no longer hard-codes the jax version the workflow reads from the list. The cache-key step tests the leg, as the other steps do. _base/environment-update.yml's own comment still names 0.12.1: the base cache key hashes that whole file, so it waits for the plotly-orca pull request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WT8bMacV9Lmy1VLUc6XJoL
Validation of
|
What
Step 3 of bringing the sites into one environment for every site (QuantEcon/project-monorepo#77), in the owner's order of 2026-11-22: intro (#84), advanced (#86), then the GPU sites, programming first. This writes the GPU leg and builds programming's site on it.
The GPU leg's shape is the owner's choice of 2026-11-22: "The scan's shape (Recommended)", that is, "setup-miniconda with python=3.13 + anaconda=2026.07 on the RunsOn machine, then the list, then jax[cuda13]==0.12.1 with a GPU check; the design note revised to match". The environment scan of 2026-11-19 built all three GPU sites in this shape with nothing new. It gives a GPU site the same environment as a CPU site: the RunsOn machine image's own Conda environment has no
anacondametapackage, so updating that one with the list (the design note's first plan) would not have.tools/site-matrixg4dn.2xlargeclass has a leg. Each entry gainsleg(containerorgpu) andmachine(RunsOn's spec:family=g4dn.2xlarge/ami=ami-0edec81935264b6d3/volume=80gb/spot=false, the AMI the GPU series' own.github/runs-on.ymlnames). A GPU site'sruns_onis RunsOn's label,runs-on=<run ID>/<machine>, from the new--run-id, and itsimageis empty. A GPU site outsideenvironment: baseis refused..github/workflows/site-build.ymlcontainer: ${{ matrix.site.image }}, empty for a GPU site, so its job runs on the machine. Three GPU steps: setup-miniconda creates_base/environment.yml;conda env updateapplies_base/environment-update.yml; thennvidia-smi,jax[cuda13]at the list's jax version, and a check thatjax.devices()is a GPU. A GPU site's cache key hashes the machine spec and_base/environment.ymlin place of/etc/quantecon-container. Staging and unstaging run in a login shell, where the GPU machine has the environment'spython.SITES: dp intro advanced programming. Comments updated._base/environment.ymlanaconda=2026.07, as the full image carries it.sites/programming/site.ymlenvironment: basetools/tests/test_site_matrix.pysites/README.md,_base/README.md,README.md_base/environment-update.ymlis not touched, so intro's and advanced's caches keep their keys. The job limit for programming is 130 minutes (cell_timeout600 s plus 120). It builds HTML alone for now.Validation
Done on 2026-11-22; the evidence is in the comment below. Programming's leg builds green and strict on the RunsOn GPU machine, with no container, in 7 minutes. jax 0.12.1 finds the T4, every page with code is executed, and all 39 pins are installed with
jax[cuda13]on top. Against two fresh builds of programming's own atbae31ee, every difference beyond their noise is install output or the working directory a cell prints. The cell-output warnings are the same, and the build log has no Sphinx warning. The container legs still build: intro and dp restored their caches under unchanged keys, and advanced built cold and green.🤖 Generated with Claude Code
https://claude.ai/code/session_01WT8bMacV9Lmy1VLUc6XJoL