Skip to content

site-build: the base environment, the full image with the list; intro joins - #84

Merged
mmcky merged 2 commits into
mainfrom
site-build/intro
Nov 22, 2026
Merged

mmcky merged 2 commits into
mainfrom
site-build/intro

Conversation

@mmcky

@mmcky mmcky commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

What

The first site brought into one environment for every site (QuantEcon/project-monorepo#77; the design note is ideas/one-environment-for-every-site.md in QuantEcon/project-monorepo), in the owner's order of 2026-11-22: intro first, then advanced, then the GPU sites (QuantEcon/project-monorepo#77, comment of 2026-11-22). site-build.yml now builds dp and intro.

site.yml gains an optional field, environment:

Value Image What the leg installs The cache key hashes
series (the default; dp) ghcr.io/quantecon/quantecon-build:latest, the lean image dp's own workflows build in extra_packages only mirror/<key>/environment.yml
base (intro) ghcr.io/quantecon/quantecon:latest, the full image (anaconda=2026.07, the base every series names) _base/environment-update.yml, through QuantEcon/actions' setup-environment, then extra_packages _base/environment-update.yml, with the key tagged base-

dp stays in series until its first shadow comparison, so that comparison measures the move of the build alone (the design note's decision 4).

File Change
.github/workflows/site-build.yml in the base environment: apply the list, set PIP_ROOT_USER_ACTION=ignore, build matplotlib's font cache; every leg records pip freeze beside its execution report; the cache key takes its file and tag from the matrix; _base/** joins the triggers; SITES: dp intro; the header comments say all of it
tools/site-matrix reads environment (default series) into environment, image, env_file and cache_tag, and refuses an unknown value
tools/pool-check the site-yml rule allows environment, series or base
tools/tests/test_site_matrix.py, tools/tests/test_sites.py the new field, both values, an unknown value and the default
sites/intro/site.yml environment: base
sites/README.md the field, the two images, the base leg's steps and the cache key

Why the two extra steps. The environment scan of 2026-11-19 built intro in this environment: 56 pages, nothing new, except pip's "Running pip as the 'root' user" warning in 15 pages' install cells, because the full image runs as root and the series' own builds do not. PIP_ROOT_USER_ACTION=ignore silences it. Building the font cache first keeps "Matplotlib is building the font cache" out of the first lecture that plots, as the series' own workflows now do.

Not in this pull request: intro's PDF and notebooks (every leg builds HTML alone, as dp publishes); a shared cache between sites of one environment (the design note's later step); the GPU leg.

Checks

  • tools/tests: 328 passed, 1 skipped.
  • tools/pool-check: clean.
  • tools/site-matrix dp intro: dp unchanged (series, the lean image, mirror/dp/environment.yml, no tag); intro in base.

Validation

Done on 2026-11-22; the evidence is in this comment. The intro leg builds green and strict (-n -W) from the pool in the full image with the list. Its HTML was compared page by page with two of intro's own builds of the same content: every difference beyond their noise is the full image's install output, citation anchors, or the merged bibliography's corrections of three of intro's malformed entries, plus one figure path in the pool's layout. The cell-output warnings are the same, and all 39 pins are installed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WT8bMacV9Lmy1VLUc6XJoL

@mmcky

mmcky commented Nov 22, 2026

Copy link
Copy Markdown
Contributor Author

Validation: intro built from the pool against intro's own builds

Builds compared, all of the same content (intro's lectures/ at 8db8a99, tree 503ecea):

Build Run What
This pull request, intro leg 44991332863 (at de0051d, before the rebase) from the pool, in the full image with _base/environment-update.yml, strict (-n -W), cold: 56 pages executed, 13.5 minutes
intro's own, fresh QuantEcon/lecture-python-intro 44991424113 cache.yml on main, 2026-11-22, the same day
intro's own, earlier QuantEcon/lecture-python-intro 44406850830 QuantEcon/lecture-python-intro#872's validation build, the same tree

Noise baseline. intro's two own builds differ on 31 of 66 pages: image file names, unseeded random outputs, timings, memory addresses and the status page. This pull request's build differs from intro's fresh build on 45 pages. Every difference beyond the noise is one of these:

  • Install cells (business_cycle, input_output, markov_chains_II, observed_distributions, bivariate_dist and others): the full image already holds each package at the list's version, so pip prints "Requirement already satisfied … /opt/conda/…" where intro's build prints "Collecting … Downloading … /home/runner/miniconda3/…". Same versions both sides (wbgapi 1.0.14, quantecon_book_networks 1.6, yfinance 1.8.0).
  • Citation anchors (on many pages, and the only difference on 12 of them, such as cagan_adaptive and cobweb): the pool's merged bibliography numbers the references page's anchors differently. Each page cites the same works.
  • Corrected references (lp_intro, heavy_tails, wealth_tax, zreferences): the merged bibliography's rulings of 2026-11-02 (QuantEcon/project-monorepo#10) correct three of intro's entries. bertsimas_tsitsiklis1997 and hu_guo2018 joined their authors with "&", so intro's pages cite "Bertsimas, 1997" and "Hu, 2018"; pareto1896cours gave "Vilfredo, Pareto", so intro's pages cite "Vilfredo, 1896". A doubled https://doi.org/https://doi.org/ link is corrected too. These are fixes, not regressions; intro's own file could take them upstream.
  • long_run_growth: one figure path, rewritten by tools/promote to the pool's layout (_static/long_run_growth/), as the ledger records.
  • Random values and timings of the noise's own kind (heavy_tails, inequality, lln_clt_intro, markov_chains_I, monte_carlo, simple_linear_regression).

Warnings. The build log has no Sphinx warning (the strict build would fail on one). Its only warning lines are Node.js deprecation notices from GitHub's own actions and conda's update notice. The cell outputs carry the same two warnings as intro's own build, once each: a UserWarning in fitting_distributions and a Pandas4Warning in inflation_history (both pre-existing, upstream).

Environment. All 39 pins in _base/environment-update.yml are installed at their versions: 32 by pip, six (pandas, plotly, rich, seaborn, statsmodels, tabulate) already satisfied at their pins in the image, and plotly-orca 1.3.1 by conda.

After the rebase on main a693c16, which adds only lectures outside intro's site and bibliography entries intro does not cite, the intro leg was built again, green and strict: 44993150231 at 579af32, cold, 11 minutes. Compared with the validated build it differs only in citation anchors (the merged bibliography grew by 31 entries, so the references page's anchors are renumbered; its entries are unchanged) and in the noise pages' own outputs (fitting_distributions, inflation_history, markov_chains_I, mle_intro, money_inflation_nonlinear, monte_carlo, simple_linear_regression, status). dp's leg is unchanged by this pull request and green on it.

@mmcky
mmcky marked this pull request as ready for review November 22, 2026 06:51
Copilot AI adaptive review requested due to automatic review settings November 22, 2026 06:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Non-string YAML environment values cause an uncaught TypeError in tools/site-matrix, producing a traceback instead of reporting invalid input and collecting other sites’ refusal reasons.

1 open issue
What changed in this PR

This PR adds intro to the shared site-build workflow as the first site using the common base environment, while leaving dp in its existing series environment.

Changes:

  • Adds an optional environment setting to select the container image, package setup, and cache key.
  • Enables intro’s HTML build with the shared package list, warning suppression, and font-cache preparation.
  • Records installed packages alongside execution reports and updates validation, tests, and documentation.
File Description
tools/​tests/​test_sites.py Tests validation of the optional environment setting.
tools/​tests/​test_site_matrix.py Tests environment defaults, image selection, cache settings, and rejection.
tools/​site-matrix Maps environments to images and cache-key inputs.
tools/​pool-check Allows and validates the environment field.
sites/​README.md Documents environment selection and the expanded build workflow.
sites/​intro/​site.yml Selects the base environment for intro.
.github/​workflows/​site-build.yml Adds intro, base-environment setup, package records, and environment-specific caching.

Review effort: Balanced (adaptive)

Comment thread tools/site-matrix Resolved
mmcky and others added 2 commits November 22, 2026 18:10
… joins

The first site brought into one environment for every site
(QuantEcon/project-monorepo#77; the design note is
ideas/one-environment-for-every-site.md there), by the owner's order of
2026-11-22: intro, then advanced, then the GPU sites.

site.yml gains an optional `environment`. `series`, the default, is what
dp builds in today and keeps until its first shadow comparison: the lean
image ghcr.io/quantecon/quantecon-build:latest, nothing installed but
extra_packages, the cache keyed on mirror/<key>/environment.yml. `base` is
the full image ghcr.io/quantecon/quantecon:latest with
_base/environment-update.yml applied by QuantEcon/actions'
setup-environment, the cache keyed on that list, its key tagged `base-`.
In the base environment the leg also sets PIP_ROOT_USER_ACTION=ignore (the
full image runs as root, and the environment scan of 2026-11-19 found
pip's root-user warning in 15 of intro's install cells, which the series'
own builds do not print) and builds matplotlib's font cache first.

tools/site-matrix reads the field (environment, image, env_file,
cache_tag) and refuses an unknown value; tools/pool-check's site-yml rule
allows series and base; both have tests. Every leg now records pip freeze
beside its execution report. _base/** joins the workflow's triggers, so a
change to the list rebuilds the sites built with it. dp's leg and its
cache key are unchanged. sites/intro/site.yml sets environment: base, and
SITES is dp intro. intro builds HTML alone for now, as dp does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WT8bMacV9Lmy1VLUc6XJoL
…not a traceback

From the review of this pull request (Copilot's thread on
tools/site-matrix, and the organiser's reviewer). `environment: [base]`
or `{name: base}`, and the same for `runner`, raised TypeError on the
table lookup, so site-matrix exited with a traceback and stopped
collecting the other sites' refusals. Both lookups now check for a string
first and refuse anything else as they refuse an unknown name; a test
covers a list and a mapping for each.

Also: the workflow records the environment with `pip list --format=freeze`,
since pip freeze gives a package conda installed (pandas, plotly, rich,
seaborn, statsmodels, tabulate) as `name @ file:///...` with no version;
and the header of _base/environment-update.yml and _base/README.md no
longer say that nothing builds with the list, that the image holds no
Chrome (both images carry Chrome for Testing 158.0.8127.36 at /opt/chrome
since QuantEcon/actions#346), or that the series are still being brought
to the baseline. No base-environment cache exists yet, so changing the
list's comments costs no cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WT8bMacV9Lmy1VLUc6XJoL
@mmcky
mmcky merged commit e1ce0bb into main Nov 22, 2026
8 checks passed
@mmcky
mmcky deleted the site-build/intro branch November 22, 2026 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants