Skip to content

feat(agents): pygal and lets-plot through the Python runtime (step 1c) - #12405

Merged
MarkusNeusinger merged 6 commits into
mainfrom
claude/vigilant-mccarthy-8euyyw
Oct 26, 2026
Merged

feat(agents): pygal and lets-plot through the Python runtime (step 1c)#12405
MarkusNeusinger merged 6 commits into
mainfrom
claude/vigilant-mccarthy-8euyyw

Conversation

@MarkusNeusinger

@MarkusNeusinger MarkusNeusinger commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

  • pygal and lets-plot get runtimes (agentic/specs/261024-agent-all-libraries.md, step 1c), next to matplotlib, seaborn, plotnine and altair. Each has a save rule (regions.SAVE_RULES: pygal's render_to_png, render_to_file and a keyword-only cairosvg.svg2png, each name pinned to its own file type; lets-plot's ggsave with path="."), a validator profile and a canvas rewrite. pygal's 4800x2700 and 3600x3600 charts save through svg2png at 3200x1800 and 2400x2400; lets-plot changes only scale=. Both were over R2's pixel cap in a third of the catalogue.

    • Validator. pygal: cssselect2, tinycss2 and cairocffi stay unreachable, plus a css/js/defs config rule and svg2png's fetch keywords refused. lets-plot: geo_data, tilesets, livemap and the browser contexts are denied, LetsPlot is allowed only as setup_html(), .props is banned, and literal tooltip-builder chains are exempt from the .format ban.
    • Normaliser. lets-plot's star import is expanded from the static lets_plot.__all__ of 4.11.0, path="." is added and the lets-plot-images epilogue removed; pygal's HTML writes become render_to_file or are dropped. Four guard rules apply to every library.
    • Harness. cairosvg loads before the guard; cssselect2's selector compiler joins CODE_BUILDERS after a review showed that CSS text cannot steer it.
    • Renderer. The lets-plot row runs with its own RLIMIT_AS of 2048 MiB and Arrow's system allocator (RuntimeCommand.rlimit_as_mb); 1024 MiB kills every export. The image adds lib-pygal, lib-letsplot, libcairo2 and DejaVu Sans (+49 MB, 1.17 GB).

  • Shared fixes.

    • Validator. Every profile refuses a save name imported under another name or used as a value (from plotnine import ggsave as keep wrote anywhere in exported code) and pins THEME to the theme block. A match capture can no longer rebind a plotnine expression name. All three passed origin/main's validator.
    • R2. The blank check moves from 98 % to 99.7 % one colour, for every library: sparse pygal charts reach 99.44 %, a chart with only a title 99.78 %. A higher threshold refuses nothing 0.98 accepted.
    • Image build. Every build of the renderer image fails when one of 39 ImageMagick delegate programs is on the sandbox PATH (agents/renderer/delegates.py), so the Cloud Build deploy enforces it too; the bridge's ImageMagick is a zero-configuration build that reads no policy. ci-image.yml renders pygal (both save forms) and lets-plot through the harness and runs the delegate check again in the built image.
    • Local backend. PythonRuntime.env, which LocalDockerBackend passes, is now the renderer's sandbox_env key for key (sandbox PATH, file-size and address-space limits), except RLIMIT_NPROC, which --pids-limit replaces because the container runs as the developer's uid.

  • Eligibility (agents.evals.eligibility, 2026-10-26):

    Library Eligible
    pygal 209 of 325
    lets-plot 284 of 325
    plotnine 267, was 264
    altair 273, was 269
    matplotlib, seaborn 285 and 287, normalised output byte-identical

    The eval fixtures gain 120 cases (360 generated, smoke 16). Nothing changes for users until AGENT_LIBRARIES names a new library.

Plan

agentic/specs/261024-agent-all-libraries.md, step 1c ("What 1c built", the decisions table, "Eligibility after 1c", "Next"). The owner's image decision of 2026-10-25 (question 1: a private toolchain base image in Artifact Registry) is recorded too.

Test plan

  • uv run ruff check and ruff format --check on agents, tests, api and core; uv run mypy api core agents; uv run python -m tools.changelog check --base origin/main
  • uv run pytest tests/unit: 18,240 passed; the 5 failures are the cloud container's known environmental ones in tests/unit/agentic/regen and test_retest_set_v2.py, unchanged by this branch
  • Two reviews of the core commit: a security review (no sandbox escape, no validator bypass with an effect) and a correctness review that rendered all 208 eligible pygal and 284 lets-plot run forms against the catalogue (pixel-identical at the same size; downscaled canvases within 1.3 grey levels; idempotent over 1,946 files and 17,549 fuzzed inputs). Its findings are fixed in ec98dc0
  • Local Docker: the renderer image builds; every renderer-image step of ci-image.yml passes against it (seaborn, plotnine and altair byte-identical to the 1b image; pygal 3200x1800 in 1.6 s and 185 MB; lets-plot 3200x1800 in 5.1 s and 665 MB); LocalDockerBackend renders scatter-basic for all six Python libraries in both themes with the new environment; a derived image with an executable gs fails the build's delegate check
  • make_fixtures --check exits 0; no existing case changed
  • CI green on 2afe4e8, the renderer-image job included
  • Owner: the matrix runs for pygal and lets-plot (AGENT_LIBRARIES naming them), then _LIBRARIES in agents/cloudbuild.yaml; for pygal, 12 of its 60 cases open blocked sessions (histogram-basic, line-timeseries), so the 80 % gate needs a decision (count only eligible pairs, regenerate the two files, or accept)
  • Owner, before enabling lets-plot: accept the bridge's containment (build-time delegate check, RLIMIT_NPROC, sandbox) or add a seccomp execve deny

Generated by Claude Code

claude added 3 commits October 25, 2026 23:50
@claude
docs(agents): record the image decision in the all-libraries plan
4d807ce 
The owner chose option B for question 1: a toolchain base image built on
lock-file changes, private in the Artifact Registry repository next to the
renderer image, rather than public on GHCR. Question 11 is settled by it, the
status line names 1a and 1b as merged with #12404, and option A's description
no longer claims an hour-long build on every change (true only without a
layer cache).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
@claude
feat(agents): pygal and lets-plot through the Python runtime (plan st…
5cf6cd7 
…ep 1c, core)

The runtime registry serves pygal and lets-plot next to matplotlib, seaborn,
plotnine and altair.

Save rules: pygal saves through render_to_png, render_to_file and
cairosvg.svg2png (a keyword-only write_to=; libcairo writes the PNG with its
own fopen, so the file rests on the static pin), lets-plot through ggsave
with path="." (SaveRule.directory). SaveRule.canvas names each library's
canvas rewrite.

Validator: the pygal profile (pygal, cairosvg; cssselect2, tinycss2 and
cairocffi unreachable; the css/js/defs config rule; svg2png's fetch keywords
refused) and the lets-plot profile (geo_data, tilesets, livemap and the
browser contexts denied; LetsPlot only as setup_html(); .props banned; a
.format exemption for literal tooltip and label builder chains). In every
profile a save name imported under another name or used other than as a
call's func is refused, closing a route the shipped plotnine profile left
open, and a match capture can no longer rebind a plotnine expression name.

Normaliser: four library-agnostic guard rules (find_spec, try/finally,
sys.modules deletion, import_module attributes); pygal's HTML writes become
render_to_file or are dropped; lets-plot's star import is expanded from the
static lets_plot.__all__ of 4.11.0, its export_ggsave alias renamed, path="."
added and the lets-plot-images epilogue removed. Canvas rewrites: pygal's
4800x2700 and 3600x3600 charts save through svg2png at 3200x1800 and
2400x2400; lets-plot changes only scale= on the PNG save.

Harness: cairosvg is imported before the guard is armed, and cssselect2's
selector compiler joins CODE_BUILDERS. Renderer: the lets-plot row runs
with its own RLIMIT_AS of 2048 MiB, Arrow's system allocator and a minimal
fontconfig file (RuntimeCommand.rlimit_as_mb; sandbox_env and the local
backend pass an in-process row's env). The image installs lib-pygal,
lib-letsplot and libcairo2 and links DejaVu Sans; ci-image.yml renders a
pygal and a lets-plot plot through the harness.

Eligibility: pygal 208 of 325, lets-plot 284 of 325; plotnine 267 and
altair 273 gain from the guard rules; matplotlib and seaborn unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
@claude
fix(agents): step 1c review findings, eval fixtures, docs and plan
ec98dc0 
Two reviews of the step-1c core (a security review and a correctness review
with renders of every eligible pygal and lets-plot file) found no sandbox
escape and no validator bypass with an effect; this commit fixes what they
did find.

- R2's blank check refuses a canvas that is 99.7 % one colour, not 98 %,
  for every library: 33 of 195 eligible pygal renders reach 98 % to 99.44 %
  legitimately, while a pygal chart with only a title is 99.78 %. A higher
  threshold refuses nothing 0.98 accepted.
- pygal: the canvas rewrite reads the root <svg> of a hand-built SVG (four
  eligible files rendered over R2's pixel cap), a size set after the chart's
  constructor makes the canvas unknown instead of 800x600, and each save
  name writes only its own file type (SaveRule.writes). html-drop takes a
  body that assigns names nothing else reads, which unblocks
  heatmap-basic/pygal (209 eligible).
- Every profile pins THEME to the theme block: pygal's PNG is written
  natively, where the plot guard cannot see the path, and origin/main's
  validator accepted a rebinding.
- lets-plot: the fit under R2's cap includes the exact scale, a
  conditionally bound name keeps the star import, every returned path is
  unwrapped, and the import order matches ruff.
- Renderer: sandbox_env sets PATH and HOME after a row's env, a CI step
  fails when an ImageMagick delegate program is on the sandbox PATH (the
  bridge's ImageMagick is a zero-configuration build that reads no policy),
  the lets-plot CI step takes its env from the command table, and the
  pygal step renders the svg2png canvas form too.

Also: 120 eval fixture cases for pygal and lets-plot (360 generated, smoke
16; histogram-basic and line-timeseries stay blocked for pygal), the plan's
"What 1c built", docs for six libraries, and the changelog fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
Copilot AI lite review requested due to automatic review settings October 26, 2026 05:42
Copilot AI reviewed Oct 26, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

One or more issues must be addressed before approval.

2 open issues
What changed in this PR

This PR extends the agent rendering pipeline to support pygal and lets-plot through the Python runtime while tightening validation, normalization, rendering limits, and evaluation coverage.

Changes:

  • Adds pygal and lets-plot runtime, save rules, validators, normalizers, canvas handling, and renderer image dependencies.
  • Adds 120 generated evaluation cases plus runtime, validator, normalization, and renderer tests.
  • Raises the blank-image threshold to 99.7% and adds native bridge/font configuration for lets-plot.
File Description
tests/​unit/​agents/​runtime/​test_runtimes.py Updated as part of this pull request.
tests/​unit/​agents/​runtime/​test_render.py Updated as part of this pull request.
tests/​unit/​agents/​renderer/​test_commands.py Updated as part of this pull request.
tests/​unit/​agents/​evals/​test_cases.py Updated as part of this pull request.
tests/​unit/​agents/​code/​test_letsplot_names.py Updated as part of this pull request.
tests/​unit/​agents/​code/​conftest.py Updated as part of this pull request.
core/​canvas.py Updated as part of this pull request.
agents/​renderer/​fonts.conf Updated as part of this pull request.
agents/​renderer/​executor.py Updated as part of this pull request.
agents/​evals/​report.py Updated as part of this pull request.
agents/​evals/​matrix.py Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​violin-basic-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​scatter-basic-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-x10/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-renamed/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-decimal-comma/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-date/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-x10/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-renamed/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-decimal-comma/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-date/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​pie-basic-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-timeseries-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-x10/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-renamed/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-decimal-comma/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-x10/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-renamed/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-decimal-comma/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​line-basic-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​histogram-basic-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​heatmap-basic-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​box-basic-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-x10/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-renamed/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-decimal-comma/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-date/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-x10/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-renamed/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-decimal-comma/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-date/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​bar-grouped-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-x10/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-renamed/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-decimal-comma/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-pygal-date/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-x10/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-renamed/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-renamed/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-n5000/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-n12/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-n12/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-decimal-comma/​data.csv Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-decimal-comma/​case.json Updated as part of this pull request.
agents/​evals/​fixtures/​cases/​area-basic-letsplot-date/​case.json Updated as part of this pull request.
agents/​evals/​eligibility.py Updated as part of this pull request.
agents/​evals/​cases.py Updated as part of this pull request.
agents/​anyplot/​settings.py Updated as part of this pull request.
agents/​anyplot/​render/​runtimes/​__init__.py Updated as part of this pull request.
agents/​anyplot/​render/​png.py Updated as part of this pull request.

Review effort: Lite

Comment thread .github/workflows/ci-image.yml Outdated
Comment thread agents/renderer/commands.py
Comment on lines +138 to +154
# lets-plot rasterises in its Kotlin/Native bridge (ImageMagick linked in). An allocation it cannot
# make under RLIMIT_AS kills the process ("Out of memory trying to allocate ... Aborting."): by
# SIGABRT (exit 134) on a developer machine and by SIGSEGV (exit 139) in the renderer image, with
# either Arrow pool (measured on 2026-10-26). pyarrow's mimalloc pool reserves a 1 GiB arena on the
# first pandas string column, which kills even a minimal 3200x1800 chart at 1024 and 1536 MiB, so
# the row takes Arrow's system allocator, as conversions do. With it, 18 of the spike's 42
# canvas-size renders still die at the configured 1 GiB; 2048 MiB held all 42 renders of the spike
# of 2026-10-26 up to 4800x2700 and 3600x3600 (VmPeak 1697 MiB) and still stops a 6400x3600 one
# (2580 MiB). In the renderer image with this row the canvas-size renders took a wall p95 of 7.69 s
# (max 15.5 s); the guarded host sweep of the 284 eligible catalogue files, canvas rewrite applied,
# 12.5 s.
return RuntimeCommand(
language,
code_file,
(),
env=MappingProxyType({"ARROW_DEFAULT_MEMORY_POOL": "system", "FONTCONFIG_FILE": FONTCONFIG_FILE}),
rlimit_as_mb=2048,

@MarkusNeusinger MarkusNeusinger Oct 26, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that the PATH check is containment, not an enforcement boundary. This PR does not enable lets-plot: AGENT_LIBRARIES keeps its default matplotlib,seaborn, and the settings accept letsplot only when the owner names it. Enabling it waits on the owner's call, which the plan's "Next" and this PR's test plan record: accept the containment, or add a seccomp filter that denies execve in the render container first.

What the containment rests on today: the security review of 2026-10-26 ran real lets-plot renders under the harness with strace -f and found no route from code the validator admits to a delegate. Text with ImageMagick coder prefixes (@, caption:, label:, msl:) never reaches a coder, geom_imshow builds an in-memory data: href and reads no file, and a plain export starts no process. With the next commit (the thread on ci-image.yml), every image build enforces that no delegate program is on the sandbox's PATH, RLIMIT_NPROC bounds the process count, and the sandbox has no network and a read-only root. Leaving this thread open for the owner's decision.

Generated by Claude Code

@codecov

codecov Bot commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.94656% with 8 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
agents/anyplot/render/harness.py 11.11% 8 Missing ⚠️

📢 Thoughts on this report? Let us know!

@claude
fix(renderer): enforce the delegate PATH check in every image build
ac14b3f 
The Copilot review of #12405: the check that no program lets-plot's native
ImageMagick can start is on the sandbox's PATH ran only in the CI image job,
while the production image is built by Cloud Build, so a base-image or
dependency change could reach production unseen. The list moves into
agents/renderer/delegates.py, the Dockerfile runs it (every build fails when
one of the 39 programs is on SANDBOX_PATH), and the CI step runs the same
module once more in the built image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
@MarkusNeusinger
MarkusNeusinger requested a lite review from Copilot October 26, 2026 05:56
Copilot AI reviewed Oct 26, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

One or more issues must be addressed before approval.

2 open issues
1 resolved since last review

Review effort: Lite

Comment thread tests/unit/agents/runtime/test_runtimes.py Outdated
@claude
Give the local render the renderer's sandbox environment
2afe4e8 
PythonRuntime.env, which the local Docker backend passes, now matches
executor.sandbox_env for the library's row key for key: the sandbox PATH
the image build's delegate check reads, the BLAS thread caps, the
deployed file-size limit and RLIMIT_AS (the row's own, or 1 GiB unless
the row switches it off). RLIMIT_NPROC stays out: the container runs as
the developer's uid, whose processes on the whole machine count against
it, so --pids-limit bounds them instead. A test pins the parity and the
deployed limit constants. The agent-network doc names the build-time
delegate check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
@MarkusNeusinger
MarkusNeusinger merged commit c1c99c4 into main Oct 26, 2026
15 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the claude/vigilant-mccarthy-8euyyw branch October 26, 2026 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@MarkusNeusinger @claude