[wmlba]

Fixes #238.

What

run_tests_for_one_example now caps the forked test process's address space before running generated code. The cap is the size inherited from the evaluator plus LCB_MEMORY_HEADROOM_BYTES:

• default 2 GiB;
• set via ROUTERARENA_LCB_MEMORY_HEADROOM_BYTES, where 0 disables it.

It is applied with resource.setrlimit(RLIMIT_AS, ...) before reliability_guard() hides the resource module. Generated code that allocates past the cap raises MemoryError, which the existing per-test except Exception records as a failed test. On non-Linux systems, where /proc is unavailable, the function does nothing.

Why

Without a memory limit, one completion that allocates without bound grows until the wall-clock timeout. On the 16 GB ubuntu-latest runner, memory-pressure handling terminates the whole evaluation job first (exit code 143), so no results are posted. See the issue for details.

Validation

Check	main	This PR
Synthetic runaway completion	reached a 24 GiB cgroup cap in 11.7 s	fails in 1.2 s, 1.6 GiB peak
Synthetic correct completion	score 1	score 1
#236 file (9,209 rows), 16 GiB cgroup, no swap	killed (CI: exit 143)	exit 0, 8.0 GiB peak
#237 file (8,400 rows), same limits	23.5 GiB peak (CI: exit 143)	exit 0, 7.9 GiB peak
KT-ModelRouter LiveCodeBench rows (385)	–	0 scores changed

For cruq-router and nadir-cascade-v2 (385 LiveCodeBench rows each), 3 and 2 rows differed between runs with and without the fix:

• the 0→1 differences match OOM kills that occurred only without the fix;
• the 1→0 differences were the same problem, which passes 3/3 with and without the fix when re-run alone.

ruff check, ruff format --check and mypy (Python 3.9 and 3.11, repository config) pass.

[wmlba]

Verification on GitHub's standard runner:

I ran this repository's own pr-evaluation.yml, unchanged, in a public repository generated from main @ 04c99fd (identical tree). The base branch had this PR's change and the head had the exact #237 prediction files (identical blob SHAs).

• Evaluation run: https://github.com/wmlba/routerarena-verification/actions/runs/40162661126
  • /evaluate completed successfully in about 18.5 min on ubuntu-latest, with 0 abnormal entries.
  • The same prediction files were killed with exit 143 during the LiveCodeBench phase in the Add Compass CoreRouter submission (compass-corerouter) #236 and Add Compass CoreRouter submission (compass-corerouter, resubmission of #236) #237 runs here.
• Pre-commit CI (ruff + mypy matrix 3.9–3.12) passed on the change: https://github.com/wmlba/routerarena-verification/pull/1/checks

The verification PR is wmlba/routerarena-verification#1. Once this is merged, I'll update #237 from main and re-run /evaluate there so the official result comes from this repository.