Skip to content

Characterize the headless harness's scenario checks and SDL driver defaults - #69

Merged
dmccoystephenson merged 1 commit into
mainfrom
feature/characterize-headless-run-checks
Nov 15, 2026
Merged

dmccoystephenson merged 1 commit into
mainfrom
feature/characterize-headless-run-checks

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Member

Summary

Stage B (unit-test expansion) cycle for headless_run.py. The README's Running without Docker or a display section states that a scenario fails if no entry is recorded in environments.json, if the render loop does not draw the requested number of frames, or if Pygame is left initialised, and that the dummy SDL drivers are used unless SDL_VIDEODRIVER or SDL_AUDIODRIVER is already set. Before this PR, only the "main() raised" path and the end-to-end frame-count path were exercised; the other checks and the driver default had no test.

Five tests are added to tests/test_headless_run.py; no production code is changed:

  • TestRunScenario — each of runScenario's three post-run checks is triggered on its own, and the exact FAIL <scenario>: <problem> line is asserted:
    • an environments.json with no entry for the grid size (main.main replaced by a function that writes {})
    • a render scenario that draws no frames (rendered 0 render-loop frame(s), expected 2)
    • Pygame reported as still initialised (pygame.get_init patched to return True, with the real main() otherwise running)
  • TestSdlDrivers — run() sets both drivers to dummy when they are unset, and leaves values that are already set in place (runScenario and installVironStubs are patched out, so no window is opened).

Test plan

  • python3 -m unittest discover -s tests: 214 tests run, OK (209 before)
  • python3 -m py_compile main.py graphik.py __init__.py headless_run.py passes
  • Mutation check: with os.environ.setdefault("SDL_VIDEODRIVER", ...) replaced by plain assignment, test_drivers_that_are_already_set_are_left_in_place fails; with the pygame.get_init() check disabled, test_a_scenario_that_leaves_pygame_initialised_fails fails. Both mutations were reverted and headless_run.py is unchanged in the diff.
  • No README change is needed: the tests assert behaviour the README already describes.

Manual validation against a live Viron is not applicable: no rendering or Viron-service code is touched.

Deferred issues

All open issues were skipped this cycle, since each one is waiting on a decision by the maintainer and none can be implemented as specified yet:

No tracking issue — gap found during triage.

This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).

🤖 Generated with Claude Code


drafted by Claude on behalf of Daniel Stephenson

…faults

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmccoystephenson

dmccoystephenson commented Nov 15, 2026 •

Copy link
Copy Markdown
Member Author

Self-review rubric (the review was performed in the same session that wrote the change, so it is not independent):

  • Scope: PASS. Only tests/test_headless_run.py is modified (+75 lines). No production, README or batch-script changes.
  • Tests-new: PASS. No new public functions were added. Each of the five tests covers an existing behaviour of headless_run.runScenario or headless_run.run that had no test before.
  • Tests-fix: not applicable, since there is no bug fix. A mutation check was used in its place: with setdefault replaced by plain assignment, test_drivers_that_are_already_set_are_left_in_place failed, and with the pygame.get_init() check disabled, test_a_scenario_that_leaves_pygame_initialised_fails failed. The "no entry" and frame-count tests were not mutation-checked. Their assertions compare the whole FAIL ... output line exactly, so they would fail if that check were removed, but this was not run.
  • Sibling structure: PASS. The new classes follow the existing Test<Subject> naming and the setUp pattern with patch.dict(os.environ, ...) from TestRun, and use the camelCase helper names already used in the file (runHeadless).
  • Sibling renames: not applicable, since nothing is renamed.
  • Docs: PASS. The README's "Running without Docker or a display" section already lists the three failure checks and the "unless SDL_VIDEODRIVER or SDL_AUDIODRIVER is already set" rule. The tests match it, so no doc row changes.
  • Issue resolution: not applicable, since there is no Closes. The PR body records why every open issue was deferred.
  • Manual validation: PASS. The test workflow passed on the PR head (run 43679982149). Locally, python3 -m unittest discover -s tests ran 214 tests, all OK. Live-Viron validation does not apply because no rendering or service code is touched.

Findings:

  • tests/test_headless_run.py:188 — test_a_render_scenario_that_draws_no_frames_reports_the_expected_count repeats the stdout-capture code from the runScenario helper instead of passing the scenario name and exitAfterCreate to it. The duplication is small (three lines). It is left as it is, because changing the helper's signature for one caller adds little.
  • tests/test_headless_run.py:179 — the Pygame test patches get_init while the real main() runs. The real RenderWindow.close() still calls pygame.quit(), so Pygame is not actually left initialised after the test.

No blocking findings.

This review was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson
dmccoystephenson merged commit 022bc62 into main Nov 15, 2026
1 check passed
@dmccoystephenson
dmccoystephenson deleted the feature/characterize-headless-run-checks branch November 15, 2026 05:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant