Skip to content

Add a guide for converting questions from the legacy grader - #38

Draft
eliotwrobson wants to merge 5 commits into
mainfrom
claude/conversion-guide
Draft

eliotwrobson wants to merge 5 commits into
mainfrom
claude/conversion-guide

Conversation

@eliotwrobson

@eliotwrobson eliotwrobson commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Splits the docs into two guides:

  • quick_start.md is for writing a question for the first time, and documents the full API. It starts with a complete example question, how to test it locally and where each kind of code belongs. After that come the reference sections: writing tests, checking answers without giving them away, giving values to student code, every ConfigObject setting, notebooks and workspaces.
  • converting.md is new and covers only converting a question from the legacy grader. It's meant to be handed to someone else's AI agent as a link. It covers when not to convert, how each legacy file and API maps to the new grader, moving code out of leading_code.py, trailing_code.py and setup_code.py, behavior differences, and how to test a conversion.

It also fixes things that were wrong in quick_start.md:

  • The list of default builtins.
  • The module-level timeout example, which used initialization_timeout. The plugin doesn't read that variable.
  • Examples that used bare assert, which shows students the expected value.

Testing

I ran the example question and the converted square question in the grader image with the docker run recipe from the guides. I also ran the local pytest layout, and checked the guides' claims about setup code, names_for_user, scoring, timeouts and trailing code.

The guides describe behavior from #39 (import_blacklist for functions) and #40 (more default builtins). Merge those first.

🤖 Generated with Claude Code

Also document leading and trailing code, query_setup and the limits on
setup code in the quick start, and fix the module-level timeout example,
which used a variable the plugin doesn't read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
eliotwrobson and others added 3 commits November 10, 2026 17:22
…verting

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sion-specific

quick_start.md now starts with a first question, local testing and where code belongs, then
the API reference. It fixes the list of default builtins and the examples that relied on
bare asserts. converting.md links to it instead of repeating general guidance, rewrites the
square example to avoid setup code, and calls out the restricted builtins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant