Skip to content

Add criteria and process graph images to the developer documentation - #317

Merged
m-messer merged 2 commits into
mainfrom
docs/criteria-graph-images
Nov 3, 2026
Merged

m-messer merged 2 commits into
mainfrom
docs/criteria-graph-images

Conversation

@m-messer

@m-messer m-messer commented Nov 3, 2026

Copy link
Copy Markdown
Member

Summary

  • Criteria graphs (app/docs/criteria_graphs/): one example of each shape of criteria graph the contexts build (symbolic equality of expressions, equations and sets; where; written as; other; physical quantity matches and order). Each graph is built the way evaluation_function builds it and drawn twice: with graphviz straight from the graph's evaluation, criterion and output nodes, and with mermaid from CriteriaGraph.to_mermaid.
  • Process graphs (app/docs/process_graphs/): example inputs run through evaluation_function (symbolic, physical quantity, and a response that fails to parse). The stages come from the stage_start events _evaluate logs, and the function calls are recorded with sys.setprofile. Each example has two graphviz images:
    • the stages, with the criteria graph the response went through and its path highlighted
    • the app and lambdafeedback functions called in each stage, up to two calls below _evaluate
  • Generator scripts: scripts/generate_criteria_graphs.py and scripts/generate_process_graphs.py. No app code changes.
  • app/docs/dev.md: new "Criteria graphs" and "Process graphs" sections, with how to regenerate the images. The stray ======= line at the top is removed.

Known issue visible in the images

The physical quantity criteria graph details show QUANTITY: response tags: {}. This comes from f-strings of ExprNodes in app/context/physical_quantity/graphs/match.py and is not changed here.

Test plan

  • Both scripts run, and the SVGs were checked visually
  • The highlighted path matches the returned tags; the parse-error example stops after parse_reserved_expressions
  • flake8 --max-line-length=127 scripts/

🤖 Generated with Claude Code

m-messer and others added 2 commits November 3, 2026 12:32
scripts/generate_criteria_graphs.py builds the criteria graph for one
example of each graph shape, the same way evaluation_function does but
without generating feedback, and draws it twice: with graphviz straight
from the graph's nodes, and from CriteriaGraph.to_mermaid with
mermaid-cli. dev.md gets a "Criteria graphs" section that explains the
node kinds and shows each graph, with the mermaid version collapsed
under it.

Also removes a stray "=======" line at the top of dev.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each example is run through evaluation_function while recording the stages
it logs and the app and lambdafeedback functions it calls, and drawn with
graphviz: the stages with the criteria graph path taken, and the function
calls in each stage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@m-messer
m-messer added this pull request to stack #319 November 3, 2026 13:46
@m-messer
m-messer merged commit 45f54a3 into main Nov 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant