Repository navigation
Reject non-identifier symbols on Python 3.12 (stacked on #80) - #81
Merged
Merged
Conversation
…er change)
Python 3.12's tokenize (PEP 701) emits characters such as '∞' as NAME tokens
instead of ERRORTOKEN, so parse_expr no longer fails on them and SymPy's
auto_symbol creates Symbol('∞'). Restore the Python 3.8 behaviour by raising
for any NAME token that is not a valid identifier; the existing parse-error
handling then reports PARSE_ERROR as before.
Fixes test_error_inappropriate_symbol[-∞--inf] on Python 3.12.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
m-messer
added a commit
that referenced
this pull request
Nov 10, 2026
* Upgrade to Python 3.12 base layer and align workflows with survey - Base image baseevalutionfunctionlayer:main-3.8 -> main-3.12 (AL2023: dnf git+findutils, PYTHONPATH, pytest for healthcheck, module-style CMD) - Rename evaluation_test.py/preview_test.py -> *_tests.py so the base layer healthcheck can import them - Workflows copied from survey: Python 3.12, deploy.yml@main, PR-triggered test-lint, database tests on production deploy; staging/production target master (default branch) - Add pytest.ini Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Bump sympy 1.10.1 -> 1.12.1 for Python 3.12 tokenizer compatibility sympy 1.10.1's parse_expr mishandles '!' (factorial) and some unicode tokens under Python 3.12's new tokenize module (PEP 701), failing test_exclamation_mark_for_factorial and test_error_inappropriate_symbol[-inf]. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep evaluation_test.py/preview_test.py names These are pytest-style test classes; under the *_tests.py names the base layer's unittest-based healthcheck imports and instantiates them and errors with 'TestEvaluationFunction() takes no arguments'. CI still collects them via pytest.ini. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Merge pull request #81 from lambda-feedback/fix-py312-invalid-identifier Reject non-identifier symbols on Python 3.12 (stacked on #80) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #80 (base:
python-3.12).Problem: on Python 3.12,
-∞withstrict_syntax: trueno longer returnsPARSE_ERROR. Prod on 3.8 still rejects it. This was the last failing test in #80 (test_error_inappropriate_symbol[-∞--inf]).Cause: Python 3.12's
tokenize(PEP 701) emits characters like∞asNAMEtokens. Python 3.8 emitted them asERRORTOKEN, which madeparse_exprfail. Checked with sympy 1.12.1 on Python 3.12.8:Fix:
parse_expressionnow tokenizes the expression first and raisesValueErrorfor anyNAMEtoken that isn't a valid identifier. The existingexceptinevaluation_functionturns that intoPARSE_ERROR, as before. Valid unicode identifiers (e.g. Greek letters) are unaffected.🤖 Generated with Claude Code