gh-153668: Replace Unicode numeric switch with binary-search tables - #156081
Open
scorpi023 wants to merge 1 commit into
Open
gh-153668: Replace Unicode numeric switch with binary-search tables#156081scorpi023 wants to merge 1 commit into
scorpi023 wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #153668.
Summary
makeunicodedata.pycurrently emits_PyUnicode_ToNumeric()as a switchwith 2,023 case labels. On Windows ARM64, MSVC's
/O2optimizer consumesexcessive memory and the x86-hosted ARM64 compiler fails with C1002 while
compiling
Objects/unicodectype.c.Generate a sorted code-point table, a parallel value-index table, and a
table of the 151 UCD numeric-value entries instead.
_PyUnicode_ToNumeric()now uses binary search and still returns
-1.0for code points without anumeric value.
NUMERIC_MASKand_PyUnicode_IsNumeric()are unchanged.This is a source-level alternative to the temporary build workaround in
#153669. That PR disables optimization for affected MSVC toolsets; this
change keeps optimization enabled by removing the compiler-stressing
generated switch.
Windows ARM64 result
VCTools 14.51.36231, Release
/O2, on a Windows 11 ARM64 lab host:mainValidation
python -m test -j1 test_unicodedata test_str: 212 tests passed.python -m test -j1 -u cpu test_unicodedata: 73 tests passed.python -m test -j0 -x test_profiling: 49,207 tests passed across442 test files.
test_profilingwas excluded because its samplingcoordinator hangs independently on this lab image.
generated switch; all 2,023 numeric mappings and non-numeric results match.
Tools\patchcheck\patchcheck.py: passed.AI assistance: OpenAI Codex assisted with the investigation, implementation,
and validation. This PR is opened as a draft so the submitter can complete
the required human review before marking it ready.