Skip to content

gh-156745: Inline CJK replacement writer call - #156744

Merged
corona10 merged 3 commits into
python:mainfrom
brittanyrey:b-perf-cjk-error-replace-inline
Sep 1, 2026
Merged

gh-156745: Inline CJK replacement writer call#156744
corona10 merged 3 commits into
python:mainfrom
brittanyrey:b-perf-cjk-error-replace-inline

Conversation

@brittanyrey

@brittanyrey brittanyrey commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Call _PyUnicodeWriter_WriteCharInline() instead of the python lib version of the method.
Removes the overhead of the boundary call between the .so and the exported function.

# decode dense invalid bytes with errors='replace'
bad = b"\x81\x40" + b"\xff"*4000
bad.decode('shift_jis', 'replace')
Benchmark base patched delta
replace, all-invalid (shift_jis) 34.06 µs 27.08 µs −20.5%
replace, all-invalid (gbk) 36.55 µs 28.20 µs −22.8%
replace, mixed valid/invalid 18.95 µs 16.85 µs −11.1%
valid input, no errors (control) 18.50 µs 18.39 µs −0.6% (noise)
ignore, all-invalid (control) 24.17 µs 24.16 µs 0.0% (noise)

@brittanyrey
brittanyrey requested a review from corona10 as a code owner August 31, 2026 20:19
@brittanyrey brittanyrey changed the title gh-NNNNNN: Inline CJK replacement writer call gh-156745: Inline CJK replacement writer call Aug 31, 2026
@brittanyrey
brittanyrey force-pushed the b-perf-cjk-error-replace-inline branch from 452b5b4 to d60dffc Compare August 31, 2026 21:40
@corona10 corona10 self-assigned this Sep 1, 2026

@corona10 corona10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@corona10
corona10 enabled auto-merge (squash) September 1, 2026 04:41
@corona10
corona10 merged commit eb087d4 into python:main Sep 1, 2026
58 checks passed
@brittanyrey
brittanyrey deleted the b-perf-cjk-error-replace-inline branch September 1, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance: Optimize cross boundary call for codec.decode() error replace calls

2 participants